Mobile Lab W5: HW

Requirements and Tips:
  • Views should be no larger than 400×400
  • If your views require image/audio/video assets, please send those along as well.
  • Be creative with your design. Think about how standard buttons and sliders can be mapped in fun and novel ways.
  • Consider using a timer and/or randomizer to modulate the signal over time.
  • You may or may not require some internal @State for your component views based on their complexity.
(edited)
___________________________________________________________________________________________
Main Test Code will look something like”
import SwiftUIstruct Signal {
    // Range: 0 - 10
    var intValue: Int    // Range: 0 - 1.0
    var floatValue: Float    // True or False
    var toggleValue: Bool
}struct ContentView: View {
    @State var signal = Signal(intValue: 0, floatValue: 0, toggleValue: false)    var body: some View {
        VStack {
            NiensController(signal: $signal)            Spacer()            NiensVisualizer(signal: signal)
        }
    }
}
  • A controller file with your name. e.g. NiensController.swift
struct NiensController: View {
    @Binding var signal: Signal    var body: some View {
        //
        // Add your buttons, knobs, etc. here.
        // Update signal appropriately.
        //        Text("Controller")
    }
}
  • A visualizer file with your name. e.g. NiensVisualizer.swift
struct NiensVisualizer: View {
    var signal: Signal    var body: some View {
        //
        // Create visuals/animations here.
        //        Text("Visualizer")
    }
}

3.4.20 – Understanding ARfoundation Image Tracking

After working through some examples / tutorials last night, I decided to sift back through the AR Tracked Image Manager documentation to see about the following:

  •  multiple targets via an XRReferenceImageLibrary 
    • encountered issues when sifting through the ARfoundation example via GitHub ❤ mostly worked though! Was having trouble showing the last 3 I added
  • dynamic + modifiable / mutable libraries in runtime 
    • how to dynamically change the image library live via ios camera or camera roll  (most likely through a MutableRuntimeReferenceImageLibrary)

 

 

Helpful References:

 

 

Looking through the ARfoundation Trackables / ImageTracking Documentation: 

  • AR Tracked Image Manager 
    • The tracked image manager will create GameObjects for each detected image in the environment. Before an image can be detected, the manager must be instructed to look for a set of reference images compiled into a reference image library. Only images in this library will be detected

 

  • Reference Library  
    • XRReferenceImageLibrary
    • RuntimeReferenceImageLibrary
      • RuntimeReferenceImageLibrary is the runtime representation of an XRReferenceImageLibrary
      • You can create a RuntimeReferenceImageLibrary from an XRReferenceImageLibrary with the ARTrackedImageManager.CreateRuntimeLibrary method

Coding Lab – 3.2.20

Today I went to Coding Lab to troubleshoot the logic roadblocks I was having with last weeks homework. Vince helped walk me through how @State & @Bindings were working in the Mobile Lab Game Kit.

We specifically walked through how avatarPosition was working across views since its needed across all layers. He mentioned how part of the magic that SwiftUI allows us with this system is that when the code goes to $avatarPosition to update the state it won’t get stuck in an infinite / recursive loop.

We then took a look at my app concept of navigating over clouds to reveal cloud names. he suggested approaching it in 3 steps:

  1.  collisionObject 
    1. that you would need to change the let to a var to allow for flexibility in the target size / shape. With the change from the finite let to the versatile var, we can now target that the size variable would need to become 2 different variables: sizeHeight & sizeWidth since in the example the trigger is a square and only needed one dimension repeated twice.
  2. contentView 
    1. Vince then walked me through the simple logic to toggle the background (shifting the color), and that the logic would be the same for switching out images. He created a helper function that is an if/else statement and then called it in the view.  He also reiterated how you cannot declare a function inside a view, that view only wants to deal with things that make instant changes to the canvas. So where calling a function in view would shift the canvas, the process of declaring it doesn’t return anything, therefore would return an error if in view.
  3. Thinking about restrictions in position ( >&< ) logic to create rollovers  
    1. that in the example we did in coding lab, we only restricted in 1 direction, in order to achieve a rollover would need to restrict in all directions

Mobile Lab W5: Cheat Sheet + Signal/Visualizer

This week we were to review the Cheat Sheet made by Nien & Sebastian. As well as create 1 controller that outputs a signal and 1 visualizer that processes the signal. “You can think of the controller as something with buttons, knobs, etc. and the visualizer will be like a processing sketch which updates based on the signal.”

Inspired by things like this:

 

Code Examples: 

The signal will have 3 components contained in a struct. Note the allowed ranges:

struct Signal {
    // Range: 0 - 10
    var intValue: Int    // Range: 0 - 1.0
    var floatValue: Float    // True or False
    var toggleValue: Bool
}

 

The Main Tester code will look somethings like this:

import SwiftUIstruct Signal {
    // Range: 0 - 10
    var intValue: Int    // Range: 0 - 1.0
    var floatValue: Float    // True or False
    var toggleValue: Bool
}struct ContentView: View {
    @State var signal = Signal(intValue: 0, floatValue: 0, toggleValue: false)    var body: some View {
        VStack {
            NiensController(signal: $signal)            Spacer()            NiensVisualizer(signal: signal)
        }
        .frame(height: 700)
    }
}

 

 

Deliverables: 

Before the start of next class, please Slack Sebastian and I directly with your 2 files and any required assets. We will take everyone’s controller and visualizer and wire it up to a master app allowing us to mix and match everyone’s components.

 

A controller file with your name. e.g. NiensController.swift

struct NiensController: View {
    @Binding var signal: Signal    var body: some View {
        //
        // Add your buttons, knobs, etc. here.
        // Update signal appropriately.
        //        Text("Controller")
    }
}

 

A visualizer file with your name. e.g. NiensVisualizer.swift

struct NiensVisualizer: View {
    var signal: Signal    var body: some View {
        //
        // Create visuals/animations here.
        //        Text("Visualizer")
    }
}

Some requirements and tips:
  • Views should be no larger than 400×400
  • If your views require image/audio/video assets, please send those along as well.
  • Be creative with your design. Think about how standard buttons and sliders can be mapped in fun and novel ways.
  • Consider using a timer and/or randomizer to modulate the signal over time.
  • You may or may not require some internal @State for your component views based on their complexity.

 

 

Cheat Sheet Concepts 

Data Flow 
– think of your Views as a function of State
– how your views express the relationship between your application’s data / info and how your user interface looks and behaves.
– Understanding how data flows through a view hierarchy is fundamental to understanding app development with swiftui
Data Flow w/ External Events
ex: Timer, CMMotionManager, URLRequest,NotificationCenter, ARSession
(how its not user interaction but a programmed event that becomes the action in the cycle that changes the State. )
@State and @Binding 
“@” prefix indicates that they are property wrappers
–> provides your properties with special behaviors
when the property value’s changes, the view automatically recalculates its appearance / body

ARKit developer references

This weekend will be exploring a range of SDK examples. Began a spreadsheet to keep track of process. Will bullet out the info here from ARKits Apple Developer Page:

 

Helpful links:

 

Essentials

 

Camera ( Get details about a user’s iOS device, like its position and orientation in 3D space, and the camera’s video data and exposure. ) 

 

QuickLook ( The easiest way to add an AR experience to your app or website.)

  • Previewing a Model with AR Quick Look
  • class ARQuickLookPreviewItem
  • Adding Visual Effects in AR Quicklook

 

Display ( Create a full-featured AR experience using a view that handles the rendering for you)

  • class ARView

    A view that enables you to display an AR experience with RealityKit.

  • class ARSCNView

    A view that enables you to display an AR experience with SceneKit.

  • class ARSKView

    A view that enables you to display an AR experience with SpriteKit.

 

WorldTracking (Augment the environment surrounding the user, by tracking surfaces, images, objects, people, and faces.)

Discover supporting concepts, features, and best practices for building great AR experiences.

A configuration that monitors the iOS device’s position and orientation while enabling you to augment the environment that’s in front of the user.

A 2D surface that ARKit detects in the physical environment.

  • class ARWorldMap

    The space-mapping state and set of anchors from a world-tracking AR session.

 

 

 

 

 

Magic Windows: The Poetics of Augmented Space

Lev Manovich, The Poetics of Augmented Space // http://manovich.net/content/04-projects/034-the-poetics-of-augmented-space/31_article_2002.pdf

“Going beyond the ‘surface as electronic screen paradigm’, architects now have the opportunity to think of the material architecture that most usually preoccupies them and the new immaterial architecture of information flows within the physical structure as a whole. In short, I suggest that the design of electronically augmented space can be approached as an architectural problem. In other words, architects along with artists can take the next logical step to 28 28 consider the ‘invisible’ space of electronic data flows as substance rather than just as void – something that needs a structure, a politics, and a poetics. “