Mobile Lab: AR Lecture

What is Augmented Reality 

  • this form of ar has been around for a while
  • ar as marketing tool  /
  • medical usage
  • gaming etc



Current state of AR / why are we seeing more AR 

  • scale of feasibility
  • increased accessibility of having AR in our pockets
  • first wave was a little cheesy now we’re getting into a deeper usage of AR
  • light ar sensor on new ipad



What makes AR possible?

  • Recent advancements in software & hardware
  • visual inertial odometry / use of camera sensors to detect sensors
  • feature points / comparing camera frames – what are the things that are constant & what are the things that are moving?
  • one thing ar does really well is measuring



Book Augmented Human 


Mobile Lab: Midtem List App

Screen Shot 2020-03-26 at 2.15.58 AMScreen Shot 2020-03-26 at 3.09.00 PM

Midterm Update

For midterm (W6&7) we were to create a list app that satisfied a certain set of criteria. Mine is a work in progress, but looking forward to continuing to work on it.

My app is called Tress On My Block, to help me get to know the trees on my block better building off of the NYC Street Tree Map data. You can see more about the Street Tree Map project in last week’s post.



  • Data model for app needs to be 2 or more levels deep
  • App needs to show list (vertical or horizontal scrolling) and allow navigation to detail
  • Must be able to input/update a data element
    • still needs the input / update aspect. would love the ability to collect photos of that treee over time or sketches of it? Using the Imagepicker code kit



Next/Future Steps:

  • implement image picker code kit to collect and add pictures of trees on block in real time to the detail pages
  • fix “about” text blocks
  • play with different table of contents
    • vector to image style
    • larger leaf size





With Text 



Without Text












Quilt Materials

Due to Covid 19 i’ve placed visiting the Garment District here on hold and have ordered some items through etsy. I also took a look at online shops from places from the Garment District for future possible purchases. With needing to maybe keep things a little more tight financially also looking to see what could be sourced as material at home from clothes, sheets or pillow cases : )  More soon ❤

Thesis: Office Hours 3.10 / Marina Zurkow


Somethings mentioned:

  • to spend some time deep thinking, writing on brainstorming around the qualities, materialities of trees and quilts, patterns of mending and the lifespan and break down of materiality of both
  • The Overstory by R. Powers –  trees as a narrative mechanic symbol and character in storytelling and poetry
  • Beryl Korot’s weaving
  • History of making and thinking through textile art
  • The idea of artifacts, the quilt as an artifact, as a map? as a protective layer of comfort and warmth
  • how does this idea of repair and mending return to AR?
    • thinking about our first lecture in magic windows


Adapting Plugin for AR

Looking at this plugin and hope to adjust it for AR ❤

Will instead stage it with Vuforia or ARfoundation image targets if can’t get it to quite work


Move to an AR session & AR session origin

  • so that you can see the camera feed live behind the current game object
  • try for 1 image, and then adjust for more gameObjects / texture loads later




Looking at Original Plugin

Items to consider how they’re functioning in Full Sample Scene:

  • Cube
  • Main Camera
  • Point Light
  • Assets


Items in Assets 

  • Sample Scene (Full)
  • Unity Full Sample scene
  • Materials
  • Textures
  • C# Script – ChangePhoto


CubeScreen Shot 2020-03-11 at 10.08.04 PM.png

  • Transform = slight rotation allows us to see that its a 3d shape from the camera perspective
  • Cube (Mesh Filter) =
  • Box Collider
    • trigger – unchecked
    • material – no physic material
  • Mesh Renderer
    • Materials
      • size 1
      • Element 0 = PhotoTexture
    • Lighting
      • Cast shadows = on
      • Receive shadows = checked
    • Probes
      • light probes = off
      • reflection probes = blend probes
      • anchor override = none (transform)
    • Additional Settings
      • motion vectors = per object motion
  • Change Photo Script
    • Target Material = photo texture
    • Default Texture = photo texture
    • Overlay Texture = cameraOverlay Texture
    • Is Save To Local file = checked
    • photo Texture = legacy shaders / diffuse
    • Main Color
      • select image


CameraScreen Shot 2020-03-11 at 10.09.03 PM.png

Main Camera

tag = main camera / layer = default







  • Plugins (look at more closely?)
    • IOS ->
      • ImagePickerLib
        • Assets/Plugins/iOS/ImagePickerLib.h
      • ImagePickerLib
        • Assets/Plugins/iOS/
      • ImagePickerUtil
        • Assets/Plugins/iOS/


Tried adding in new ARfoundation Packages:

  • AR subsystems (3.0.0)
  • ARkit XR Plugin (3.0.1)
  • AR Foundation (3.0.1)

Magic Windows: Final

Little sleepy plant friends ❤  For this project / sketch I kept playing with AR Foundations place multiple objects example. Thinking of our class conversation on animism and some of the examples shown in the class lecture on augmenting objects.  Would be better as an object / image tracking build.




Ideal Functionality: 

When you tap a plant a dialogue bubble pops up and its facial emotion changes.

Also would like to try instantiating the same object with different textures, similar to what Hayk showed us in class last week with his Augmenting the urban space. That way there could be a linear dialogue being clicked through, or could toggle from asleep to awake? Would also like to play with transparency layers


Current Functionality: 

Right now its more like a sticker, where you tap the screen a little below the pot to have the prefabs appear ( a zzz dialogue box) and set of sleeping eyes). Since the angle is based on where your phone is at time of ar session launch, the stickers wont always be at the right perspective when used.  Would be better to have it centered around object recognition? Have had a couple issues with using images as materials? They seem to be getting reversed. I think I also left the UI example on that helps guide user through movement for better plane detection.



Misc Quilt Tests 

Also played around some with images & stitching appear on the quilt as a simple test. This also uses the AR foundation Place multiple objects tutorial. Would like to try it for image targets (either the AR foundation example or starting with Vuforia for tests). For Some reason the textures were wrapping upside down? Need to take a closer look and make sure i didnt get lost in my XYZ system



ExploreQuiltMode_01ExploreQUiltMode_02ExploreQuiltMode3Screen Shot 2020-03-10 at 1.05.37 AM