ARKit developer references

This weekend will be exploring a range of SDK examples. Began a spreadsheet to keep track of process. Will bullet out the info here from ARKits Apple Developer Page:

 

Helpful links:

 

Essentials

 

Camera ( Get details about a user’s iOS device, like its position and orientation in 3D space, and the camera’s video data and exposure. ) 

 

QuickLook ( The easiest way to add an AR experience to your app or website.)

  • Previewing a Model with AR Quick Look
  • class ARQuickLookPreviewItem
  • Adding Visual Effects in AR Quicklook

 

Display ( Create a full-featured AR experience using a view that handles the rendering for you)

  • class ARView

    A view that enables you to display an AR experience with RealityKit.

  • class ARSCNView

    A view that enables you to display an AR experience with SceneKit.

  • class ARSKView

    A view that enables you to display an AR experience with SpriteKit.

 

WorldTracking (Augment the environment surrounding the user, by tracking surfaces, images, objects, people, and faces.)

Discover supporting concepts, features, and best practices for building great AR experiences.

A configuration that monitors the iOS device’s position and orientation while enabling you to augment the environment that’s in front of the user.

A 2D surface that ARKit detects in the physical environment.

  • class ARWorldMap

    The space-mapping state and set of anchors from a world-tracking AR session.

 

 

 

 

 

Magic Windows: The Poetics of Augmented Space

Lev Manovich, The Poetics of Augmented Space // http://manovich.net/content/04-projects/034-the-poetics-of-augmented-space/31_article_2002.pdf

“Going beyond the ‘surface as electronic screen paradigm’, architects now have the opportunity to think of the material architecture that most usually preoccupies them and the new immaterial architecture of information flows within the physical structure as a whole. In short, I suggest that the design of electronically augmented space can be approached as an architectural problem. In other words, architects along with artists can take the next logical step to 28 28 consider the ‘invisible’ space of electronic data flows as substance rather than just as void – something that needs a structure, a politics, and a poetics. “

Magic Windows: Reading – Domesti/City – the dislocated home in augmented space

Domesti/City – the dislocated home in augmented space is a thesis paper by Keiichi Matsuda 

“Augmented space disrupts the long established dichotomies of public/private
and home/work embedded in the city, and calls for new terms to describe our
inhabitation of it. As mobile technology and wireless fields of presence envelop
the built environment, the electronomad is empowered to define her own use of
space and subjective reading of the augmented city.”

 

PT 2 – Class 3

Class 3: February 25

Small group reviews with Advisor and Residents

Prep for Midterms

Assignments:

Keep working and posting!!!

Prepare and practice your 5 minute midterm presentation. It should include: 

    • YOUR THESIS STATEMENT
    • WHY YOU CHOSE THIS TOPIC? WHY IS IT IMPORTANT / IMPORTANT TO YOU / IMPORTANT TO US? 
    • YOUR PROGRESS SO FAR
    • WHAT WILL YOU HAVE DONE BY THE END OF THE SEMESTER? 
    • (OPTIONAL BUT HIGHLY ENCOURAGED) A QUESTION OR TWO TO GUIDE YOUR FEEDBACK

Sign up for your presentation time 

Add your slides to this doc and check them in an incognito window before class if you have video!

Watch at least two thesis presentations: https://itp.nyu.edu/thesis2019/, https://itp.nyu.edu/thesis2018/, https://itp.nyu.edu/thesis2017/

Perhaps write a post about what lessons you learned that can apply to your own process!

 

Magic Windows: Reading – Mixed Reality Service

Mixed Reality Service (2016) by Mark Pesce – link to article

The article reiterates the importance of connectivity and collaboration of the web and the consequent  need for an open foundation of protocols that support connectivity for VR and MR. He then unpacks the importance of the Mixed Reality Service (MRS) which provides the ability “to map URIs to arbitrary geospatial or 3D coordinates”.

 

From the abstract:

The Mixed Reality Service (MRS) provides registration and discovery services binding the real world of geospatial coordinates to the virtual world of Universal Resource Identifiers (URIs). The MRS protocol consists of three commands: ‘add’ and ‘delete’, which allow additions and deletions to a mapping of geographical coordinates to URIs; plus ‘search’, which performs searches by geospatial coordinates, returning a list of matching mapped URIs. Potential uses of MRS include mixed reality applications, guidance for autonomous vehicles and drones, and vastly simplified delivery of nearly all location-based services. Simple modifications to MRS make it suitable for shared virtual worlds.”

 

 

Magic Windows: W5 – augmenting the urban space

news for the week:

 

 

========================================================================
Keiishi Matsuda – Hyper Reality
https://www.youtube.com/watch?v=YJg02ivYzSs
http://km.cx/projects/augmented-hyper-realityKeiishi Matsuda – Augmented City
https://vimeo.com/14294054Nexus Studios – HotStepper
https://nexusstudios.com/work/hotstepper/

Tom Ermitage,  Pan Studio and Gyorgyi Galik – Hello Lamp Post
https://tomarmitage.com/projects/hello-lamp-post/

Janet Cardiff and George Bures Miller – Alter Bahnhof Video Walk
https://www.youtube.com/watch?v=sOkQE7m31Pw

Timo Arnall – Immaterials: Light Painting WiFi
http://yourban.no/2011/03/07/making-immaterials-light-painting-wifi/
https://vimeo.com/20412632

Choy Ka Fai – Crossing Borders: a visualization of private spaces in public photography
https://vimeo.com/7918122

========================================================================

Hands-on:

Eyes-on:

Lev Manovich, The Poetics of Augmented Space
http://manovich.net/content/04-projects/034-the-poetics-of-augmented-space/31_article_2002.pdf

Keith Matsuda, ‘Domesti-city’
https://drive.google.com/open?id=1EVBmR_B3NOpnt-d0P7CCU_ldnWa5jOcx
http://km.cx/projects/domest-city

Mark Pesce, The Mixed Reality Service

Mobile Lab: HW4

For homework this week we were to play with the Mobile Lab Game Kit and to build out a visual metaphor with a possible win state. The organization of the codekit is to encourage us to structure our code with the root/main code with branches / separate swift files. An emphasis on:

  • App architecture (data flow) review
  • @State, @Binding, “Source of truth”
  • Metaphor
  • View Composition, Refactoring

 

IMG_2678 2.JPG

I was inspired by a book on our bookshelf The Cloud Collector’s handbook by the Cloud Appreciation Society. Hoping to build a iteration of the codekit where a spaceship or some avatar navigates through different tropospheric clouds:

  • cumulonimbus
  • cirrocumulus
  • altocumulus
  • cumulus

 

Troposhperic Cloud Classification IMG_2681 2

IMG_2697 3

IMG_2696

IMG_2694

giphy.gif

 

Quick References:

 

Magic Windows: HW4 – Augment an Object [ Exploring Teachable Machines: image & Audio classification ]

QuiltAR_step1 copy.jpg

Test 2: A quick speech test, filling in color of a leaf with your voice as you move through the words tulip – poplar .  link to p5js present mode

Test 1: A quick speech test, filling in color of a leaf with your voice as you move through the phonemes. tu-lip pop-lar

 

QulitAR_step2.jpg

 

References 

Teachable Machine 1: Image Classification 

  • how to save the model you train and bring it into a p5js sketch + ml5s library

 

Teachable Machine 2: Snake Game

  • flipping camera footage around

 

 

 

Questions

How to allow for a lower tolerance for speech accuracy? As well as audio sample time. (had to be pretty rigid with my syllables / record in short words, reads). Take a look at how the code was working inthis test last week to compare and contrast ❤

How to do voice/ speech commands using vuforia/ unity see office hours w Rui post. (thinking about Unity article & Dialogue flow)