Month: February 2020

AR Strengths / Weaknesses

bdmLeave a comment

https://www.youtube.com/watch?v=MNP77K9RsjIFor this week thesis teacher Sarah asked me to playtest as many AR apps as I could and see what their strengths and weaknesses were. Then to think which strengths make sense to design for into my mvp / first prototype. For this round I got to test out about 1o different ones and hope to keep exploring more:

 

David Bowie AR Exhibit 

Screen Shot 2020-02-10 at 3.41.56 PM.png

  • good range of media available (costumes, liner notes, videos, etc)
  • liked the map of gallery lay out / table of contents
  • good onboarding
  • audio levels of various media (narrator vs song clips) were surprisingly different / a little too jarring in difference?
  • reiterated the importance of mastering / unifying & double checking levels for possible use cases (ex: with earbuds vs over ear headphones)

 

Wonderscope

  • voice activation to move story forwards
    • encourages interaction with characters / makes the illusion feel more believable as a dialogue exchange
    • builds reading skills
  • some stories have difficulty with scale if immobile (like tucked in for bed)? but it is meant to encourage the user to move around
  • good on boarding to make sure the room is light enough and that the mic & camera permissions are on

                IMG_2501.PNG

augmented-reality-reading-app.jpg

  • Wonderscope Story – Red Riding Hood
    • uses the ability to go into spaces like entering the house with your phone moving through the door
    • like the others, can click on items that the character needs to move the story forward
      • has a visual indicator of sparkle particles that radiate vertically off the item to be found / clicked (helpful with
    • increases interactivity by having user read verb commands that then affect the animations like “spin” or “sprinkle”
    • liked the educational aspect of the bigdipper & the north star

 

  • Wonderscope Story – Sinclair Snakes – Museum Mischief
    • liked the use of different tool interactions
      • flashlight to search for hiding snake
      • duster to dust for footprints
      • how the ar museum rooms highlight

 

Big Bang in AR : Cern 

Weird Type – Zach Lieberman 

Starwalk

 

 

My Caterpillar 

Some nature apps (non ar)

Picture this

Seek  

 

Magic Windows: Vuforia & Image Targets

bdmLeave a comment

Getting Started

Step 1: create dev account @ https://developer.vuforia.com/

 

Adding personal target tests: embroidered ash leaves

Screen Shot 2020-02-10 at 2.53.50 PMScreen Shot 2020-02-10 at 2.53.37 PM

Step 2: image targets tutorial 

    • On the developer portal go to Develop/License Manager and generate a license (keep it handy!)
    • under Develop/Target Manager create image target and add your target Image(s). Please note its rating and features quality. Below info is pulled from the Vuforia Developer portal:
      • “Image Targets are detected based on natural features that are extracted from the target image and then compared at run time with features in the live camera image. The star rating of a target ranges between 1 and 5 stars; although targets with low rating (1 or 2 stars) can usually detect and track well. For best results, you should aim for targets with 4 or 5 stars. To create a trackable that is accurately detected, you should use images that are:”
        • Attribute Example
          Rich in detail Street-scene, group of people, collages and mixtures of items, or sport scenes
          Good contrast Has both bright and dark regions, is well lit, and not dull in brightness or color
          No repetitive patterns Grassy field, the front of a modern house with identical windows, and other regular grids and patterns
  • download img database package (as Unity Editor package)
  • make sure running Unity on a 2019.2.xx version
  • follow guides to set up on ios or android

 

 

Iterating off in class tutorial

Image Target class example

  • create new project (3D)
  • create new scene
    • Open build settings and make sure the right platform is selected
    • Open Player settings: make sure to add your package/bundle name, etc
    • Under XR settings: activate Vuforia
  • delete camera and directional light from the project hierarchy
  • add ARcamera
    • config Vuforia settings
      • add App License Key

 

  • under Assets/Import Package select Custom Package and select the Samples.unitypackage or your own target image database
  • add IMG target prefab to the hierarchy
    • make sure to select the right database
    • make sure to select the right image target
  • add a sphere primitive as a child of the image target (name it: ‘eye’)
    • scale xyz .2
    • pos (0.0, .148, 0.0)
    • create and apply white eye material
  • add second sphere as child of ‘eye’ and name it ‘pupil
    • scale xy z .25 .25 .1
    • pos 0, .012 .471
    • create and apply new black pupil material
  • create a new c# script,  name it ‘LookAtSomething’

public class LookAtMe : MonoBehaviour {

// transform position from the object we want to be looking at

public Transform target;

// Update is called once per frame

void Update () {

// use the coordinates to rotate this gameobject towards the target

this.transform.LookAt(target);

 

}

}

  • add the new script to the ‘eye’ GameObject in the hierarchy
  • make sure to link the ARCamera (what we want the eye to be looking at) to the target transform property
  • test! Move around and check if the eye is always looking at the camera
  • make a prefab out of ‘eye’
  • add any more necessary instances of the ‘eye’ prefab
  • Have fun!

 

 

 

Questions for Rui

  • module for mobile environment setup?

 

 

 

 

 

 

So you want to build an AR project…

bdmLeave a comment

This week we had residents Ilana and Terrick talk to us about building out AR/VR projects for thesis. Since I hope to explore the AR route potentially as well as for Magic Windows class wanted to note some of the points here too. Office hours this week with thesis advisor Sarah was also an assignment was two fold:

1: to play with and dissect as many AR apps as possible – what’s working and what’s not?

2: to stark making and that playing with quick AR creation tools like EyeJack will help get at what interactions for my quilt feel successful verses not.

We also talked about the target audience of kids and connecting them to local ecology through plant identification.

 

Think about an AR/VR project?

In the thesis presentation from the residents they raised some good consideration points. The importance of content and design, concept first and the tech to then help lift it up.

  • Why is AR the right medium for your project?
  • Content First and Design
  • Technology “second”
  • Understand what your project is about
    • consider – is it the right medium? if yes, why?
  • Look for references and understand what they did right/wrong
    • what platforms did they use? what is the scale of the project?

Some AR References

Anna Ridler – Wikileaks: A Love Story (2016)

IMG_6729

 

Zach Lieberman – Weird Type App (2019)

 

Planeta – David Bowie Is – AR exhibition App  (2018)

 

Wonderscope (2018)

 

 

What is your minimal MVP (Minimal Viable Product)?

  • What are your goals / MVP? 
    • Playing off of what we talked about in thesis class last week, always check back in with yourself about what your MVP is – A version of your final thesis output that has jus enough features to satisfy your benchmarks and provide feedback for future product development? Remember that the ITP thesis is more of a “Proof of Concept” prototype that you can then apply to grants, residencies, etc for incubation / final productions
  • Think about Content and Design, A lot! 
    • “According to what your project goal and MVP is, understand what is the right content to present, interactions you will need to create and aesthetics. This will guide you to decide on the tech.”
  • Sketch your experience and think about the user 
    • “When/where will people use it? How easy/hard should it be for people to access it?”
  • Think about the aesthetic you aim to create 

 

Advice for project builds

Terrick gave us a great breakdown of his learnings from his VR thesis and Ilana of her AR thesis. Both offered great advice for building out projects:

  1. Don’t underestimate the planning stage
  2. Users won’t always do what you want them to
  3. Photorealism isn’t always the best approach to creating believable experiences
  4. Spend ample time on sound

 

  1. Users are not used to have their mobile cameras as a ‘portal’. People still see it only as a docummenting tool. So keep in mind you will have to educate the user all the time.
  2. Create Gaze cues.
  3. Play with scale.
  4. Make a great Onboarding. Assume people won’t realize things by themselves and test.
  5. Understand that you have two spaces: the 2D UI and the 3D space. Design for both.
  6. Consider also that you have two spaces for gathering user input. Take advantage of that.
  7. Don’t forget about sound.

 

Technology

Define your project – Is it:

  • functional product
  • gaming
  • storytelling
  • experiment / artistic

 

PossibleAR tech: 

 

Tech Learnings 

From Ilana:

  1. Building takes time. A lot. With that the development process is super slow, mainly when you think about geolocation projects. So plan accordingly.
  2. To scale things accurately, you will be building in your device and testing all the time. A good tip is to build a system in the UI that will be only for your development process.
  3. If you are working with geolocation, don’t expect things to be in the exact coordinates in real space that you place them in your code. Our phone’s GPS is not accurate.
  4. This is a fast paced tech environment. Things change, versions get updated and sometimes, if you are working with different libraries and SDKs, at some point they may not work together anymore. So always have your MVP in mind and goals to prioritize.
  5. If you plan on putting your app on the AppStore, make sure to test it on multiple devices.
  6. It is a new medium and there is not a lot of people doing projects with this technology. Send e-mails, participate in communities and meetups. People will be excited to talk to you and share experiences.
  7. Just user test a lot. Don’t assume anything will work well until it does.

 

From Terrick:

  1. Not everyone knows what VR is. Onboarding is important!
  2. It’s very easy to cause motion sickness.
  3. Try to use existing assets when you can.
  4. Know the version of your project!

From both – User Test a Lot

 

Some AR tools:

Mobile Lab: W1 assignment – 1 button app

bdmLeave a comment

 

For week one we were to “Create a One Button app” using the one button code kit.  I  decided to make a “free museum access” app that lists all the museums we can get into free with our NYU student ID. It’s such a great benefit and am always forgetting! This helps me keep the list in one place but also as a visual suggestion of somewhere to go that day. The list is pulled from the post on the NYU site here.

After adding Images to the list I realized that the white text was hard to read so looked around and found a way to create a background box with opacity. However the background box is not responsive when you rotate the phone from portrait to landscape. Something to trouble shoot for in next steps.

I also tried it as a randomized list but enjoyed the more straightforward click through to help reiterate what the waived admissions options were better. Initially was thinking of a “What museum should I go to today?” randomized app and it reminded me just how often I forget about the free access we get with our student ids at certain ones.

Could see a next step iteration combining some of the things learned from the Swift UI lab with map location. Or having  the address/train stops in a Vstack below the museum name.

 

Questions:

  • how to outline text, like if using white text how to create a drop shadow or black outline to better separate from background image
  • how to have font color responsive if people select light or dark mode on their phones
  • best practices for crediting assets in app code
  • how to have the app restart from beginning when reopened, currently will start from last frame clicked

 

 

 

 

Magic Windows: HW 1.1 & 1.2

bdmLeave a comment

Hw – 1.1 Augment a Space 

For this assignment I knew I wanted to explore and learn a little of Madmapper. I’ve always heard about it but have never gotten to try – their free demo projects their logo so please ignore the large lettering ❤ I love projection mapping that highlights details of architectural spaces. In the lines of “if these walls could talk” thought about all the sounds and music the walls around me have absorbed over the years, including a music video collection for me when younger was The work of Directors: Spike Jonze, Chris Cunningham, and Michel Gondry. 

Several videos introduced me to idea of “augmenting” or hacking our perception and reality and felt like a great building block so to speak. Although this exploration isn’t necessarily dynamic, there is no human interaction, there still feels like a special moment of altering an every day surrounding, a brick wall in our apartment here. Could imagine making it interactive where tapping one of the blocks would have a ripple effect of some kind.

Original 5 music videos inspired by the Work of Directors series I grew up watching a lot that inspired the selection below:

81UqwhMV8WL._SL1500_.jpg

 

 

Hw 1.2 – Give a favorite AR/XR inspiration / example 

I’ve enjoyed playing with the Wonderscope app recently. Its use of voice activation to help the user feel more connected to the characters of the story while building reading skills felt compelling. With all voice assisted technologies and speech recognition it always leaves the question of which voice is it trained for? But felt like it was definitely a playful use of animation & ar. It also provides  a great onboarding process before a story unfolds.

 

Mobile Lab: Week 1 Labs

bdmLeave a comment

For this week we had two labs to get us familiar with Xcode and Swift.

 

Xcode lab

  • learned about different sections of Main.storyboard view
    • scene graph
      • shows hierarchy of components in your views
    • canvas
      • specific views that make up the app
    • utility area
      • multiple tabs that show information about the state of the user interface and a set of tools to modify it.
      • UI widgets to drag and drop into the canvas / can size and position by value
  • Object library (cmd+shift+L) or click + sign in top menu
    • Labels
      • blue dashed lines appear when text is centered
      • playing with custom font, size, and alignment fields
  • Safe Areas / Auto Layout Constraints
    • a new concept introduced in iOS11
    • allows for responsive interaction
    • safe area is basically the guaranteed area of the screen where content will not be blocked by other elements
      • ex: designing for the iphone notch
  • Running an app simulator
  • Running an app on hardware

 

 

Swift UI tutorial: Creating and Combining Views 

Screen Shot 2020-02-04 at 3.11.45 PMScreen Shot 2020-02-04 at 3.20.47 PMScreen Shot 2020-02-04 at 3.36.27 PM

 

Pt 2 – Class 1

bdmLeave a comment

Review/Update/ Implementation Schedule

Give a short(5 minute) review/update to us all about your project. Has anything changed? Did you do research over the holiday that changed or augmented your thinking?

You can use this as a template for your presentation.

 

Break into small groups for further discussion: What will you finish by the end of the semester? What are your goals? Look at the requirements for a thesis on the Thesis Journal so you can incorporate them into your schedule. Make an implementation/production schedule (help on the Resources Section of the Thesis Journal)

 

Assignment 0 – post your personal , rubric, matrix

 

Assignment 1 – due in two weeks

Your final project is due in 2 weeks! Just kidding, sort of.  

Bring the first prototype of your project to class. Now you are out of your head–make it. If it’s screen based, make a demo sketch of  it. If it’s an object, make it (cardboard is great). Try to focus on making something that has some *testable* aspect. 

 

Assignment 2 – due in one week  

Send Sarah + residents your broad-strokes implementation plan. We’ll  review these together in class!