Exploring quilt AR experience w Reality Composer

Screen Shot 2019-12-05 at 12.29.03 AM

Screen Shot 2020-04-14 at 11.49.25 AM

After office hours with Mobile lab and Thesis have decided to explore my project’s AR side with Reality Composer / Reality Kit + SwiftUI.

Since travel is limited could test with family in TN also by emailing them the image targets and testing the app via Testflight. Could test UI aspects with Sketch +InVision, and have a link that leads to a clickable playtest version ❤

This week now that some of the quilt patchwork and embroidered targets are done, revisiting the interaction and experience. It can be easy to fall down the rabbit hole of learning a new tool or software. A common question in past office hours whether with Nien, Sarah, Marina, Rui or Ilana has been less about how to make it work technically but to focus more on conceptual side and storytelling.

For W10 I plan to focus on 1 tree interaction then moving to the others. Focusing first on the maple tree on Fairfax.

Screen Shot 2020-04-14 at 11.49.36 AMScreen Shot 2020-04-14 at 11.49.04 AM

IMG_2985 2.jpeg

IMG_2989IMG_2997.jpeg

 

Thesis Update + Mobile lab / 4.15.20

Navigating Reality Composer

 

After office hours with Mobile lab and Thesis have decided to explore my project’s AR side with Reality Composer / Reality Kit + SwiftUI.

Since travel is limited could test with family in TN also by emailing them the image targets and testing the app via Testflight. Could test UI aspects with Sketch +InVision, and have a link that leads to a clickable playtest version ❤

This week now that some of the quilt patchwork and embroidered targets are done, revisiting the interaction and experience. It can be easy to fall down the rabbit hole of learning a new tool or software. A common question in past office hours whether with Nien, Sarah, Marina, Rui or Ilana has been less about how to make it work technically but to focus more on conceptual side and storytelling.

IMG_1041.pngIMG_1042.png

Office Hours (Mobile Lab 4.14.20)

Screen Shot 2020-04-14 at 11.49.36 AM.png

  • Instead of Vuforia – How to combine an Ios image to texture plugin with Reality Composer?
  • How to test a veryyy simplified mvp with family w testflight?
    • printing out ar targets on their end
  • had been looking at this sample code as a beginning point
    • but realized it was already using UIimagepicker as main function, might be easier to build out on swift side vs vuforia?

 

 

ExploreQuiltMode

Screen Shot 2020-04-14 at 11.49.25 AM

 

 

 

 

 

ExploreQuiltMode_0ExploreQuiltMode_01ExploreQuiltMode_01ExploreQUiltMode_02ExploreQuiltMode3

 

 

 

UIScreen Shot 2020-04-14 at 12.19.03 PM

IMG_2783

Screen Shot 2020-03-10 at 1.05.37 AM

sidebysideAsh

^ embroidery target test

 

giphy

^plugin test

 

 

UX

IMG_2541

 

 

 

Inspiration

augmented-reality-reading-app

 

Inspired by Wonderscope + Shanon Mattern’s Essay on Maintenance & Care

Thesis / building the top quilt

 

Screen Shot 2020-04-16 at 10.41.43 AM

To help me explore quilting have read through parts of Simple Geometric Quilting by Laura Preston of Vacilando Quilting Co.

Today was looking through the section ” Building a Quilt”. Looking for piecing tips to build out the pinwheel shape based on my great grandmother’s pinwheel. Will be using old sheets and pillowcases here in the apartment to create a cream & white pinwheel. In the user testing one of the favorite aspects was the filling in of color, to unlock color and images through the mobile device. That it helped tie the two together, creating a feeling of dependency between the two aspects vs two disparate things.

Piecing

  • considering the right & wrong side of the quilt
    • right side = top side of quilt
    • wrong side = backside of fabric
    • want stitches to be on back side
  • seam allowance
    • seam allowance for piecing  = 1/4″  (6mm)

 

Setting up sewing machine

last holiday i got a Janome Arctic Crystal beginner sewing machine,   with trying to stay home when we can to flatten the curve have set it up in our apartment. Newer to machine sewing I facetimed with my mom for a quick setup lesson. The instructions with the machine had forgotten to mention an essential step for winding a bobbin, and within minutes Bonnie was able to help set me up, with her camera view on her machine. It reminded me a little of an article from NYT about facetiming with family and friends in ways that are new for some of us.

 

 

Pinwheel Block History 

 

Misc References

 

Mobile Lab: W9

Screen Shot 2020-04-09 at 3.22.12 PM 

For my final app I plan to keep iterating on the Trees on My Block list app from midterm. Inspired by NYC Tree map site and its JSON set. I hope to translate some of the things that I love so much about the project into app form while also keeping it hyper local, one neighborhood block view at time. Would love for it to be a more fluid vs hardcoded experience so I could share the app with friends and family in NYC too. Things that I would like to try for:

 

2 Explore modes

Map View – using geolocation data see where you are on the map and trees you’re next to
Species List View- navigate to clickable list of species

Ability to update user location data + tree map load based on address input 

design/build to where a user could input an address and would load their block’s tree map. When address is entered, would load both their current location and your blocks tree map with corresponding tree species / based on the NYC Treemap Json.

 

A more intimate / less tutorial driven design aesthetic

 

Mobile Lab: W8

This week for hw we were to explore Reality Composer. Pick 3 separate areas (AR Health, AR Learning, AR Retail, AR Work and AR Entertainment) and design an app concept for that area.

 

AR Swatch helper (AR work / AR Retail)

When looking at possible knitting textile projects or a new pattern theres always the swatch gauge that is supposed to give you a guesstimate of how many stitches = a measured surface area based on needle size, material, and tightness of stitch. Being new to knitting I still forget the importance of making a swatch size and dive straight in realizing maybe its too small of a stitch project for me right now? Although hopefully that intuition will develop over time would love to have an ar app that could parse the swatch gauge measurement for me and generate one in real scale. Sometimes having a visual to scale in your everyday view is the most helpful : )

 

Gauge_Woes_Swatches

 

 

 

 

Object Xray Machine (AR Learning)

media-1049361-o-mouse-inside
from https://www.edn.com/under-the-hood-optical-wireless-mouse-rodentia-analogus/

Object Xray Machine is an educational AR app that overlays a “teardown” view of everyday electronic objects. Learn more about the behind the scenes parts that makeup our electronic tools.

This quick prototype uses an image recognition trigger of the mouse + area around it. Ideally would use object recognition so could be held in hand with overlay, not dependent on feature recognition targets outside the object itself for activation. When the camera recognizes the device like a mouse or phone, it would then reveal the motherboard & components within, with clickable components for more information.

 

https://vimeo.com/user89676923

 

AR plant care (AR Entertainment / Learning) – Object Tracking 

IMG_2892.jpeg

An AR app that logs & can remind you when to water your plant friends. Was able to snag a screenshot when it triggered correctly, but only worked for a quick second. Need to do a better object scan for object detection. Instead of a speech bubble could list last time watered and count down to next.

 

 

Future Ideabank:

  1. Untold Histories (AR Learning)
  2. Stretch for the stars (AR Health) 
  3. Object based radio generator (AR Entertainment)

 

Concepts to think on:

  • object tracking
  • screen triggers
  • tap triggers
  • scene transitions

 

Resources: 

 

 

Mobile Lab: AR Lecture

What is Augmented Reality 

  • this form of ar has been around for a while
  • ar as marketing tool  /
  • medical usage
  • gaming etc

 

 

Current state of AR / why are we seeing more AR 

  • scale of feasibility
  • increased accessibility of having AR in our pockets
  • first wave was a little cheesy now we’re getting into a deeper usage of AR
  • light ar sensor on new ipad

 

 

What makes AR possible?

  • Recent advancements in software & hardware
  • visual inertial odometry / use of camera sensors to detect sensors
  • feature points / comparing camera frames – what are the things that are constant & what are the things that are moving?
  • one thing ar does really well is measuring

 

 

Book Augmented Human 

 

Mobile Lab: Midtem List App

Screen Shot 2020-03-26 at 2.15.58 AMScreen Shot 2020-03-26 at 3.09.00 PM

Midterm Update

For midterm (W6&7) we were to create a list app that satisfied a certain set of criteria. Mine is a work in progress, but looking forward to continuing to work on it.

My app is called Tress On My Block, to help me get to know the trees on my block better building off of the NYC Street Tree Map data. You can see more about the Street Tree Map project in last week’s post.

 

Requirements:

  • Data model for app needs to be 2 or more levels deep
  • App needs to show list (vertical or horizontal scrolling) and allow navigation to detail
  • Must be able to input/update a data element
    • still needs the input / update aspect. would love the ability to collect photos of that treee over time or sketches of it? Using the Imagepicker code kit

 

 

Next/Future Steps:

  • implement image picker code kit to collect and add pictures of trees on block in real time to the detail pages
  • fix “about” text blocks
  • play with different table of contents
    • vector to image style
    • larger leaf size

 

 

 

 

With Text 

 

 

Without Text

 

 

 

 

 

 

 

Resources: