NYC Media Lab Future Imperfect: Open AI Firesidechat

Tuned in to NYC Media Lab’s Future Imperfect

Fireside chat w Greg Brockman of OpenAI

  • importance of being mindful of humanity what we are advancing in AI
  • GPT-3
  • Turing Test paper “Computing Machinery and Intelligence”
  • how to maximize the positive and minimize the negative, exploring how to release responsibly GPT-2 for use cases. how to see impact of use cases as they scale up and impact reality.
  • Cortex
  • Realization that powerful open source AI might be dangerous
  • Exploring use cases can be difficult which is why the community is so important

how do we mitigate bias and make sure that the systems are aligning with good

Book – what technology wants

AR Quilt

Screen Shot 2020-05-07 at 3.34.54 PM

Title: Memory Quilt: Lets Go For a Walk

Description:   Memory Quilt is an AR project for my nephews. Embroidered leaves of neighborhood trees found on the quilt become shifting repositories of memories, unlocked through a companion AR mobile app.


Screen Shot 2020-05-07 at 3.01.14 PM.png

Screen Shot 2020-05-07 at 3.02.43 PM
celebrating their routine walks, shared love of nature and how to share those memories with one another

Screen Shot 2020-05-07 at 3.02.38 PM.pngScreen Shot 2020-05-07 at 3.04.31 PM

User Testing 

Screen Shot 2020-04-14 at 11.49.25 AM

Some questions

  • How to share new and old memories through objects
  • How might generational storytelling inform social AR experiences
  • How to design digital experiences that encourage off screen exploration  
  • What does it mean to craft personalized apps, can it offer a certain kind of warmth
  • Possible aesthetic and interaction design decisions to better integrate the two

Some points of inspiration 

“by learning from a traditional medium, we’ve got a new technique  for drawing of computers whose value extends way beyond simply the automotive industry of tape drawing where it originated from and can benefit anybody putting geometry into a computer ”

SVA MFA interaction design student work from:

  • Abbot device for outdoor exploration paper
  • Bill Buxton’s tape drawing –> translation of tools / physical making to digital
  • SVA Interaction design MFA project (tree rings)
  • Embroidered landscapes from Museum in Akeyuri Iceland
  • The rise of digital naturalism and outdoor exploration apps

Notes from presentation w/ guest reviewers

What an amazing set of presentations from classmates! The reviewers also had such a wonderful and amazing set of insights for everyone.  Feeling so inspired by everyone. Here’s some feedback/takeaways:

Q’s = feedback from reviewers

A’s = solutions for next draft


Technical clarity  

Q: Unsure which quilts are being used/ mended. Is it one? is it all of them, any of them? 

A: Include a layered final quilt rendering / breakdown of influences. could be a visual timeline of how different family quilts influenced the design decisions or a gif of final quilt build


Q:  Whats the technical journey of the memory. Illustrate the mechanisms making it happen, from captured on mobile to discoverable through quilt to generated tap-able virtual leaves, each revealing a memory (technical demo). an illustrated pathway making the opaque black box transparent

ex: how does a maple leaf memory become digitally tied to image target. How does app know to generate virtual leaves and reveal the correctly associated memories.  Is it connected to a cloud database like Firebase to help with world saving?

A: (technical diagram, moving beyond the UX,UI. showing the data model, flow diagrams of how things are paired, the pathways from camera to swift ui image picker to reality composer to swift ui)


Q: Are the leaves tapped during augmented reality sequence real or virtual

using image of the real leaf in UX of AR/ Explore mode rendering was visually appealing, but confused how the app is used

A: have more explicit text/arrow that says “tap virtual leaves” right now it says just “leaves around you”.

Q: Is it limited to just photos? What kind of memories are being stored / range of media.

    • could it be a larger range of media? storytelling through quilts is an oral history tradition, would be nice to have more audio storytelling / sound and music incorporated. maybe video or gifs

A: have a mvp of images, but reiterate that ideally it would be the range, maybe even audio recording ability vs written with each memory that is stored.


Q: how many and how they then manifest as virtual leaves

          • How is world saving and file management addressed? (technical)
          • How does reality composer communicate with swift ui
        • GPS / Map of memory locations
        • digital background that makes sense?


Conceptual clarity 

        • continuing to pull the thread of looking at past ways of making and telling stories to inform new ones
  • wants to hear more stories like the nephew one woven throughout. 

Mobile Lab: W12




App Title: Memory Quilt / Lets go for a walk

Short description: Memory Quilt is an AR quilt project for my nephews. Created by mending a family quilt, it celebrates their family’s love for nature, time spent together, and how they might share those stories with one another. Embroidered leaves of neighborhood trees back in Tennessee found on the quilt become shifting repositories of memories, unlocked through a companion AR mobile app.

Update: For this week I focused on building a mockup of the “Add to Collection Mode” and tested through sketch remotely with my brother back in TN. Continuing to work on  the AR explore mode – thinking both about the interaction and balance of Screen/World space.



1st Sketch PrototypeScreen Shot 2020-04-24 at 1.03.14 AM

User tester 1  

  • would like ability to write a small something about the memory, what makes the moment unique
  • date & location, an ability to geotag


User tester 2 

  • to have the main buttons closer to where the thumb naturally lies, at the bottom of screen
  • to have it stay as an option on each leaf page in stead of a single screen asking whether to take new picture or from camera roll




2nd Sketch Prototype Screen Shot 2020-04-30 at 1.18.34 PM

User tester 3

  • ability to edit memory




Thesis References

Anderson , C., Miller, B., Eitel, K., Veletsianos, G., Eitel, J., & Hougham, R. (2015). Exploring Techniques for Integrating Mobile Technology into Field-Based Environmental Education. Electronic Journal of Science Education , 19(6). Retrieved from

Augmented Nature, 2019. (n.d.). Retrieved from

Brain, T. (2018). The Environment Is Not a System. A Peer-Reviewed Newspaper Journal About Research Values, 7(1), 152–165. Retrieved from

Barss, K. (2018). Outdoor Exploration Through Technology. Parks & Recreation , 53(9), 18–20. Retrieved from

Delprino, A. F., Piva, C., Tommasi, G., Gelsomini, M., Izzo, N., & Matera, M. (2018). ABBOT: a smart toy motivating children to become outdoor explorers. ACM Digital Library – AVI ’18 Proceedings of the 2018 International Conference on Advanced Visual Interfaces . Retrieved from

Empty Day. (n.d.). Retrieved from

Fessenden, M. (2015, October 8). Thanks to Augmented Reality, These Coloring Books Come to Life. Retrieved from

Gabrys, J. (2016). Re-Thingifying the Internet of Things.

Haptic Lab. (n.d.). Haptic Lab – Park Quilts. Retrieved from

Hougham, R. J., Nutter, M., & Graham, C. (2018). Bridging Natural and Digital Domains: Attitudes, Confidence, and Interest in Using Technology to Learn Outdoors. Journal of Experiential Education, 41(2), 154–169. doi: 10.1177/1053825917751203

House, B. (n.d.). Quotidian Record. Retrieved from .

Jessup, L. (2019). gARden cards – 2019 Reality Virtually Hackathon at the MIT Media Lab . Retrieved from .

Kelley, B. (2014, April 2). Anti-loneliness augmented quilt comforts children in hospital (Wired UK). Retrieved from .

Kimmerer, R. W. (2014). Braiding Sweetgrass: Indigenous Wisdom, Scientific Knowledge, and the Teachings of Plants. milkweed editions. doi: .

Mattern, S. (2018). Maintenance and Care. Places Journal, (2018). doi: 10.22269/181120 Retrieved from .

Morton, T. (2012). The ecological thought. Cambridge, MA: Harvard University Press.

Morton, T. (2014). Hyperobjects: philosophy and ecology after the end of the world. Minneapolis: University of Minnesota Press.

Nature-Cooper Hewitt Design Triennial. (n.d.). Retrieved from .

Odell, J. (2019). How to do nothing: resisting the attention economy. Brooklyn, London: Melville House.

Painted Earth: The Planet as our canvas. (n.d.). Retrieved from .

Rothman, J., & Niekrasz, J. (2015). Nature anatomy: the curious parts & pieces of the natural world. North Adams, MA: Storey Publishing.

Schwartz, C. (2019, April 18). Now Pay Attention, Dear Reader. No, Not to That Article, to This One. Retrieved from .

Simon, N. (2010). The participatory museum. California: Museum 2.0.

Simple Geometric Quilting: Modern, Minimalist Designs for Throws, Pillows, Wall Decor and More. (n.d.). Retrieved from .

Sculpture Cam. (n.d.). Retrieved from .

Seek by iNaturalist · iNaturalist. (n.d.). Retrieved from .

Solnit , R. (2001). Wanderlust. Penguin Books. doi: 

Weintraub, L. (2012). To life!: eco art in pursuit of a sustainable planet. Berkeley: University of California Press

Wonderscope – Wonderscope: an Augmented Reality Ios App For Kids. (n.d.). Retrieved from .


Explore Quilt AR mode: Scene flow brainstorm

Screen Shot 2019-12-05 at 12.29.03 AM


Screen Shot 2020-04-22 at 10.05.21 PM.png








  • The main question is how should an image appear that’s been tagged to the species?
  • hoping to start with the inspiration of this image, the act of picking up leaves from the ground to look at them closer
  • Might also be good to look at Adobe Aero



Some Influences

I’ve also been revisiting some projects that I feel influenced this one as well on a subconscious level.  An early Kimchi & Chips exploration as well as some really beautiful embroidered icelandic landscapes from the turn of the 20th century at the museum Safnasafnid in Aykeuri.



Explore Quilt Mode: building out custom behaviors 

Animated GIF

Image Anchor:  Embroidered Maple leaf  

  • falling leaves =  primitive shapes + physics enabled 
  • on tap behavior to pull up UI / tagged leaf  memory 




R&D – Quilt AR + Timeboxing

Today Im testing out a method Sarah suggested to us in thesis of Timeboxing. With timeboxing you choose a certain time frame for the completion of a specific task, pencils down on that task no matter what when the time ends, no judgement on quality this round.

Since my mobile lab final is my thesis app/ar side, hoping to apply this technique to get me back to where i hope to be timeline wise. Need to do last weeks W9 & W10’s assignment. After office hours it sounds like focusing this week on the R&D will be most helpful.


11:15pm – 12:15pm

Beginning this next hour+ with a brainstorming activity on the similarities, differences and important aspects for two main parts of project: quilts & trees:

Brainstorm on Trees 20 min

Brainstorm on Quilts 20 min

Brainstorm on how possible interactions / experience paths 20 min



Prototype with reality composer 

12:45pm – 1:45pm

Choose an interaction

What are technical limitations to consider? Or questions that come to mind?



Past brainstorms to look at as well:

  • From inspiration to ideas
  • thesis freewrite
  • ar lotus flower
  • user tests