Category: Thesis

Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms  by Hiroshi Ishii and Brygg Ullmer @ MIT Medialab / Tangible Media Group. It is not a proposed solution but hopes to “raise a new set of research questions to go beyond the GUI (graphic user interface)”

In the paper they break down 3 essential ideas of Tangible bits:

• interactive surfaces 
• coupling of bits with graspable physical objects
• ambient media for background awareness 

Using 3 prototypes for illustration:

• metaDESK
• transBOARD
• ambientROOM

Tools throughout history

It begins with the reflection on how before computers people created a rich and inspiring range of objects that measure the passage of time, predict the planets movements, to compute and draw shapes. A lot of them are made of a range of beautiful materials from oak to brass( such as items in the Collection of Historical Scientific Instruments).

” We were inspired by the aesthetics and rich affordances of these historical scientific instruments, most of which have disappeared from schools, laboratories, and design studios and have been replaced with the most general of appliances: personal computers. Through grasping and manipulating these instruments, users of the past must have developed rich languages and cultures which valued haptic interaction with real physical objects. Alas, much of this richness has been lost to the rapid flood of digital technologies.”

Bits & Atoms 

What has been lost in the advent of personal computing? How can back physicality and its benefits to HCI. This question brought them to the thought of Bits & Atoms. That “we live between two realms: our physical environment and cyberspace. Despite our dual citizenship, the absence of seamless couplings between these parallel existences leaves a great divide between the worlds of bits and atoms”. Yet we’re expected at times to be in both worlds simultaneously, why do our interfaces not reflect or provide this bridging(thinking at the time of the published paper)?

Haptic & Peripheral Senses 

Weather changes can be swift, Campeche Bay, Mexico.

They mention that we have created ways of processing info through working with physical things like Post it notes.  Or how we might be able to sense a change of weather through a shift in ambient light. However these methods are not always folded into the developing of HCI design. That there needs to be a more diverse range of input/output media – currently there is “too much bias towards graphical output at the expense of input from the real world”

From Desktop to Physical Environment: GUI to TUI

Xerox Star workstation laid the foundation for the first generation of GUI, a desktop metaphor,

https://youtu.be/zVw86emu-K0?t=220

” The Xerox Star (1981) workstation set the stage for the first generation of GUI, establishing a “desktop metaphor” which simulates a desktop on a bit-mapped screen. Also set several important HCI design principles, such as “seeing and pointing vs remembering and typing,” and “what you see is what you get.”

Apple then brought this style of HCI to the public in 1984. Pervasive through Windows and more. In 1991 Mark Weiser published “Ubiquitous Computing.”

Keywords tangible user interface, ambient media, graspable user interface, augmented reality, ubiquitous computing, center and periphery, foreground and background. It showed a different style of computing / HCI that pushes for computers to be invisible. Inspired by this paper they seek to establish a new type of HCI called “TUIs”. Tangible User Interfaces, augmenting the real world around us by pairing with digital information to everyday things and spaces.

Bricks: Laying the Foundations for Graspable User Interfaces

• • series of exploratory studies on hand gestures / grasping
  • interaction simulations using mockups / rapid prototyping tool
  • working prototype and sample application called GraspDraw
  • initial integration of graspable UI concepts into a commercial application

What does it mean to augment objects, both technically and conceptually ?

This week Rui brought to focus up a few thinking points as well as walked us through a vuforia tutorial.  Class began thinking about the philosophy of animism as a powerful approach to thinking about object augmentation.

“Animism is the belief that objects, places and creatures all possess a distinct spiritual essence.  Potentially animism perceives all things — animals, rocks, rivers, weather systems, human handiwork, and perhaps even words as animated and alive (from wiki)”. 

Also thinking about developmental psychology theorist Piaget and how kids are egocentric in that for a set amount of years tend to only have themselves as a reference, as a way they experience the world, in and in that thinking embody objects as a relational exercise. As an example how sometimes instead of we got lost we say the car got lost when really it was a combination of us, our gps etc.

Rui challenged us to move beyond thinking about projection, past the hologram assistant.

He also talked about how to augment objects as an interface. How to extend the current tool and its current functionality? Can a waterbottle also tell you about the amount of water drunk over the day? How to invoke, look at the form and materiality of things and how to blend its form into the ar space / more naturally tied

He then brought up to the sensation of Pareidolia ,   seeing faces in everyday objects. How the brain reads for easy patterns sometimes and fills in the dots. A quick assesment as a signal to noice mechanic, reaching conclusions that aren’t always correct like an outlet being a face. Or Rui’s parallel example of hearing Portugese vowel sounds when listening to Russian. Think also about how computer vision / facial recognition works too.

Then we walked through a bunch of fun examples of augmenting objects.

Suwappu – Berg London

Garden Friends – Nicole He

Tape Drawing – Bill Buxton

https://www.billbuxton.com/tapeDrawing.htm

Smarter Objects

• https://dspace.mit.edu/handle/1721.1/91844
• https://www.core77.com/posts/24829/mit-media-lab-fluid-interfaces-groups-smarter-objects-interface-design-24829

Invoked computing

• http://www.k2.t.u-tokyo.ac.jp/perception/invokedComputing/
• https://www.wired.com/2011/11/augmented-reality-university-of-tokyo-invoked-computing/

MARIO

https://www.youtube.com/watch?v=SpshRCmlX5Yhttp://hanyuool.com/projects/mario/?ckattempt=3

• http://hanyuool.com/projects/mario/?ckattempt=3

Paper Cubes – Anna Fusté, Judth Amores

• https://experiments.withgoogle.com/paper-cubes

InForm

• https://tangible.media.mit.edu/project/inform/

when objects dream – ECAL

• https://www.dezeen.com/2016/04/12/ecal-students-when-objects-dream-exhibition-future-everyday-objects-virtual-reality-connected-milan-design-week-2016/

HW for next week:

1. Eyes on
   Pls read at least one of the following:
   • Developing Augmented Objects: A Process Perspective
   • Bricks: Laying the Foundations for Graspable User Interfaces
     • Video
   • Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms

   Extra reading: Radical Atoms
   it is a nice pairing with Ivan Sutherland’s ‘The Ultimate Display’

    

2. Hands on
   Write a brief paragraph about your idea on augmenting an object.
   This should be initial concept for next week’s assignment.
   Be clear about your idea, inspirations and potential execution plan.
   
   
3. Brain on
   List potential input (sensory) and output( feedback) modes that you can leverage in the real world – translate these into technical possibilities using current technology (you can use your mobile device but feel free to bring any other technology or platform (Arduino? Etc..) that you can implement.
   Choose on elf each (input, output) and create a simple experience to show off their properties but, also, affordances and constraints.
   

Getting Started

Step 1: create dev account @ https://developer.vuforia.com/

Adding personal target tests: embroidered ash leaves

Step 2: image targets tutorial 

• • eyes on Image Targets documentation: https://library.vuforia.com/content/vuforia-library/en/articles/Training/Image-Target-Guide.html

• • On the developer portal go to Develop/License Manager and generate a license (keep it handy!)

• • under Develop/Target Manager create image target and add your target Image(s). Please note its rating and features quality. Below info is pulled from the Vuforia Developer portal:
    • “Image Targets are detected based on natural features that are extracted from the target image and then compared at run time with features in the live camera image. The star rating of a target ranges between 1 and 5 stars; although targets with low rating (1 or 2 stars) can usually detect and track well. For best results, you should aim for targets with 4 or 5 stars. To create a trackable that is accurately detected, you should use images that are:”

      • Attribute	Example
        Rich in detail	Street-scene, group of people, collages and mixtures of items, or sport scenes
        Good contrast	Has both bright and dark regions, is well lit, and not dull in brightness or color
        No repetitive patterns	Grassy field, the front of a modern house with identical windows, and other regular grids and patterns

• download img database package (as Unity Editor package)
• make sure running Unity on a 2019.2.xx version
• follow guides to set up on ios or android

Iterating off in class tutorial

Image Target class example

• create new project (3D)

• create new scene
  • Open build settings and make sure the right platform is selected
  • Open Player settings: make sure to add your package/bundle name, etc
  • Under XR settings: activate Vuforia

• delete camera and directional light from the project hierarchy
• add ARcamera
  • config Vuforia settings
    • add App License Key

• under Assets/Import Package select Custom Package and select the Samples.unitypackage or your own target image database

• add IMG target prefab to the hierarchy
  • make sure to select the right database
  • make sure to select the right image target

• add a sphere primitive as a child of the image target (name it: ‘eye’)
  • scale xyz .2
  • pos (0.0, .148, 0.0)
  • create and apply white eye material

• add second sphere as child of ‘eye’ and name it ‘pupil
  • scale xy z .25 .25 .1
  • pos 0, .012 .471
  • create and apply new black pupil material

• create a new c# script,  name it ‘LookAtSomething’

public class LookAtMe : MonoBehaviour {

// transform position from the object we want to be looking at

public Transform target;

// Update is called once per frame

void Update () {

// use the coordinates to rotate this gameobject towards the target

this.transform.LookAt(target);

}

}

• add the new script to the ‘eye’ GameObject in the hierarchy
• make sure to link the ARCamera (what we want the eye to be looking at) to the target transform property

• test! Move around and check if the eye is always looking at the camera

• make a prefab out of ‘eye’

• add any more necessary instances of the ‘eye’ prefab

• Have fun!

Questions for Rui

• module for mobile environment setup?