Augmented Environments Lab

AR SPOT is the first augmented-reality authoring environment for children. An extension of MIT’s Scratch project, this environment allows children to create experiences that mix real and virtual elements. Children can display virtual objects on a real-world scene observed through a video camera, and they can control the virtual world through interactions between physical objects. This project aims to expand the range of creative experiences for young authors, by presenting AR technology in ways appropriate for this audience. In this process, we investigate how young children conceptualize augmented reality experiences, and shape the authoring environment according to this knowledge.

The development process is one of iterative user-centered design:
First, we perform experimental studies to determine how people conceptualize mixed-reality environments. Then, we use this knowledge to create authoring tools which allow the generation of more experiences. Finally, we repeat the process to provide better tools while we learn more about users.

Details

The source code for Scratch was modified to include a camera feed, and novel functions were added to the library of programming blocks. The video feed is processed by an external DLL which detects and tracks special objects in the physical environment. Information about the physical objects is then accessible through the programming blocks.

Users can interact through two types of objects: cards and knobs. The system tracks the square marker patterns in order to detect position and orientation of the objects.

There are three ways in which actors can be rendered in the physical scene. One of these preserves the 2D dimensionality of the existing system, by having actors simply follow the position of the object on the screen (if physical objects are moved, the actors match the object 2D position on the screen; however, the actors do not change size or orientation if the objects are distanced or rotated). In two other cases, the actors become more 3D – they can lay or stand on the physical objects, and their appearance changes to follow the 3D location of the object.

New blocks were added to the programming library to make these interactions possible. Motion blocks were added to tell sprites to stick/unstick to the physical objects. Sensing blocks were added for detecting object properties such as distance, rotation, touch, etc. New events were also added which signal when objects become visible or touch.

Future Outlook
This environment should entice more children to create computer-based experiences, at the same time as it generates a lot of data about what mappings are intuitive between physical and virtual worlds. By leveraging the real world, mixed-reality technology can make it easier for children to explore the capabilities of virtual environments. This is because physically interacting with virtual environments may be more intuitive than through a typical keyboard/mouse interface. In crafting projects through this environment, Scratch programmers will  generate  mappings  between tangible interactions and behaviors of programs; thus, a set of  frequently-used  natural mappings will  emerge  from  the user community.  Researchers  may  also  create  applications which make use of novel interaction techniques, and use the community  to  determine  if  and  how  children  grasp  these concepts.  In this sense, we expect  that  the community will become a research partner for tangible interface research.

Resources

Video showing demo examples (VMW, 15.6 MB)

No comments