Living in an Augmented Reality

I ran across at article about some amazing AR art at the TodaysArt Festival when I stumbled on an article here.  Pablo Valbuena makes these very cool art pieces that appear to be created by careful projection onto architectural spaces.

It’s exactly this kind of work that will push the field of AR forward, because it’s compelling and not focused on the technology.  I am also a fan of Julian Oliver’s work, especially his Levelhead piece.

Over on Wired there is an article about a new Nintendo DS possibly having a camera and other support for creating AR games. Ori also comments on this.

I am also really excited by this; anything that pushes handheld AR will be a good thing for those of us currently engaged in it (and it’ll give my students more job opportunities!).  There is the practical issue of creating “good handheld AR games” that are practical for mass marketing and consumption. The DS/PSP/Gizmondo aren’t powerful enough to do “real computer vision” (like the PS3 could do), and so some sort of physical props (e.g., cards with markers on them) will be needed. As soon as your start requiring props, that makes the games less portable … a conundrum to be sure.

Hopefully, some of the students in the “handheld AR game design” class I’m doing at GT right now (in collaboration with a class at SCAD Atlanta) will come up with some compelling examples and help drive interest!

Lately, there have been a lot of folks getting into the “AR excitement”, especially with the iPhone SDK becoming available (i.e., all kinds of folks _have_ to have an AR demo on the iPhone, even though the camera sucks and you can’t even distribute an app that uses video legally because the SDK doesn’t support it and it’s not “legal” to reverse engineer unsupported APIs).  ”Sekai Camera” has gotten a ton of press, for example.  As did “Enkin” before that (a mockup of an Android app on a mac, pre-Android phone release).  Various companies have “point and know” kinds of technology, where the pitch is “using GPS and orientation information, combined with our vast wonderful backend database, you can point your phone at things and learn what they are.”

The problem, of course, is that these are really hard problems, and all of these systems only kinda-sorta work, even in their restricted demo modes.  Can I really point at that doggy in the window (as the google folks suggest you’ll be able to some day)?  Certainly not now.  And, most likely, not any time soon!  Could I point at the shop?  Perhaps.  At the items in the display case?  Not likely. 

The issue, of course, is that most of these so-called AR applications are more alluring than real.  One huge problem is that the amount of information needed to deliver on the hype is mind-boggling;  it’s the scale of information that will never be available in a closed system, in just the sort of system most of these demos are pushing.  

And, like the VR hype before this, and the AI hype before that, the worry is that (since none of these systems will do what they purport to do) the overhype will kill the potential industry and possible market.  There are companies who are tackling more modest problems, but they don’t get the PR and can’t create web memes because they aren’t as flashy.  That’s shame.

Because I’d hate to see AR creep back into the lab with it’s tail between it’s legs.  None of us who’ve been working on AR for decades want it to be the next “Big AI”.

As I sit in the ISMAR panel on “the future of AR”, I’m reminded of work I did with Rob Kooper (a former student in my lab) back around 2000, the final publication of which was here:

The idea behind the real-world wide web (admittedly, not a great name) is to create a wearable AR interface to the web;  the fundamental assumption on which it is based was that you could author and query information based on location, not just content.  In that work, we brought up a lot of issues that would need to be addressed to make such an interface work, to make it safe and to make authoring and query practical.

The world has changed a lot;  Web 2.0 has arrive;  mobile phones are powerful;  RSS feeds, user-generated content and social networking are concepts familiar to a large part of the digital society.  And now, the AR community is finally starting to turn toward this problem.

In his keynote at AGDC, Bruce Sterling pushed the idea that in the future, all games will be AR games.  His premise of coming back from the future as a student sent by his older self, to talk about the industry in 2043 (35 years from now) is a bit trite, but how can I argue with the observation that

And the games of 2043? “They’re not the kind of games that were developed for flat glass screens — cumbersome,” he said. “We don’t pretend that a flat glass screen is a window into a virtual world… the idea sounds silly to us.”

Then what do the games of 2043 look like? “I think you would call [them] ‘augmented reality’ but we don’t,” Sterling continued. “We think that reality is real — you can have a lot of fun with [an overlaid] game interface.” To Sterling, the games of the future scale from personal “body games” to global games and space games and everything in between — including “neighborhood games”. More importantly, “[In 2043] we’ve got 70 years of computer games — that’s what we’ve got that you don’t have — and we got it from you. All kinds of dead intellectual properties and platforms, all being continually re-released.” 

It sounds like it was a bit of a light, motivational talk, but it seems everyone is getting the AR bug these days.

I am planning on doing some posts from ISMAR, to highlight what I think is cool.  First, of all, of course, is the city.  Cambridge is wonderful.

Nothing much else to post right now, except to suggest you head over to Games Alfresco, where Ori is covering the conference in depth.  Thanks Ori, it cuts down on what the rest of us need to type.

Well, I guess I’ll answer that question with “no.”   But, on the night before ISMAR starts here in Cambridge, England, I was sitting around with some fellow AR folk, having a pint and some fish and chips in The Eagle, and I let the unwashed iPhone-less masses indulge in Spore on my iPhone.

And it struck me as somehow appropriate that we were playing Spore in the Eagle Pub.   For those who don’t know,  the Eagle is the place where Francis Crick interrupted patrons’ lunchtime on 28 February 1953 to announce that he and James Watson had “discovered the secret of life” after they had come up with their proposal for the structure of DNA. 

Perhaps this is only amusing to those of us with jet lag.  Off to bed, then, and off to ISMAR tomorrow!

As usual, Ian has some very insightful, cut through the hype, things to say about the gaming and social media.  While he isn’t talking about AR, his comments about the nature of personal media and game-creator tools will be even more poignant as handheld AR games and social spaces “get out there.”  Once people can start “attaching their content” to the world, for anyone to see and experience, their will be a stronger push to move from static/linear content (e.g., photos, notes, videos) to dynamic media (e.g., animations that react to the viewers actions or to nearby activity in the world).

Inevitably, these will be called “games” and the kind of hype Ian refers to will pervade the media.  I tend to agree with Ian that focusing on the personal and shared nature of these “experiences,” the notion of “AR snapshots” if you will, will be key to understanding what people will want to do.  It won’t matter if they are “good games” or “clever art”, but that people can make their mark “for themselves”, for their friends and family.

Ian’s comments make me look forward to the day “the world is filling with trite and silly 3D litter” rather than bemoan having yet another channel of noise to wade through.  Like youtube and flikr, the key will be making it easy for users to author, to control what they see, and to serendipitously stumble on “random fun.”

While popping into Ori’s blog, I say this post: Unveiling Tonchidot: A Cool Parallel World - on the iPhone

Other coverage seems to indicate that it’s not real, and given the lack of detail (and the state of the art in mobile AR) it’s hard to imagine it can do what the video seems to want to imply that it does.  Of course, if it’s just grabbing information based on the location, and then using interesting layout algorithms, it could really work.  For example, given that the iPhone location could be pretty accurate, you could do some simple heuristic-based search to associate a “geo-tagged” piece of data (e.g., put this note at this intersection) with a part of the image stream (e.g., put it over this bit of the view of the world, like a sign).  It would likely be very fragile, but given that the labels seems to reposition a lot in the video, perhaps that’s what it’s doing.

Of course, like many of these demos (think about that one on Android, called Enkin), it’s amazing how they seem to be “inventing” the same idea over and over. 

And not addressing any of the “real” problems that will come up.  Consider what happens when thousands of people leave hundreds of notes in that mall the video appears in.  How does arfilter filter that?  How do people author content?  Deal with privacy, security and permissions?  Of course, as we all know, we must take baby-steps, but I think these questions are fascinating to consider.  Especially when the commentators refer to this as evoking something like Vinge’s “Rainbow’s End,” where these concerns are central.