Living in an Augmented Reality

While I usually reserve this spot to comment on the AR stuff other folks are working on, I thought I’d take this opportunity to comment on something we just released.

My colleague Maribeth Gandy and I create a small startup this year called Aura Interactive;  while we are doing a variety of things with the company, ranging from consulting to product development, our main goal is to create some fun AR games and toys.

And so I’m happy to share that we’ve just had our first iPhone application (done in collaboration with our colleague Kim Spreen) go live in the iTunes App Store, a little toy we call SantaVision (click here to go directly to the app in the iTunes Store).

Like many people, we’ve struggled with the inability to “touch” the live video on the iPhone, and the corresponding inability to do accurate, fast 3D tracking, of the sort we leverage in our research prototypes (e.g., the Zombie shooter ARhrrrr!).  At the same time, we’ve want to avoid creating yet-another shoot-the-<xxxxx>-around-you compass-based game, a couple more of which appeared today.  We think we have come up with a fun middle ground, leveraging the accelerometer and compass to let people immerse themselves inside small AR Christmas scenes they create themselves, using the “stickers” (static and animated images), sound effects and music loaded into the application.  And, using the GPS, you can share your geo-located scenes with the world.

If you try it, let me know what you think.  It’s not perfect, but we’re having fun with it!

I’m not quite sure what this means, but it is probably a good thing:

Zemoga, a pioneering digital marketing and innovation firm, today announced the launch of its augmented reality (AR) practice, designed to further the company’s work in this groundbreaking technology that has taken the marketing world by storm.

[...]

“We built the prototype of the human heart and could visually demonstrate in a dynamic 3D environment the effect a new drug would have on the heart,” explained Zemoga CEO and co-founder, DJ Edgerton.

I wonder if they are teaming up with an more experienced AR folks (if they are in New York, Steve Feiner’s group at Columbia seems like a good choice), to more rapidly leverage the research that’s been done over the years?  Regardless, though, it’s interesting.

Looking back over the summer, it’s almost comical to re-read the sometimes-frenzied excitement at the possibility that the iPhone will support AR with OS 3.1. The possibility was exciting, even to those of us not trying to keep a small company afloat;  the idea that there would finally be a platform that would let us get our ideas, games and products to millions of eager customers was an unfamiliar feeling for those of us who have been doing AR for a (very long) while. And for the dozen or so small AR companies, surviving on contract work for movies web sites and eye-candy web advertising, it must have been intoxicating.
Read the rest of this entry »

Ok, this is cute.  I wish I thought of it.  A use of FLAR (Flash AR) for something … almost useful?  And, most ironic of all, it’s by the US Postal Service!  I’m almost giddy!

It looks like someone may actually make an AR game for the DSi;  the GhostWire game that someone created on the Nokia phone is now going to be ported to the DSi.

Some folks would say this is not “really” AR, since the graphics (ghosts) are not tightly, accurately, spatially registered with the world.  Normally, I would agree;  as is obviously from my work and other posts, I think it is important to distinguish between different kinds of systems that relate the physical and virtual worlds, because different approaches have different affordances and capabilities.  I tend to call “AR” things that tightly register graphics with the physical world, because it creates experiences that afford certain things (e.g., allows people to leverage their spatial and perceptual abilities to remember where virtual things are, refer to them, perform actions in relation to them, collaborate, and so on). Conversely, I usually refer to experiences like this, that crudely position graphics in a space, as mixed reality (MR).

However, the ghost theme is the one theme that is amenable to “faking AR” on low end devices.  Over the years, we’ve done a lot of ghost-themed experiences, first on Auburn Avenue (an historic area in Atlanta) and later at the nearby Oakland Cemetery (where we had students do class projects for years in our Augmented and Mixed Reality Experience Design class).   One of the reasons ghosts “work” is that we all “know” how ghosts “behave”, and one of these behaviors is that they don’t stick to the real world, they float and shift and even appear to go through objects or walls.  Which is exactly what happens when the graphics aren’t registered.

So, the upside is, I’ll look forward to this game coming out, and like to see how it works.  There are lots of questions that remain to be answered before one can say if this will be good or not, but the idea is interesting and promising.

Ori posted a video of the demos, over on his blog.  He called it the first AR demo at GDC, which I don’t think is really true.  Sony must have demonstrated Eye of Judgement years ago, for example.  Plus, NVidia is demoing our game on their Tegra in their booth (so, we at least tied!).  :)

But, I don’t think any of that is important.  The Metaio demos are nice, and it’s good to see them combined with the Vuzix displays in the GDC expo!

Here’s a direct link to the video Ori took:

Writing an article about what’s wrong with software patents is old news, but as it touches my research area again, I find my getting infuriated with the ludicrous nature of the system. This latest round is brought to us by Thom Kidrin, the CEO at Worlds.com, who insists that he is

not out to put anyone out of business, he just wants to be paid licensing fees for what he considers his lawful intellectual property

IP passes from company to company, sure.  So, having lawful rights to IP you didn’t create  is perfectly fine.

The issue I have is the notion that these guys can claim, in public with a straight face, that their IP is actually valid and broad enough to cover all MMOs.  It’s plausible that they might have unique and new approaches to specific approaches to scalable worlds (although I seriously doubt it, given the work that had been going on in the military and research worlds before their 1997 work).

But, the specific architectures of SecondLife and WoW (the two worlds mentioned in the quote in this article) are completely different, so if he thinks he has patents that cover both, it means BY DEFINITION one or more of the following

1. He has no idea what he’s talking about.
2. He knows his patents are not enforceable but is hoping to make a quite buck off companies hoping to avoid expensive litigation.
3. The patents are overly broad and thus should be thrown out.
4. He knows his patents are not enforceable but is hoping to make a quite buck off companies hoping to avoid expensive litigation.

Patents are supposed to be non-obvious, and are supposed to be on inventions (mechanisms, machines, processes, etc), not concepts.  So, it may be reasonable for Worlds.com to have a specific architecture for scalable worlds patented, but since SL and WoW use different architectures, it seems unlikely that this is what the patents are about.  

Now, it may be the cases that they have patents on all plausible architectures. However, that brings up novelty and prior-art.  SIMNET, HLA and other highly scalable military simulation systems were implemented and published (and often times had open source implementations available).  Many researchers were presenting ideas for scalable worlds that were not commercially implemented, but were clearly documented.  

Or, to put it another way, the ideas behind the architectures of all of these MMOs where already publicly known, or at least were “obvious” based on what was publicly known, well before the late 1990’s.

(Just so folks know, I did a PhD from 1991 through 1998 on distribute virtual worlds software for augmented reality, so I’m not “unfamiliar” with this work).

It really is such a shame to see such a potentially good idea miss the mark.  A bunch of stories are running around the blog-o-sphere today, likely fueled by the article in the New York Times about Topps’ idea to save the baseball trading card business: AR baseball cards (go here and here and here, for example.  I like the blogs, since many of the comments express my feelings with much more colorful language).

 The idea is seemingly simple, to associate some virtual content with the card so that it serves as a bridge to the virtual world. It looks like an application tailor made for Total Immersion’s software (or software from a number of other folks, like Metaio).  Of course, a few clever Georgia Tech (or Graz and Oxford or Cambridge or MIT or …) students could hack the underlying system together in a few weeks with the Flash AR Toolkit, a bit of OpenCV and a database (of course, building the content is another story!). 

The problem with this concept is not the technology (although the graphics are pretty lame).  Rather, it is the failure to seemingly even consider trying to answer the question I drill into my students:  So What?  Why would anyone care? Putting a pointless, cute little 3D model on an advertisement is actually not a bad idea, since the technology is young and the point of advertising is to generate a bit of buzz and hopefully some associated mindshare.  

But, trading cards are different.  Here, the card is the business, and you are in effect asking people to spend real money for what is nothing more than a cute gimmick.   To quote the NYT article:

Total Immersion, a French company, brought Topps the augmented reality technology. It has already been used in a theme park and for some auto design work. Using the technology, card collectors see a three-dimensional version of a player and can play elementary pitching, batting and catching games using the computer keyboard.

Seeing 3D versions of your favorite players is … ummm … cute.  But there are more fun ways to do that (can you say “XBox 360″ or “PS3″?), and these have fun (not “elementary”) games associated with them. 

The NYT article quotes Eisner (of post-Disney fame) as being behind the idea;  does this mean he actually thinks people (kids, collectors?) will pay money for this?  I suspect, instead, this is a classic example of folks sitting around saying “they will buy this” even though the people coming up with the idea wouldn’t themselves buy it.  

Mr. Eisner said Topps expected to ship 10 million packs of Series 1 (12 cards for $2) and Topps Attax cards this year (5 for $1). Scott Kelnhofer, editor of Card Trade, an industry publication, says the Total Immersion technology could strike a chord with boys. “This is the boldest technology idea we’ve seen in sports cards so far. The key is not to have it be a novelty and then it’s on to the next one.”

Bold does not equal good.  As I say to the students in my class when they do experience or game designs:  if you are designing for some imagined person who “likes doing that stuff” (whatever that stuff is) even though you think it’s not fun, you are not going to succeed.  Or, to use a simpler HCI concept “get to know your user, don’t just trust your instincts”.  Stereotypes are fun, but they shouldn’t be the basis of a business plan.

There is, of course, room for this to grow into something interesting.  But, not by following this route.  I look forward to the day when AR is commonplace and a wide range of creative professionals are empowered to develop ideas.  Right now, we’re stuck in the AR-on-a-marker mindset (yes, these cards are effectively just pretty versions of the much-maligned ugly-black-and-white marker), where the “reality” is left out of “augmented reality”.  Perhaps we should start calling this “augmented video”?  

Of course, you might say “talk is cheap”.  Do I have big ideas on how to improve on this concept?  Sure, but I’ll reserve my detailed critique on how to improve this idea for in-class discussion (we’re teaching an AR Design class right now, no point giving away what our excellent Georgia Tech students are paying for!), in-lab brainstorming (my research group needs to publish, after all) and my consulting work (well, of course!).   But the basics are obvious:  leverage the real world, provide something of value that the users could not get any other way.  Here, the card could just link to a web site with the 3D model, and with video and stats and fun (not “elementary”) games.  Putting the little model on a card adds nothing, and (much like Eye of Judgement) will not hold collectors attention.

Of course, perhaps I’ll be wrong.  Perhaps this should be viewed as an elaborate advertising campaign?  If they can get a small percentage of current and potential customers to buy a pack or two, the whole thing may pay for itself and help them limp through these troubled times.  On the surface, however, taking the idea as it’s being presented, it’s a dud.

This videos are pretty slick.  While it might be possible to nitpic on some of them, as a whole they are very cool and compelling.  There seems to be the Surface group’s “fingerprints” all over them (lots of them appear to be Surface in different form factors), and some of it is a bit implausible, but over all it’s cool to really see what a lot of our ideas might look like.  After all, form factor is important, and its often hard to imagine what something would be like in practice when the current implementation is so far from reality.

I particularly like the small interactive cards (yes, not really AR) and the various AR (both the tablet sized ones and the one or two HMD-based ones).

Here are some links:

http://www.kzero.co.uk/blog/?p=2715 

http://www.istartedsomething.com/20090228/microsoft-office-labs-vision-2019-video/

Just read this blog post, containing an interview with a guy named Robert Rice.  While much of it is a bit over the top self-promotion, pushing his “new vision” (read: things he’s going to be trying to sell this year or next), it’s refreshing to at least read commentary by someone who gets that AR isn’t really about “putting graphics in the world”, but about the what, who, why and how.  Or, as he puts it:

it is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc.

Of course, he then uses this to irrationally broaden the definition of AR to dismiss the idea that it includes putting graphics in the world.  As those who know me would expect, this irritates me for a few reasons.  First, it makes “discussion” impossible if everyone just goes off and redefines terms that have perfectly good existing definitions, just to suit their purposes (read:  lots of people want AR to be more broad because then it includes whatever it is they are working on). Second, AR drives peoples imagination precisely because of the “putting graphics in the world” aspect of it;  there are lots of other terms to describe location-based, context-aware, mobile applications (e.g., ubiquitous computing, pervasive computing, mixed reality, etc) without having to try and latch your non-AR concept onto the excitement AR generates!

All that said, it sounds like these folks are going to try and actually do mobile AR, which will be very cool.  Of course, it’s easier said then done, but I’ll be pretty excited if they manage to release something!

Until then, those of us doing AR research will continue plugging along.  I’m excited by this years projects, especially the mirror worlds work we are doing, where we are going to start combining terrascale virtual worlds with mobile AR worlds!  Just the kind of thing this guy is hinting at, although not on the scale he wants (we’ll stick with the campus for now, thanks!)