Living in an Augmented Reality

Looking back over the summer, it’s almost comical to re-read the sometimes-frenzied excitement at the possibility that the iPhone will support AR with OS 3.1. The possibility was exciting, even to those of us not trying to keep a small company afloat;  the idea that there would finally be a platform that would let us get our ideas, games and products to millions of eager customers was an unfamiliar feeling for those of us who have been doing AR for a (very long) while. And for the dozen or so small AR companies, surviving on contract work for movies web sites and eye-candy web advertising, it must have been intoxicating.
Read the rest of this entry »

Writing an article about what’s wrong with software patents is old news, but as it touches my research area again, I find my getting infuriated with the ludicrous nature of the system. This latest round is brought to us by Thom Kidrin, the CEO at Worlds.com, who insists that he is

not out to put anyone out of business, he just wants to be paid licensing fees for what he considers his lawful intellectual property

IP passes from company to company, sure.  So, having lawful rights to IP you didn’t create  is perfectly fine.

The issue I have is the notion that these guys can claim, in public with a straight face, that their IP is actually valid and broad enough to cover all MMOs.  It’s plausible that they might have unique and new approaches to specific approaches to scalable worlds (although I seriously doubt it, given the work that had been going on in the military and research worlds before their 1997 work).

But, the specific architectures of SecondLife and WoW (the two worlds mentioned in the quote in this article) are completely different, so if he thinks he has patents that cover both, it means BY DEFINITION one or more of the following

1. He has no idea what he’s talking about.
2. He knows his patents are not enforceable but is hoping to make a quite buck off companies hoping to avoid expensive litigation.
3. The patents are overly broad and thus should be thrown out.
4. He knows his patents are not enforceable but is hoping to make a quite buck off companies hoping to avoid expensive litigation.

Patents are supposed to be non-obvious, and are supposed to be on inventions (mechanisms, machines, processes, etc), not concepts.  So, it may be reasonable for Worlds.com to have a specific architecture for scalable worlds patented, but since SL and WoW use different architectures, it seems unlikely that this is what the patents are about.  

Now, it may be the cases that they have patents on all plausible architectures. However, that brings up novelty and prior-art.  SIMNET, HLA and other highly scalable military simulation systems were implemented and published (and often times had open source implementations available).  Many researchers were presenting ideas for scalable worlds that were not commercially implemented, but were clearly documented.  

Or, to put it another way, the ideas behind the architectures of all of these MMOs where already publicly known, or at least were “obvious” based on what was publicly known, well before the late 1990’s.

(Just so folks know, I did a PhD from 1991 through 1998 on distribute virtual worlds software for augmented reality, so I’m not “unfamiliar” with this work).

I realize this really doesn’t have much to do with AR, but I’ve finally had enough.  I’ve used TeX and LaTeX for too many papers (because some of my students and colleagues have been seduced by the Dark Side) and for one of my dissertations.  Enough is enough.  No more TeX.  

In fact, I hate it so much, I created a mug (and shirt) on my Zazzle site to express my feelings.  I will drink coffee proudly from this mug, knowing that from my bravery, those around me may derive the courage they need to join me in coming out about their feelings.  

Repeat after me:  TeX Sucks.  Documents should be written, not programmed.

Last september, I was quoted in an AJC (Atlanta Journal Constitution) article on the iPhone and Apple’s App Store. No big deal, just another quote. Yesterday, I stumbled upon what appears to be the result of running that article through a two-way automated translation program … anyway have a guess what language it went through? Italian, perhaps? It’s hard to tell.

The original article is here, and the surreal version is here.  What fun.

Just read this blog post, containing an interview with a guy named Robert Rice.  While much of it is a bit over the top self-promotion, pushing his “new vision” (read: things he’s going to be trying to sell this year or next), it’s refreshing to at least read commentary by someone who gets that AR isn’t really about “putting graphics in the world”, but about the what, who, why and how.  Or, as he puts it:

it is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc.

Of course, he then uses this to irrationally broaden the definition of AR to dismiss the idea that it includes putting graphics in the world.  As those who know me would expect, this irritates me for a few reasons.  First, it makes “discussion” impossible if everyone just goes off and redefines terms that have perfectly good existing definitions, just to suit their purposes (read:  lots of people want AR to be more broad because then it includes whatever it is they are working on). Second, AR drives peoples imagination precisely because of the “putting graphics in the world” aspect of it;  there are lots of other terms to describe location-based, context-aware, mobile applications (e.g., ubiquitous computing, pervasive computing, mixed reality, etc) without having to try and latch your non-AR concept onto the excitement AR generates!

All that said, it sounds like these folks are going to try and actually do mobile AR, which will be very cool.  Of course, it’s easier said then done, but I’ll be pretty excited if they manage to release something!

Until then, those of us doing AR research will continue plugging along.  I’m excited by this years projects, especially the mirror worlds work we are doing, where we are going to start combining terrascale virtual worlds with mobile AR worlds!  Just the kind of thing this guy is hinting at, although not on the scale he wants (we’ll stick with the campus for now, thanks!)

Well, the new year is here, and the AR future looks very promising.  Last year was quite a year, both for AR and for me;  there was good and bad, but we have a new year now, and it promises to be quite a breakout year for AR.

Last month, Ori asked if I’d contribute some comments to his Wrap up 2008: Your Greatest Augmented Reality Moments post;  sadly, the end of last semester was way too busy, and I just wanted a break over the holidays, so I didn’t reply immediately … and then forgot.

But, I enjoyed his post, and thought I’d add a few comments on it here.  

Before I do, though, I must say that 2008 was an amazing year for AR, and 2009 only looks to be better!  

1. The Most fundamental AR milestone in 2008

Much technology continued to advance;  ISMAR 2008 was great, and the Tracking Competition saw some excellent progress demonstrated. New companies appeared (and I was involved with one), and a few new AR commercial demos happened (e.g., the Mini ad).

However, for me, it was all about handhelds.  The folks at Graz/Imagination have made great progress on getting tracking working better in their StbES system.  TI and NVidia released development kits for new mobile chipsets that will blow the socks off what we have in handhelds today, and Qualcomm and Intel have theirs on the horizon (the iPhone and G1 pale in comparison, but the thought of iPhone 2.0 with an OMAP3 or APX2500 makes me giddy!).  Heck, things are good enough that we managed to run a class across Georgia Tech and SCAD-Atlanta, where mixed teams of students from the two schools built 3 handheld AR games EACH over the course of a semester (videos will be up soon).  That would have been impossible a year ago.

Of course, the G1 and the iPhone were responsible for most of the hype, despite not actually being able to really do “real” AR yet (iPhone hacks by us and others notwithstanding).  

2) The best AR device of the year

To me, as you can tell from above, it’s undoubtably the new mobile chipsets.  Not really “devices” yet, but they are pretty amazing creatures.  When phones start appearing this year, amazing things will happen.

3) Best AR Demo

That’s really hard, especially because I’m fond of the ones we’ve been doing here.    I actually think a couple of the games that were done in my class could rank up there, but we haven’t posted them for others to see yet, so they aren’t really “in the running.”

I agree with Charles on Ori’s blog, the Haunted Book was excellent.

I think the Mini ad deserves mention, for it’s wide distribution.  I was also surprised by the FlashAR demo that is floating around;  getting AR working inside Flash is pretty cool.

4) Person of the AR year

Ha!  I’ll recuse myself here, for fear of offending friends who I don’t pick.  I really don’t think any one person deserves it, though.  There was nothing THAT amazing done by a single person to earn such an honor!

5) The most significant AR deal of 2008

I’m not sure anything stands out, really.  Lots of smallish things happening, lots of announcements of work starting (MediaPower’s AR efforts, Metaio’s publishing deal, Total Immersion, Imagination, etc), but nothing that has landed and been successful and blows away the rest.  

There are some amazing things likely to happen this year, though! 

6) A disappointment

I, too, am sad the Giz hasn’t yet come back.  We actually used original Giz’s in the class I mentioned, and they are a pretty capable device, considering their age.  A few limitations, but still better than anything else we could use!  Let’s hope for 2009!

7) Last minute find: A Surprising Simplicity in AR

I actually am not too excited about the Sketchup/AR Media combo.  Much more exciting is the AR-in-Flash demo; totally cool work.

Lately, there have been a lot of folks getting into the “AR excitement”, especially with the iPhone SDK becoming available (i.e., all kinds of folks _have_ to have an AR demo on the iPhone, even though the camera sucks and you can’t even distribute an app that uses video legally because the SDK doesn’t support it and it’s not “legal” to reverse engineer unsupported APIs).  ”Sekai Camera” has gotten a ton of press, for example.  As did “Enkin” before that (a mockup of an Android app on a mac, pre-Android phone release).  Various companies have “point and know” kinds of technology, where the pitch is “using GPS and orientation information, combined with our vast wonderful backend database, you can point your phone at things and learn what they are.”

The problem, of course, is that these are really hard problems, and all of these systems only kinda-sorta work, even in their restricted demo modes.  Can I really point at that doggy in the window (as the google folks suggest you’ll be able to some day)?  Certainly not now.  And, most likely, not any time soon!  Could I point at the shop?  Perhaps.  At the items in the display case?  Not likely. 

The issue, of course, is that most of these so-called AR applications are more alluring than real.  One huge problem is that the amount of information needed to deliver on the hype is mind-boggling;  it’s the scale of information that will never be available in a closed system, in just the sort of system most of these demos are pushing.  

And, like the VR hype before this, and the AI hype before that, the worry is that (since none of these systems will do what they purport to do) the overhype will kill the potential industry and possible market.  There are companies who are tackling more modest problems, but they don’t get the PR and can’t create web memes because they aren’t as flashy.  That’s shame.

Because I’d hate to see AR creep back into the lab with it’s tail between it’s legs.  None of us who’ve been working on AR for decades want it to be the next “Big AI”.

Well, here it is, the first post on yet another blog.  

I’m a professor and researcher focusing on Augmented Reality (AR), and have decided to create a blog to give me a place to record some of my thoughts and reactions to things related to AR.  Over the years, I’ve wanted to start blogging some of my thoughts on things related to augmented reality (AR), but as with many people, I felt “too busy” to keep it up.  However, AR has gotten very exciting recently, and will be “real” very soon, given the rapid advance of mobile “application processors” (e.g., the chipsets in mobile phones) from companies such as TI and NVidia.

I personally have been doing AR research since 1991, when I started my Ph.D at Columbia University, working with Steven Feiner.  Steve was one of the first academics to really have the “nerve” to push this nascent technology as a research agenda, and I was lucky to be the first student of his working on it.  During my first semester, Doree Seligmann and I built KARMA (the Knowledge-based AR Maintenance Assistant) and, guided by Steve, published a paper on it that eventually turned into a 1993 Communications of the ACM paper that is still one of the most highly cited papers on AR.  

I graduated and took a job at Georgia Tech, where I’ve been doing AR for the past 8 or so years.  I’m lucky to be at a University that actually supports and encourages collaboration (as opposed to giving lip service to it, but then rewarding people for creating ivory towers), and the GVU Center manifests that support by creating an environment that is easily the best place in the world to do “all things related to the interplay between humans and computers”.  As a result, I’ve been collaborating with folks like Jay Bolter (a famous professor in New Media here at Tech), Simon Julier (a talented robotics/vision researcher who was at the Naval Research Lab in DC, and is now at University College London), Michael Mateas (the co-creator of Facade, was here for a few years, now at UC Santa Cruz), and on and on.  Fortunately, GT is an awesome technical school, has the worlds #1 HCI research group, a top vision-robotic center, and is one of the best places to do research on computer games and new media:  all the thing I need to explore AR. 

I could go on and on about where I work, but at the end of the day, the real point is that I get to work with a group of students and faculty studying interesting things related to AR;  games, industrial applications, the interaction between mobile AR and virtual worlds, new forms of computer vision, business collaboration, education, etc.

Anyway, I doubt anyone will read this far, but if you do, welcome!