But, I wonder about some of the non-obvious ways ways these games are different.  Today, as I was walking to the coffee shop to get beans for our espresso machine, I was thinking about the ways games enforce rules, and how board games and card games are fundamentally different than computer games.  With board and card games, rule enforcement is left up to the players;  they know the rules, and they abide by them.  Even when there is nobody around to “catch” them people naturally follow the rules;  to cheat removes the fun.  My dad played solitaire for hours to unwind after working shifts in an auto plant, and as far as I know, he never cheated.  What would be the point?

Computer games, on the other hand, encourage players to do anything the game allows to win.  Because the system is closed and the rules are enforced by the computer, finding ways to get around the system is part of the fun for many players.  If the game let’s you do it, it must be ok!

But, this presents a problem, which I’ll put this way:  will players treat hybrid computer/board games (or computer/card games) as board/card games or as computer games?  Will they play along and follow the rules, even if nobody is there to “call them on it”, or will they do what they can to win?  When there is a high score board, and achievements to unlock,  will players be content to take what the deck gives, or will they stack the cards?

This question is more than academic, because it impacts the kinds of games we can create.  Consider Sony’s Eye of Judgement, which is designed assuming players will treat the game as a computer game, and thus is structured to allow the computer to enforce the rules.

Click here to view the embedded video.

For those of you who have played the game, you know how awkward it is.  After playing for a while, you realize that you are playing a relatively complex card game (ala Magic), but one where the computer’s main job is to enforce the rules. The computer is used for little else, beyond adding eye-candy to justify playing the game in awkward space of the PS Eye camera.

But beyond the awkwardness, one can’t help but wonder what the game might have been like if the players were trusted to follow the rules and enlisted in the gameplay.  What kind of games could we create if the player was allowed to move cards fluidly, was allowed to manipulate their environment naturally, and not treated as a hostile, untrustworthy interloper?  What if the rules were presented and it was assumed people would follow them?

Imagine a single player augmented card game that starts with the player shuffling a deck and laying out 9 cards in a grid.  No checking by the computer, no proactive rule enforcement.  BUT, like traditional solitaire, the placement of the cards determines much about how the game unfolds.  Would such a game work?  It’s hard to say.  If a player approaches it like a card game, where the goal is to enjoy a pleasant diversion for a few minutes, and challenge yourself to solve a puzzle, perhaps such a game would work.

But, will players even consider that approach?  After years of being trained by computer games to take any advantage offered by the computer, will they “cheat” and chose a card layout designed to give them the best start to the game?  Will they even consider this “cheating”?

I’m not sure, but it certainly impacts the kinds of games we can create.

Even if we ignore the various bloggers and other hobbyist posting to the web (like I’m doing now), there was a surge of interest in AR from trade magazines (such as Wired and Computer Weekly) and the mainstream press (such as New York Times, CNN once, CNN a second time, the New Scientist, and others). And that’s just the published articles that I kept track of. Even sci-fi-author-turned-technical-prophet-turned-AR-fan Bruce Sterling has regularly devoted his Wired blog to AR topics (and gave a talk at the Layar “launch” to boot!).

With so much buzz, it will be interesting to see what happens (both to the AR industry and to the interest from the press), since much of the above-mentioned interest was driven by the overhyped press-releases from the various small companies trying to get a foothold in this nascent industry (as I talked to reporters, the conversations often started with “I saw a press release from XXX, and was wondering …”). Now that the time is here, now that the promised AR apps can be published in the iTunes store, will they be able to live up to their claims, or will they (and their claims) fade away?

I suspect things will die down for a little while. At least, I hope things die down for a while, given the paltry offerings coming out. Until “real” AR is possible, where the video from the camera can be analyzed and used for tracking and other effects, it will be hard to create applications that live up to peoples expectations. For example, AcrossAir has finally gotten a ream of apps in the iTunes App store, but none are particular exciting (as of today). As I pointed out to the reports who wrote the news articles above, the sensors in the iPhone (and Android phones) stink if your goal is to accurately overlay graphics on the camera’s view of the physical world; there’s no point pretending otherwise. The camera on the iPhone is pretty bad (in contrast to, say, the cameras and lenses in some Nokia phones), the magnetometer and accelerometers are noisy and imprecise (preventing fast motion or accurate 3D orientation estimation), and the GPS accuracy is pitiful (unless you consider ~25m good). So, given all this, companies like AcrossAir are perhaps doing the best they can.

But, have they set themselves up for a backlash of bad press?  The most recent of the above articles, in the New Scientist, hints at such a possibility.  Even the title (“Augmented Reality gets off to a wobbly start”) is worrying.

I worry about this because there is a beautiful irony in these so-called AR applications: what excites people about these “informational AR apps” is the idea of seeing information overlaid on the world around them … the world they can see nearby. But, the quality of the sensors in these phones (especially the GPS) means that none of these applications can align information with anything closer than a few hundreds of meters (or more). So, when you look for information, anything that relates to what you probably care about (the stuff nearby) is not where it should be, and the stuff that might be vaguely correct refers to things very far away in the physical world. It is not surprise that the first Wikitude video overlaid a point of interest on a distant mountain.

Consider the first AR app that was snuck into the iTunes store, the Monocle extension to Yelp. The app is hamstrung not only by these sensors, but by inaccurate location data in their user-generated review site. So, when I stepped out in front of my building (the Technology Square Research Building at Georgia Tech, in Atlanta) and looked around, the overlaid information was completely incorrect. Nearby restaurants where off (in some cases shown in the opposite direction from where I was looking), or completely absurd (such as a restaurant blocks away in one direction, showing up beside a nearby one). Here’s a video I took (poor quality because I pointed my junky point and shoot at the iPhone screen) showing what I’m talking about (I realize it will probably only make sense for those in Atlanta).

Click here to view the embedded video.

Aside from the errors, what does this video show you? What does Monocle “do” that wouldn’t be done better as a tradition 2D Google Map mashup? (Yes, I realize this was just a fun hack they put in there, but the quality and utility questions go beyond this app.) Here’s another Yelp screenshot, this time from my kitchen, showing a bunch of restaurants in a nearby neighborhood … overlaid on my daughters artwork on the kitchen wall. (We’ll ignore that I’m looking about 40 degrees to the left of where I should be to see these labels, even though the iPhone is actually placing me on the street in front of my house, close enough to were I am.)

So as not to just pick on Yelp, let’s look at an app that just got released in the iTunes store, just a few days ago, and one who’s sole purpose is to do AR: AcrossAir’s PhotosAR. I donated my 70% of $1.99 to the company to try it out, and the results are .. let’s just say “interesting.” Again, sitting in my kitchen, I see some images of Piedmont park a few miles away. The bottom picture isn’t actually too bad.

But there’s more photos of the park, and so the obvious thing for the app to do is … stack them? While this is “cute” in a “gee, there are incomprehensible photos floating above me in 3D, that is so cool” kind of way, it doesn’t seem like the best choice.  In this case, you can scroll the stack up and down, too, but this seems like a pretty arbitrary choice. But it highlights another important issue: how do we organize an overwhelming amount of 3D data in a way that benefits the user?   AR developers would do well to start looking at information visualization, 2D and 3D UI conference papers and textbooks.

As with the Yelp app, I’m left wondering if, perhaps, just retrieving the photos, grouping them into sets on a 2D Google Map, and letting users browse each of these sets in the built-in coverflow widget might be a much more useful solution?

As various reporters and business folks talked to me over the summer, I tried to emphasize the need to take this exciting new application space with a grain of salt. The sensors in the phones just aren’t up to the task; the lack of access to the actual video data on the iPhone will largely prevent anything great from happening on it any time soon. And, to temper expectations, I emphasized then (and continue to emphasize) that this first crop of AR apps will largely be silly little bits of eye candy, designed to (hopefully) generate some income for these companies, along with some press.

In the end, I think that this is probably ok, since many of these companies are serious about AR and will likely use that revenue to stay in business.  Which is what we in the community want, as long as the rush for quick profits doesn’t relegate AR to the dustbin of “Big AI” and “Virtual Reality.”

not out to put anyone out of business, he just wants to be paid licensing fees for what he considers his lawful intellectual property

IP passes from company to company, sure.  So, having lawful rights to IP you didn’t create  is perfectly fine.

The issue I have is the notion that these guys can claim, in public with a straight face, that their IP is actually valid and broad enough to cover all MMOs.  It’s plausible that they might have unique and new approaches to specific approaches to scalable worlds (although I seriously doubt it, given the work that had been going on in the military and research worlds before their 1997 work).

But, the specific architectures of SecondLife and WoW (the two worlds mentioned in the quote in this article) are completely different, so if he thinks he has patents that cover both, it means BY DEFINITION one or more of the following

1. He has no idea what he’s talking about.
2. He knows his patents are not enforceable but is hoping to make a quite buck off companies hoping to avoid expensive litigation.
3. The patents are overly broad and thus should be thrown out.
4. He knows his patents are not enforceable but is hoping to make a quite buck off companies hoping to avoid expensive litigation.

Patents are supposed to be non-obvious, and are supposed to be on inventions (mechanisms, machines, processes, etc), not concepts.  So, it may be reasonable for Worlds.com to have a specific architecture for scalable worlds patented, but since SL and WoW use different architectures, it seems unlikely that this is what the patents are about.  

Now, it may be the cases that they have patents on all plausible architectures. However, that brings up novelty and prior-art.  SIMNET, HLA and other highly scalable military simulation systems were implemented and published (and often times had open source implementations available).  Many researchers were presenting ideas for scalable worlds that were not commercially implemented, but were clearly documented.  

Or, to put it another way, the ideas behind the architectures of all of these MMOs where already publicly known, or at least were “obvious” based on what was publicly known, well before the late 1990′s.

(Just so folks know, I did a PhD from 1991 through 1998 on distribute virtual worlds software for augmented reality, so I’m not “unfamiliar” with this work).

In fact, I hate it so much, I created a mug (and shirt) on my Zazzle site to express my feelings.  I will drink coffee proudly from this mug, knowing that from my bravery, those around me may derive the courage they need to join me in coming out about their feelings.  

Repeat after me:  TeX Sucks.  Documents should be written, not programmed.

The original article is here, and the surreal version is here.  What fun.

it is inherently about who YOU are, WHERE you are, WHAT you are doing, WHAT is around you, etc.

Of course, he then uses this to irrationally broaden the definition of AR to dismiss the idea that it includes putting graphics in the world.  As those who know me would expect, this irritates me for a few reasons.  First, it makes “discussion” impossible if everyone just goes off and redefines terms that have perfectly good existing definitions, just to suit their purposes (read:  lots of people want AR to be more broad because then it includes whatever it is they are working on). Second, AR drives peoples imagination precisely because of the “putting graphics in the world” aspect of it;  there are lots of other terms to describe location-based, context-aware, mobile applications (e.g., ubiquitous computing, pervasive computing, mixed reality, etc) without having to try and latch your non-AR concept onto the excitement AR generates!

All that said, it sounds like these folks are going to try and actually do mobile AR, which will be very cool.  Of course, it’s easier said then done, but I’ll be pretty excited if they manage to release something!

Until then, those of us doing AR research will continue plugging along.  I’m excited by this years projects, especially the mirror worlds work we are doing, where we are going to start combining terrascale virtual worlds with mobile AR worlds!  Just the kind of thing this guy is hinting at, although not on the scale he wants (we’ll stick with the campus for now, thanks!)

Last month, Ori asked if I’d contribute some comments to his Wrap up 2008: Your Greatest Augmented Reality Moments post;  sadly, the end of last semester was way too busy, and I just wanted a break over the holidays, so I didn’t reply immediately … and then forgot.

But, I enjoyed his post, and thought I’d add a few comments on it here.  

Before I do, though, I must say that 2008 was an amazing year for AR, and 2009 only looks to be better!  

1. The Most fundamental AR milestone in 2008

Much technology continued to advance;  ISMAR 2008 was great, and the Tracking Competition saw some excellent progress demonstrated. New companies appeared (and I was involved with one), and a few new AR commercial demos happened (e.g., the Mini ad).

However, for me, it was all about handhelds.  The folks at Graz/Imagination have made great progress on getting tracking working better in their StbES system.  TI and NVidia released development kits for new mobile chipsets that will blow the socks off what we have in handhelds today, and Qualcomm and Intel have theirs on the horizon (the iPhone and G1 pale in comparison, but the thought of iPhone 2.0 with an OMAP3 or APX2500 makes me giddy!).  Heck, things are good enough that we managed to run a class across Georgia Tech and SCAD-Atlanta, where mixed teams of students from the two schools built 3 handheld AR games EACH over the course of a semester (videos will be up soon).  That would have been impossible a year ago.

Of course, the G1 and the iPhone were responsible for most of the hype, despite not actually being able to really do “real” AR yet (iPhone hacks by us and others notwithstanding).  

2) The best AR device of the year

To me, as you can tell from above, it’s undoubtably the new mobile chipsets.  Not really “devices” yet, but they are pretty amazing creatures.  When phones start appearing this year, amazing things will happen.

3) Best AR Demo

That’s really hard, especially because I’m fond of the ones we’ve been doing here.    I actually think a couple of the games that were done in my class could rank up there, but we haven’t posted them for others to see yet, so they aren’t really “in the running.”

I agree with Charles on Ori’s blog, the Haunted Book was excellent.

I think the Mini ad deserves mention, for it’s wide distribution.  I was also surprised by the FlashAR demo that is floating around;  getting AR working inside Flash is pretty cool.

4) Person of the AR year

Ha!  I’ll recuse myself here, for fear of offending friends who I don’t pick.  I really don’t think any one person deserves it, though.  There was nothing THAT amazing done by a single person to earn such an honor!

5) The most significant AR deal of 2008

I’m not sure anything stands out, really.  Lots of smallish things happening, lots of announcements of work starting (MediaPower’s AR efforts, Metaio’s publishing deal, Total Immersion, Imagination, etc), but nothing that has landed and been successful and blows away the rest.  

There are some amazing things likely to happen this year, though! 

6) A disappointment

I, too, am sad the Giz hasn’t yet come back.  We actually used original Giz’s in the class I mentioned, and they are a pretty capable device, considering their age.  A few limitations, but still better than anything else we could use!  Let’s hope for 2009!

7) Last minute find: A Surprising Simplicity in AR

I actually am not too excited about the Sketchup/AR Media combo.  Much more exciting is the AR-in-Flash demo; totally cool work.

The problem, of course, is that these are really hard problems, and all of these systems only kinda-sorta work, even in their restricted demo modes.  Can I really point at that doggy in the window (as the google folks suggest you’ll be able to some day)?  Certainly not now.  And, most likely, not any time soon!  Could I point at the shop?  Perhaps.  At the items in the display case?  Not likely. 

The issue, of course, is that most of these so-called AR applications are more alluring than real.  One huge problem is that the amount of information needed to deliver on the hype is mind-boggling;  it’s the scale of information that will never be available in a closed system, in just the sort of system most of these demos are pushing.  

And, like the VR hype before this, and the AI hype before that, the worry is that (since none of these systems will do what they purport to do) the overhype will kill the potential industry and possible market.  There are companies who are tackling more modest problems, but they don’t get the PR and can’t create web memes because they aren’t as flashy.  That’s shame.

Because I’d hate to see AR creep back into the lab with it’s tail between it’s legs.  None of us who’ve been working on AR for decades want it to be the next “Big AI”.

I’m a professor and researcher focusing on Augmented Reality (AR), and have decided to create a blog to give me a place to record some of my thoughts and reactions to things related to AR.  Over the years, I’ve wanted to start blogging some of my thoughts on things related to augmented reality (AR), but as with many people, I felt “too busy” to keep it up.  However, AR has gotten very exciting recently, and will be “real” very soon, given the rapid advance of mobile “application processors” (e.g., the chipsets in mobile phones) from companies such as TI and NVidia.

I personally have been doing AR research since 1991, when I started my Ph.D at Columbia University, working with Steven Feiner.  Steve was one of the first academics to really have the “nerve” to push this nascent technology as a research agenda, and I was lucky to be the first student of his working on it.  During my first semester, Doree Seligmann and I built KARMA (the Knowledge-based AR Maintenance Assistant) and, guided by Steve, published a paper on it that eventually turned into a 1993 Communications of the ACM paper that is still one of the most highly cited papers on AR.  

I graduated and took a job at Georgia Tech, where I’ve been doing AR for the past 8 or so years.  I’m lucky to be at a University that actually supports and encourages collaboration (as opposed to giving lip service to it, but then rewarding people for creating ivory towers), and the GVU Center manifests that support by creating an environment that is easily the best place in the world to do “all things related to the interplay between humans and computers”.  As a result, I’ve been collaborating with folks like Jay Bolter (a famous professor in New Media here at Tech), Simon Julier (a talented robotics/vision researcher who was at the Naval Research Lab in DC, and is now at University College London), Michael Mateas (the co-creator of Facade, was here for a few years, now at UC Santa Cruz), and on and on.  Fortunately, GT is an awesome technical school, has the worlds #1 HCI research group, a top vision-robotic center, and is one of the best places to do research on computer games and new media:  all the thing I need to explore AR. 

I could go on and on about where I work, but at the end of the day, the real point is that I get to work with a group of students and faculty studying interesting things related to AR;  games, industrial applications, the interaction between mobile AR and virtual worlds, new forms of computer vision, business collaboration, education, etc.

Anyway, I doubt anyone will read this far, but if you do, welcome!