Living in an Augmented Reality

I found this blog post interesting, when viewing high tech gadgetry from the viewpoint of mass appeal. I don’t necessarily agree with all of it, but those of us embedded in the world of high tech sometimes forget that most people don’t have the newest gadgets or will take a long time to get them, so it’s good to be reminded of this.  

I think it’s clear that the best strategy will be to come up with approaches to advertising and marketing that span the spectrum, from low to high tech, especially in the mobile sphere.  Which begs the question, what’s the “low tech” you want to support on a mobile device?  SMS?  MMS?  J2ME?  Do we “care” about folks who have the lowest-end of the lowest-end phones right now?

After all, if you look at the other side of the argument about what to assume in terms of mobile device capabilities, it’s clear that churn is pretty rapid in mobile devices, so shooting too low may be a bad strategy.  After all, how many of the masses does a “mass” marketing campaign have to hit?   Perhaps shooting for a subset of phones (1%? 5%?  10%?) right now is reasonable, since the owners of those may be the folks you want to hit anyway (tech savvy, disposable income, young?).  By the time we see if a technology works or not, those numbers will have likely risen (5?  10%?  25%?).  And what is the goal of a high-tech mobile advertising campaign right now?  Perhaps the PR from a well executed campaign may be a bigger benefit than the “click through” of a piece of technology.

Of course, it would be easy to say that I’m a technical optimist and not a marketing person.  Both of which are true.  And I’m working in the highest end of the high end of mobile technology (mobile AR).  However, given the cost of creating technology and porting to todays plethora of phones, one has to be careful and reflective on deciding what to try and what to invest in;  the low end approaches don’t seem (to me) to offer a lot of potential when compared to the web or print or TV.  

Unless you are leveraging location (GPS or otherwise), the camera (for AR, or QR codes, or other forms of tracking and recognition), local ad-hoc networks, and other unique mobile technologies built into the device, it’s unclear to me what mobile marketing has to offer.

First, in the spirit of full disclosure, I will point out that one of the students in my lab built this demo.  So, I’m obviously biased.  

That said, I am pretty impressed with what she managed to pull off in a very short time.  Ignoring the work involved in hacking the iPhone to get video out of the camera, and porting a little 3D loader/rendering to the phone, she managed to work in quite a few of the ideas we’ve been batting around about how to take advantage of the iPhone for handheld AR;  a bunch of things are still not in there, but will be eventually.  

The idea motivating this project is the old “Virtual Pet”, with the goal of having an individual’s pet be available on different platforms (handheld 3D, handheld AR, desktop 3D, etc), and of having multiple people’s pets be able to play with each other.  More interestingly, we hope to have the environment’s in which you play with your pet affect their growth and behavior.  We’ll see how far we get!

Here’s a video of the current prototype.

I just saw this post showing a video of an AR demo done in flash.  It’s cool;   I didn’t think Flash was fast enough to run the ARToolkit.

It looks like there are more reports on the DSi and it’s camera.  There is the potential for AR games here, but the big worry is that the device is too slow.  There are some details/rumors on the hardware floating around, but it’s not looking as promising as I’d like:  doing computer vision for any sort of tracking is expensive, and you still need to actually run the game!

That said, being able to do ANY AR on a commodity device will be awesome, as it will create an outlet for AR and drive people’s imagination.  And focusing on a kids device is key:  iPhone games are great when targeted at the > 20 year old set, but so many interesting AR games and “magic books” and so forth are targeted to much younger kids.  The iPhone isn’t going to be bouncing around in middle-school backpacks any time soon!

Picked up a pointer to an AR Book being shown at a German book fair.  While this blogger, and the blogger he points at, note that this is a “real” book (meaning, I assume, it will be for sale) rather than a demo, the video leaves the viewer wondering “so what?” …

We’ve built some AR books, and my friends and colleagues in Graz and HITLabNZ have too.  The critical issue whenever we try them with real people is “so what?” … meaning, why would I want to “read” this.

Look at the video being pointed by these articles and ask yourself that question.  At least in the video, these books appear to be nothing more than a placeholder for a (non-?)interactive bit of 3D and audio content;  does this look fun (minus the gee-wiz factor)?  Now look at how it’s being “read” … the “reader” is holding it up FACING AWAY FROM HER toward a camera on a computer screen and having to “read” it on the screen;  does this look fun?

I realize that there needs to be a “first” one, and that figuring out what the sweet spot and magic sauce will be that makes these work is non-obvious (otherwise, we’d all be reading them now, right?).  But, I wonder if the folks trying to create these have asked anyone besides their engineers “What is the real market for these?”  ”Who’s going to want to read something like this?”  ”What makes a 3D AR book good?” 

There are books:  they have certain appeals, they have certain affordances. There are computer games:  ditto, certain appeals and affordances.  ”Magic books” of this sort appear, to me, to combine the WORST of both, not the best.

I don’t think it has to be so.  It’s clear that there is an intuitive appeal to having books become 3D.  But, like print books before them, it’s the actually content (not the technology) that matters.

The metaio tech looks pretty good, if they can get it off the laptop and onto a device that might actually be more conducive to the “book” experience.  And if someone can get some good content in there.

This article on cnn.com paints a nice path toward the eventual deployment of handheld AR games.  Right now, they are not doing AR, just geolocated content, but by focusing on devices and concepts that require magnetic compasses in the phones, they will hopefully help push manufacturers toward creating devices with full 3d orientation sensors.  Again, as the article points out, we need both position (good GPS, better than currently available), a good compass AND tilt/roll sensing (full 3D orientation) to know “where you are looking.”  

Eventually, 3D position and orientation like this will just be the starting point, and the devices will use other information in the world (models of buildings, such as this system demonstrates, or the imaged-based approaches being worked on by Nokia Palo Alto researchers).  When such systems (which will require a massive amount of information about the real world, much more significant that systems like Google Earth and Google Streetview) are deployed, we can finally start doing REAL AR in the physical world.  

Some day, some day …

I ran across at article about some amazing AR art at the TodaysArt Festival when I stumbled on an article here.  Pablo Valbuena makes these very cool art pieces that appear to be created by careful projection onto architectural spaces.

It’s exactly this kind of work that will push the field of AR forward, because it’s compelling and not focused on the technology.  I am also a fan of Julian Oliver’s work, especially his Levelhead piece.

Over on Wired there is an article about a new Nintendo DS possibly having a camera and other support for creating AR games. Ori also comments on this.

I am also really excited by this; anything that pushes handheld AR will be a good thing for those of us currently engaged in it (and it’ll give my students more job opportunities!).  There is the practical issue of creating “good handheld AR games” that are practical for mass marketing and consumption. The DS/PSP/Gizmondo aren’t powerful enough to do “real computer vision” (like the PS3 could do), and so some sort of physical props (e.g., cards with markers on them) will be needed. As soon as your start requiring props, that makes the games less portable … a conundrum to be sure.

Hopefully, some of the students in the “handheld AR game design” class I’m doing at GT right now (in collaboration with a class at SCAD Atlanta) will come up with some compelling examples and help drive interest!

Lately, there have been a lot of folks getting into the “AR excitement”, especially with the iPhone SDK becoming available (i.e., all kinds of folks _have_ to have an AR demo on the iPhone, even though the camera sucks and you can’t even distribute an app that uses video legally because the SDK doesn’t support it and it’s not “legal” to reverse engineer unsupported APIs).  ”Sekai Camera” has gotten a ton of press, for example.  As did “Enkin” before that (a mockup of an Android app on a mac, pre-Android phone release).  Various companies have “point and know” kinds of technology, where the pitch is “using GPS and orientation information, combined with our vast wonderful backend database, you can point your phone at things and learn what they are.”

The problem, of course, is that these are really hard problems, and all of these systems only kinda-sorta work, even in their restricted demo modes.  Can I really point at that doggy in the window (as the google folks suggest you’ll be able to some day)?  Certainly not now.  And, most likely, not any time soon!  Could I point at the shop?  Perhaps.  At the items in the display case?  Not likely. 

The issue, of course, is that most of these so-called AR applications are more alluring than real.  One huge problem is that the amount of information needed to deliver on the hype is mind-boggling;  it’s the scale of information that will never be available in a closed system, in just the sort of system most of these demos are pushing.  

And, like the VR hype before this, and the AI hype before that, the worry is that (since none of these systems will do what they purport to do) the overhype will kill the potential industry and possible market.  There are companies who are tackling more modest problems, but they don’t get the PR and can’t create web memes because they aren’t as flashy.  That’s shame.

Because I’d hate to see AR creep back into the lab with it’s tail between it’s legs.  None of us who’ve been working on AR for decades want it to be the next “Big AI”.