Living in an Augmented Reality

I was just reading Mike Zyda’s article in the December CACM on games and computer science education.  It discusses the technical game education program they’ve created at USC in the CS department, and gives a nice overview of why they are doing things the way they are.  Seems like a reasonable degree.

Yet, whenever I read about someone’s technical game degree program, I’m always left wondering about the jobs we’re pointing these students at, and the “unwritten pack” we make with students we accept into our programs.  I’ve thought a lot about this, because the topic of a “game degree” comes up occasionally.  I’m also teaching 3 game classes this semester (a game-oriented capstone, a game prototyping lab, and an augmented reality game design class), so I’m very focused on the issue of game education.  Yet, with plenty of course and interest in games here at Georgia Tech, we haven’t created a full-blown game degree;  our game education activities are folded into a number of other degrees that offer a broader education beyond games.

This article isn’t about our choices at GT, though.  Rather, I interested in the opinions of others who might read this.  I believe there is an implicit suggestion that, if we have a focused degree program in a technical area, it’s educating the students in preparation for actually doing something.  With a liberal arts education, the goal is to give a broad education, and the students understand that there aren’t “liberal arts jobs” per se.  But, I suspect that students don’t generally get a degree in Mechanical Engineering, Information Security, or Pre-Med just because they want to broaden their horizons and open their minds;  they get these sorts of degrees (presumably) because they want to work in these areas after they graduate (or move on to other degrees, in the case of pre-Med or pre-Law).

So it is, I think, with a computer science or technical game-oriented degree.  Which brings me back to the topic of the post.  I wonder how many “game degrees” are being created because the school honestly believes that there is an industry need they are fulfilling (industry has unfilled job positions and needs to have more folks educated to fill them) or because the university has a need they are fulfilling (the student enrollments are dropping and they need students to fill up the classes).  For a number of years, CS enrollments have been down at many schools (we’ve been doing OK at Georgia Tech, and if you include our cross-over degrees like Computational Media, we’re actually doing better than OK, both at attracting students, but also at attracting students who aren’t young, white boys).  Over the past few years, a number of schools have created game degrees that have attracted a lot of attention, not least because it appears that they attract a lot of students and because their graduates get to go and work in the game industry.  That’s all well and good;  a few schools (USC’s MS program in the School of Cinema TV, CMU’s MS in Educational Technology, our combined undergrad and grad degrees across CS, CM and Digital Media, for example) have a great reputation with the game industry, and the students coming out of the program have generally had good success in getting the jobs they want.

But, how many schools are just putting lipstick (and new name) on a pig (their dying CS degree programs) to attract new students?

Overall, my sense is that there aren’t that many great jobs in the game industry, at least not in the numbers that are needed to employ an increasing number of game program graduates.  Especially the jobs many students seem to dream about (game design, game engine programming, etc.); here, I’m talking about technical and design jobs, I’m not talking about testing and QA, or level design, or content-oriented jobs (a few schools, like SCAD and RISD and so on, are doing a great job educating those students).

Unfortunately, when I read articles like Mike’s, I’m reminded of the increasing trend of CS departments to offer Game Degrees (note:  I’m not saying this is what USC is doing;  Mike has a long history of game education, going back to the Naval PostGraduate School, and actually came to USC to create a game program in the CS department).  A game degree is a great fit … for the school.  There is virtually no area of CS that isn’t applicable to the technical side of a modern game, especially the blockbuster console games;  these games require everything a major CS degree offers, and then some.  Taking your existing CS courses, faculty, labs and infrastructure, putting some new makeup on it, and calling it a game degree can be a great way to attract students.  And (based on anecdotal evidence) for many schools, it seems to be helping.  Helping them, that is, attract students.

My worry, though, is that as more and more schools offer game degrees, we’re going to turn out a generation of pseudo computer science students who can’t get the jobs they want.  After all, how many jobs are there?  I was chatting with a game industry exec at a recent conference, and he joked that, given the low turn-over in the good jobs, the good opportunities number in the hundreds … not in the thousands or more that will be needed to place these students.

So, I’d love to hear other folks thoughts on this, or get pointers to hard data about jobs and graduates.

It looks like someone may actually make an AR game for the DSi;  the GhostWire game that someone created on the Nokia phone is now going to be ported to the DSi.

Some folks would say this is not “really” AR, since the graphics (ghosts) are not tightly, accurately, spatially registered with the world.  Normally, I would agree;  as is obviously from my work and other posts, I think it is important to distinguish between different kinds of systems that relate the physical and virtual worlds, because different approaches have different affordances and capabilities.  I tend to call “AR” things that tightly register graphics with the physical world, because it creates experiences that afford certain things (e.g., allows people to leverage their spatial and perceptual abilities to remember where virtual things are, refer to them, perform actions in relation to them, collaborate, and so on). Conversely, I usually refer to experiences like this, that crudely position graphics in a space, as mixed reality (MR).

However, the ghost theme is the one theme that is amenable to “faking AR” on low end devices.  Over the years, we’ve done a lot of ghost-themed experiences, first on Auburn Avenue (an historic area in Atlanta) and later at the nearby Oakland Cemetery (where we had students do class projects for years in our Augmented and Mixed Reality Experience Design class).   One of the reasons ghosts “work” is that we all “know” how ghosts “behave”, and one of these behaviors is that they don’t stick to the real world, they float and shift and even appear to go through objects or walls.  Which is exactly what happens when the graphics aren’t registered.

So, the upside is, I’ll look forward to this game coming out, and like to see how it works.  There are lots of questions that remain to be answered before one can say if this will be good or not, but the idea is interesting and promising.

I’ve been quiet for a while (family stuff), but will start posting again.  I normally don’t focus on my own work, but I’m so proud of the work my students did this year, I want to share.  First, as Ori points out over on his blog, I finally posted the videos from our class projects last fall (wow, did that take a while).  For those that don’t know, last fall I taught a class (coordinated with a class taught by Tony Tseng at SCAD-Atlanta) on handheld AR game design.  It was pretty rough on the students (old, flaky hardware; new, work-in-progress software), but they did a great job.  We had them design and build a new game every month, so it was fast paced and fun.  Here’s a link to the playlist of the videos I posted (not all the games from the class, as a few games never resulted in videos on my computer …).  The game voted best game was Candy Wars, which I’ll embed here.

We also wrote a paper on the class, which was presented at the ACM Foundations of Digital Games conference. We also have a paper in the SIGGRAPH Games Track (on our collaborative AR game, Art of Defense), but I haven’t posted the final video yet (I’ll do that in a few weeks).

Finally, Kim and David improved our zombie shooter game (that I showed off at GDC); there’s a new, much cooler video, up. The game is now called ARhrrrr …

Ok, I’ve tried to avoid focusing on my own work here, but it’s been a fun week at GDC 2009 so far (the Game Developer’s Conference).  I gave a talk on Handheld AR Games on Monday at the GDC Mobile Summit (Ori wrote about that), and met a lot of interesting people after it.  There is clearly a growing understanding that AR is going to be a significant feature in the landscape of mobile AR, which is really exciting!

I also met with some folks from TouchArcade, a really great site for iPhone game reviews.  We talked for a while about our little iPhone game demo,and I showed them the other demo game we created on NVidia’s reference platform. They wrote a small story about our chat; the big point is that, while the current AR games on mobiles are still a bit rough around the edges, when you see what the new platforms (whether NVidia’s Tegra or TI’s OMAP3 or Qualcomm’s SnapDragon), you begin to understand what handheld AR might be like!

Not to leave out some video, here’s a short video of what I showed them on the NVidia Tegra.

And, for reference, our older iPhone pet game:

This article on cnn.com paints a nice path toward the eventual deployment of handheld AR games.  Right now, they are not doing AR, just geolocated content, but by focusing on devices and concepts that require magnetic compasses in the phones, they will hopefully help push manufacturers toward creating devices with full 3d orientation sensors.  Again, as the article points out, we need both position (good GPS, better than currently available), a good compass AND tilt/roll sensing (full 3D orientation) to know “where you are looking.”  

Eventually, 3D position and orientation like this will just be the starting point, and the devices will use other information in the world (models of buildings, such as this system demonstrates, or the imaged-based approaches being worked on by Nokia Palo Alto researchers).  When such systems (which will require a massive amount of information about the real world, much more significant that systems like Google Earth and Google Streetview) are deployed, we can finally start doing REAL AR in the physical world.  

Some day, some day …

Over on Wired there is an article about a new Nintendo DS possibly having a camera and other support for creating AR games. Ori also comments on this.

I am also really excited by this; anything that pushes handheld AR will be a good thing for those of us currently engaged in it (and it’ll give my students more job opportunities!).  There is the practical issue of creating “good handheld AR games” that are practical for mass marketing and consumption. The DS/PSP/Gizmondo aren’t powerful enough to do “real computer vision” (like the PS3 could do), and so some sort of physical props (e.g., cards with markers on them) will be needed. As soon as your start requiring props, that makes the games less portable … a conundrum to be sure.

Hopefully, some of the students in the “handheld AR game design” class I’m doing at GT right now (in collaboration with a class at SCAD Atlanta) will come up with some compelling examples and help drive interest!

In his keynote at AGDC, Bruce Sterling pushed the idea that in the future, all games will be AR games.  His premise of coming back from the future as a student sent by his older self, to talk about the industry in 2043 (35 years from now) is a bit trite, but how can I argue with the observation that

And the games of 2043? “They’re not the kind of games that were developed for flat glass screens — cumbersome,” he said. “We don’t pretend that a flat glass screen is a window into a virtual world… the idea sounds silly to us.”

Then what do the games of 2043 look like? “I think you would call [them] ‘augmented reality’ but we don’t,” Sterling continued. “We think that reality is real — you can have a lot of fun with [an overlaid] game interface.” To Sterling, the games of the future scale from personal “body games” to global games and space games and everything in between — including “neighborhood games”. More importantly, “[In 2043] we’ve got 70 years of computer games — that’s what we’ve got that you don’t have — and we got it from you. All kinds of dead intellectual properties and platforms, all being continually re-released.” 

It sounds like it was a bit of a light, motivational talk, but it seems everyone is getting the AR bug these days.

Well, I guess I’ll answer that question with “no.”   But, on the night before ISMAR starts here in Cambridge, England, I was sitting around with some fellow AR folk, having a pint and some fish and chips in The Eagle, and I let the unwashed iPhone-less masses indulge in Spore on my iPhone.

And it struck me as somehow appropriate that we were playing Spore in the Eagle Pub.   For those who don’t know,  the Eagle is the place where Francis Crick interrupted patrons’ lunchtime on 28 February 1953 to announce that he and James Watson had “discovered the secret of life” after they had come up with their proposal for the structure of DNA. 

Perhaps this is only amusing to those of us with jet lag.  Off to bed, then, and off to ISMAR tomorrow!

As usual, Ian has some very insightful, cut through the hype, things to say about the gaming and social media.  While he isn’t talking about AR, his comments about the nature of personal media and game-creator tools will be even more poignant as handheld AR games and social spaces “get out there.”  Once people can start “attaching their content” to the world, for anyone to see and experience, their will be a stronger push to move from static/linear content (e.g., photos, notes, videos) to dynamic media (e.g., animations that react to the viewers actions or to nearby activity in the world).

Inevitably, these will be called “games” and the kind of hype Ian refers to will pervade the media.  I tend to agree with Ian that focusing on the personal and shared nature of these “experiences,” the notion of “AR snapshots” if you will, will be key to understanding what people will want to do.  It won’t matter if they are “good games” or “clever art”, but that people can make their mark “for themselves”, for their friends and family.

Ian’s comments make me look forward to the day “the world is filling with trite and silly 3D litter” rather than bemoan having yet another channel of noise to wade through.  Like youtube and flikr, the key will be making it easy for users to author, to control what they see, and to serendipitously stumble on “random fun.”