Monday, March 12, 2007

TR10: Augmented Reality

Markus Kähäri wants to superimpose digital information on the real world.

By Erika Jonietz

Boxes appear on the phone’s screen, highlighting known businesses and landmarks, such as the Empire State Building. The user can click one of these boxes to download information about that location from the Web. In Nokia’s mobile-augmented-reality prototype, a user can point a phone’s camera at a nearby building; the system calculates the building’s location and uses that information to identify it. Credit: Jean Probert

This article is one in a series of 10 stories we're running this week covering today's most significant emerging technologies. It's part of our annual "10 Emerging Technologies" report, which appears in the March/April print issue of Technology Review.

Finding your way around a new city can be exasperating: juggling maps and guidebooks, trying to figure out where you are on roads with no street signs, talking with locals who give directions by referring to unfamiliar landmarks. If you're driving, a car with a GPS navigation system can make things easier, but it still won't help you decide, say, which restaurant suits both your palate and your budget. Engineers at the Nokia Research Center in Helsinki, Finland­, hope that a project called Mobile Augmented Reality Applications will help you get where you're going--and decide what to do once you're there.

Last October, a team led by Markus Kähäri unveiled a proto­type of the system at the International Symposium on Mixed and Augmented Reality. The team added a GPS sensor, a compass, and accelerometers to a Nokia smart phone. Using data from these sensors, the phone can calculate the location of just about any object its camera is aimed at. Each time the phone changes location, it retrieves the names and geographical coördinates of nearby landmarks from an external database. The user can then download additional information about a chosen location from the Web--say, the names of businesses in the Empire State Building, the cost of visiting the building's observatories, or hours and menus for its five eateries.

The Nokia project builds on more than a decade of academic research into mobile augmented reality. Steven Feiner, the director of Columbia University's Computer Graphics and User Interfaces Laboratory, undertook some of the earliest research in the field and finds the Nokia project heartening. "The big missing link when I started was a small computer," he says. "Those small computers are now cell phones."

Despite the availability and fairly low cost of the sensors the Nokia team used, some engineers believe that they introduce too much complexity for a commercial application. "In my opinion, this is very exotic hardware to provide," says Valentin Lefevre, chief technology officer and cofounder of Total Immersion­, an augmented-reality company in Suresnes, France. "That's why we think picture analysis is the solution." Relying on software alone, Total Immersion's­ system begins with a single still image of whatever object the camera is aimed at, plus a rough digital model of that object; image-­recognition algorithms then determine what data should be super­imposed on the image. The company is already marketing a mobile version of its system to cell-phone operators in Asia and Europe and expects the system's first applications to be in gaming and advertising.

Nokia researchers have begun working on real-time image-recognition algorithms as well; they hope the algorithms will eliminate the need for location sensors and improve their system's accuracy and reliability. "Methods that don't rely on those components can be more robust," says Kari Pulli, a research fellow at the Nokia Research Center in Palo Alto, CA.

All parties agree, though, that mobile augmented reality is nearly ready for the market. "For mobile-phone applications, the technology is here," says Feiner. One challenge is convincing carriers such as Sprint or Verizon that customers would pay for augmented-reality services. "If some big operator in the U.S. would launch this, it could fly today," Pulli says.