Today, Microsoft researchers will demonstrate software that can, in real time, superimpose computer-generated information on top of a digitized view of the real world.

Adding additional visual data to a video display is a technique known as augmented reality. Michael Cohen, principal researcher at Microsoft Research, in Redmond, WA, says that the approach could add another dimension to future smart phones. "You could be out on the street, hold the device up, and it could recognize a restaurant and deliver ratings and the menu," he says. A smart phone featuring an augmented-reality display could also overlay a bus route and an estimate of when the next bus is due on top of a particular street. "It essentially becomes your portal to information," Cohen says.

Cohen and his colleagues will demo the augmented-reality technology at TechFest, an annual showcase of Microsoft's research projects, in Redmond. Their software, which runs on a small portable computer, analyzes scenes from a camera, matches to those stored in a database, and overlays supplementary information on the display. The researchers note that a smart phone with augmented reality could help allow engineers to "see" the pipes or electrical cables below a street. In the demonstration given at TechFest, the software will be used to lead people on a treasure hunt to a hidden prize of a (virtual) pot of gold.

Augmented reality has been an active area of research for more than a decade, although it has often required a head-mounted display and a backpack's worth of computing equipment. In recent years, cell phones and portable computers with cameras and other sensors have become powerful enough to handle the computational workload needed to run an augmented-reality system. Researchers at Nokia and Columbia University, for instance, are also developing augmented-reality systems, and a Japanese startup called Tonchidot hopes to turn the concept into a product.

Most augmented-reality systems must be able to orient themselves accurately in order to function reliably. Some locate their position using GPS or by triangulating several Wi-Fi signals, and determine which way they are pointing using an accelerometer and a digital compass. Microsoft's augmented-reality device focuses on being able to recognize objects within a scene, using sophisticated computer-vision algorithms. Since the demonstration is carried out in the controlled environment of the conference hall, the researchers are not using location-detecting sensors; instead, they're relying solely on computer vision.