3-D Modeling Advance

Continued from page 1

To process an image, the algorithm divides the still image into tiny pieces or segments, says Ng. "It tries to take each of these small pieces and simultaneously figure out their 3-D position, angle, and orientation in the image."

When a new image is uploaded on the site, it only takes a couple of minutes for the algorithm to reconstruct it to a 3-D model and make a movie of the scene. However, the website is not yet optimal, so it takes about an hour for the user to receive an e-mail message indicating that her visualizations are ready. A user can store images and movies in a personal gallery on the site. The researchers are working to connect their site to photo-sharing sites like Photobucket and Flickr, says Saxena.

Make3D can also take two or three images of the same location to create a 3-D model similar to Microsoft's Photosynth application. (See "Microsoft's Shiny New Toy.") But Photosynth is a more expansive project that uses hundreds of images to reconstruct a scene, and when there are that many images to work with, computing the depth of scenes is not as mathematically complicated and is more accurate, says Hoiem. Make3D's focus is on processing single images for the general consumer, who might only take one image of a scene, says Ng.

Alex Daley, the group product manager for Microsoft Live Labs, says that there is a complementary relationship between single-image processors and multiple-image processors: improving single-image processing will ultimately make it easier for other systems to match multiple photos together. "Mixing and matching these for the right set of images will provide the best set of results," Daley adds. (He says that Microsoft is open to working with applications such as Make3D, but the company has not yet spoken with the Stanford researchers.)

Make3D's current algorithm only works on outdoor scenes or landscapes and a few kinds of indoor scenes, such as those that focus on staircases, and it's meant to help users share experiences or relive their own. The researchers are working to extend the algorithm to a broader range of settings so that it can recognize things like humans and coffee mugs and be used to create real-life environments for gaming and virtual worlds. Saxena is also working to incorporate the technology into robots to improve navigation and assist them at carrying out such tasks as unloading a dishwasher.

CMU's Efros says that the work provides a new perspective on the computer-vision problem and will hopefully result in a deeper understanding of how human vision functions.