Skip to Content

Google’s Experimental Smartphone Captures a Future Mapped in 3-D

Early testers are building a range of prototypes from drones to immersive video games using Google’s 3-D mapping smartphone.

Four months after Google unveiled Project Tango—a prototype Android smartphone with cameras and sensors that capture the phone’s environment in 3-D—developers are using the device to make cheap drones for surveying zones, more immersive video games, and even a body-part scanning system that could lead to a better-fitting suit.

Let’s dance: A Project Tango smartphone serves as the navigational brains for a drone at the University of Pennsylvania.

As well as a normal front-facing camera and sensors that measure orientation and motion, the device has a fish-eye camera on its rear that sees 180 degrees and a depth-sensing camera that uses infrared to capture a 3-D representation of the surrounding world. So far, 200 of the phones have been handed out to researchers and companies, who are doing things like strapping them to drones and using them to scan and model body parts.

Researchers and developers experimenting with Project Tango say it could make it easier, quicker, and cheaper to scan and map all kinds of places and objects in three dimensions. Vijay Kumar, a University of Pennsylvania professor who studies autonomous robots, says that Tango cuts the cost of making a flying robot, and is well-suited to performing both sensing and control because it’s so lightweight and compact. He’s experimenting with one strapped to an autonomous quadcopter; the device allows the copter to continuously return to a preprogrammed midair spot even after being pushed and pulled away from it.

Kumar says getting the Tango device to work with his robot was relatively easy, taking several hours of programming, rather than several weeks. In part, this is because smartphone processors are good enough to handle the data robots need to operate, thus avoiding the need to hand the data off to another computer, he says.

Kumar’s goal for Tango is for the handset and robots to one day be made cheaply enough that he could buy 100 of them and send the swarm into a disaster site, where they would collectively build a 3-D interior map within a few seconds.

“That would be cool,” he says.

Closer to earth, Amir Rubin, CEO and cofounder of 3-D indoor map startup Paracosm, is considering entertainment-oriented possibilities for Tango: he’s working on a video game demo.

Paracosm’s demo involves using the handset to scan a room that you can then “play” as a level in the game. You move through that same room—in reality—to control the first-person gameplay on the device’s screen (eventually you will control a character within the game, he says).

Johnny Lee, Project Tango’s tech lead, wouldn’t say when devices using the technology could hit the market. And since the existing Tango phones are just prototypes, there are many issues to work out before you’d want to buy one. For example, while developers praise its ability to combine a number of features—3-D scanning capabilities, position and orientation sensors—into a smartphone-sized package, they also say its battery life is relatively short (though they decline to say how long it lasts, exactly); it can’t do much graphics processing; and it can overheat and shut down.

Kumar says he can’t run the Tango-equipped drone for more than five minutes at a time without risking it shutting down midflight. “That would be like a bird flying and then you suddenly chop off its brains and it’s going to crash,” he says.

Developers expect improvements over time.

“It’s still remarkably good,” says Raj Sareen, CEO and cofounder of Styku, a 3-D body-scanning startup. His company is working on a demo that uses its scanning software and Project Tango to create 3-D models of people’s heads—meant to show how effectively it can scan body parts.

To use the demo, you hold a Tango phone in front of someone’s face and move it in a half circle around them. Once this is done, the data is uploaded to a Styku server that builds a 3-D model from the depth data that is then downloaded to the phone (the current version of the phone isn’t powerful enough to build the model itself; but Sareen expects this to change in the next iteration).

Styku currently uses other scanning systems, including Microsoft’s Kinect. The technology, which is used by stores like Nordstrom to help customers find the best-fitting clothes based on precise body measurements, costs about $5,000 to $6,000. If 3-D depth sensors are built into phones, he says, people could just scan themselves using a Styku app—making it a lot easier to buy custom-made clothing online. Or people might track their health and fitness by comparing the measurements of different body parts over time.

“I think we’re still a few steps removed from that,” he says. “But we strongly believe that’s the future.”

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

How scientists traced a mysterious covid case back to six toilets

When wastewater surveillance turns into a hunt for a single infected individual, the ethics get tricky.

The problem with plug-in hybrids? Their drivers.

Plug-in hybrids are often sold as a transition to EVs, but new data from Europe shows we’re still underestimating the emissions they produce.

It’s time to retire the term “user”

The proliferation of AI means we need a new word.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.