Skip to Content

Microsoft’s HoloLens Will Put Realistic 3-D People in Your Living Room

The HoloLens augmented-reality headset projects believable images of people into real settings.

Demonstrations of augmented-reality displays typically involve tricking you into seeing animated content such as monsters and robots that aren’t really there. Microsoft wants its forthcoming HoloLens headset to mess with reality more believably. It has developed a way to make you see photorealistic 3-D people that fit in with the real world.

Sensors in the HoloLens headset help it figure out how to present virtual objects so they fit in with the real world.

With this technology, you could watch an acrobat tumble across your front room or witness your niece take some of her first steps. You could walk around the imaginary people just as if they were real, your viewpoint changing seamlessly as if they were actually there. A sense of touch is just about the only thing missing.

That experience is possible because Microsoft has built a kind of holographic TV studio at its headquarters in Redmond, Washington. Roughly 100 cameras capture a performance from many different angles. Software uses the different viewpoints to create a highly accurate 3-D model of the person performing, resulting in a photo-real appearance.

The more traditional approach of using computer animation can’t compare, according to Steve Sullivan, who works on the project at Microsoft. He demonstrated what Microsoft calls “video holograms” at the LDV Vision Summit, an event about image-processing technology, in New York on Tuesday. More details of the technology will be released this summer.

“There’s something magical about it being real people and motion,” he said. “If you have a HoloLens, you really feel these performances are in your world.”

Microsoft is working on making it practical and cheap enough for other companies to record content in this form. It might one day be possible to visit a local studio and record a 3-D snapshot of a child at a particular point in life, said Sullivan.

Demonstrations of the technology included holographic videos of theatrical and acrobatic performances. Sullivan also showed how the format could be used in sports instruction. Someone looking for help with golf technique, for example, could wear a HoloLens to examine recordings of a pro golfer swinging different clubs at full or reduced speed, viewing the instructor up close and from different vantage points.

Microsoft has also recorded catwalk models using its system. That could help Internet shoppers by showing them how an item of clothing looks and hangs more realistically than is possible with still photos or 2-D video, said Sullivan. It should also be possible to use captured performances as the basis for animated characters in games or other applications, he said.

HoloLens uses a novel holographic display technology that can trick the eye into perceiving 3-D objects more effectively than conventional stereoscopic displays (see “Microsoft Making Fast Progress with HoloLens”). Sensors in the headset allow the device to figure out how to present virtual objects so they fit in with the real world. Sullivan showed how holographic videos can also be played back in 2-D on a tablet, in a special player that lets you drag your fingers to change your viewpoint on the action.

Several companies are working on ways to capture live action such as sports or movies for viewing on more conventional 3-D headsets like the Oculus Rift. But they capture only the 3-D view from the position of the camera at the time of shooting. Devices like the Rift also use a less sophisticated method of tricking the brain into perceiving 3-D objects, and they cannot mix virtual content with the real world.

A startup called Magic Leap, backed by Google, is developing its own wearable augmented-reality device based on display technology that’s similar to Microsoft’s (see “10 Breakthrough Technologies 2015: Magic Leap”). So far, however, Magic Leap’s demonstrations of its technology have involved animated content, not live action recorded in three dimensions.

Keep Reading

Most Popular

The Steiner tree problem:  Connect a set of points with line segments of minimum total length.
The Steiner tree problem:  Connect a set of points with line segments of minimum total length.

The 50-year-old problem that eludes theoretical computer science

A solution to P vs NP could unlock countless computational problems—or keep them forever out of reach.

section of Rima Sharp captured by the LRO
section of Rima Sharp captured by the LRO

The moon didn’t die as early as we thought

Samples from China’s lunar lander could change everything we know about the moon’s volcanic record.

conceptual illustration of a heart with an arrow going in on one side and a cursor coming out on the other
conceptual illustration of a heart with an arrow going in on one side and a cursor coming out on the other

Forget dating apps: Here’s how the net’s newest matchmakers help you find love

Fed up with apps, people looking for romance are finding inspiration on Twitter, TikTok—and even email newsletters.

ASML machine
ASML machine

Inside the machine that saved Moore’s Law

The Dutch firm ASML spent $9 billion and 17 years developing a way to keep making denser computer chips.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.