Microsoft’s New Idea: A Hologram Headset to Rewrite Reality

Microsoft today showed off an augmented reality headset slated for release later this year. The compact, visor-style device, called Hololens, generates holograms that make 3-D objects appear to the wearer to be overlaid on the real world.

Depth sensors in the device mean that virtual objects can be interactive and integrate smoothly into the real world. For example, a virtual game character could appear to be sitting on your coffee table, and respond to a prodding finger. Headphones built into the device can also give the impression that virtual objects are producing sound, even if they are behind you.

Microsoft unveiled Hololens at a press event Wednesday on its campus in Redmond, Washington. Demonstrations of the new device included an app that showed elements of the popular game Minecraft, which Microsoft acquired last year, sitting on top of real objects. Another app let the wearer build 3-D objects by manipulating a virtual toolbox and using voice commands.

“We’re not talking about putting you into virtual worlds,” said Alex Kipman, a technical fellow at Microsoft who worked on Hololens, and previously led development of the Kinect depth-sensing game controller. “Holograms behave just like real-world objects.”

Kipman didn’t discuss the technology in depth, but suggested that Hololens uses a more sophisticated approach to tricking the human brain than is used by 3-D movies or virtual reality headsets such as the Oculus Rift. Those devices create the illusion of depth by showing different, flat images to each eye. Holograms provide a more authentic illusion by recreating the complex “light fields” that real objects produce.

Light field displays have been tinkered with in research labs before, but no one has previously shown off a compact, wearable system, let alone what looks like a more or less finished product. A startup called Magic Leap last year received $542 million in funding from investors led by Google to work on headsets that use light fields for augmented reality (see “How Magic Leap’s Augmented Reality Works”). However, public descriptions of its technology suggest it is currently in the form of bulky tabletop devices.

Nothing was said about how Microsoft had made the technology so compact, or how long its battery life might last on a charge. The device’s sensors and complex functions could use a lot of power. Kipman said that the device has general purpose and graphics processor chips, as well as a new “holographic processing” chip developed to handle calculations involved in displaying holograms.

Hololens was unveiled at the tail end of a presentation mostly about Windows 10, the next version of Microsoft’s operating system. Kipman said the new device would be “available in the Windows 10 timeframe,” which would mean a release before the end of the year.

Microsoft is now working with other companies to develop apps and other experiences for Hololens. Kipman invited Oculus, Magic Leap, and people working on content for Google’s wearable computer, Glass, to develop content for his device.

Microsoft has already collaborated on one app for its Hololens. Data from NASA’s Mars rovers is used to give the impression of actually standing on the red planet. The app also provides the ability to use gestures to annotate what you see. NASA says it will use the technology to control its Mars rovers this summer.