Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Connectivity

Microsoft Researchers Are Working on Multi-Person Virtual Reality

Researchers in Jaron Lanier’s lab at Microsoft are exploring ways for people to share the experience of mixed reality.

Numerous companies are developing virtual reality products, but it’s not yet clear what the killer apps will be.

Augmented or mixed reality, which renders virtual images in a view of the real world, can be spectacular to experience. But it may be even more fun when you bring a friend.

In an example of shared mixed reality, two students see a visual effect as they shake hands.

Researchers in the Microsoft lab of Jaron Lanier are experimenting with multi-person augmented reality, and the results of their work could help shape the way the technology is commercialized and used. Lanier was a pioneer of virtual and augmented, or mixed, reality in the 1980s.

Microsoft is testing a commercial augmented reality product, called HoloLens. Lanier stresses that his work is separate from HoloLens and does not reflect how that product will develop. Still, multi-person mixed reality is a long-standing challenge for those interested in the technology. Beyond gaming, there is hope that virtual and augmented reality could prove useful for communications, collaboration, and for new ways of accessing and handling information.

Lanier’s project is called Comradre (and pronounced “comradery”). A video produced by Lanier’s lab shows several projects developed by student interns in which more than one person interacts with the same virtual object or phenomena. These headsets were made using smartphones and laptop computers, and the tracking of a person’s head movements are performed using external sensors. And most of the footage was shot through one of the headsets.

One of Lanier’s students, Andrea Stevenson Won, wears a headset developed in his lab at Microsoft.

The video shows a number of experimental applications developed for the platform. Andrea Stevenson Won of Stanford University is testing the social implications of the technology with a system that highlights physical interactions with virtual effects. Judith Amores and Xavier Benavides from the MIT Media Lab created a way for real-world blocks to correspond with virtual objects and animations, offering a way for children to play in mixed reality, even if not everyone is wearing a headset.

Other students in the group created more utilitarian uses. Andrzej Banburski of the Perimeter Institute for Advanced Physics, for example, developed a tool for visualizing mathematical equations. And Kishore Rathinavel, from the University of North Carolina, Chapel Hill, created a system that allows sound waves to be detected and then shown visually. In one scene from the group’s video, the sound produced by Lanier’s flute playing is shown in mixed reality. Two other students, Victor Mateevitsi, at the University of Illinois at Chicago, and Gheric Speiginer of Georgia Tech, were also part of the group.

Both augmented reality and virtual reality—which provides a completely immersive experience—are currently hot areas of research and commercial development at several major technology companies. Facebook, Google, and Sony are all developing headsets and software for virtual reality. Meanwhile, a company called Magic Leap has raised more than $500 million in funding to develop a convincing form of augmented reality using holographic technology (see “Breakthrough Technologies 2015: Magic Leap”).

Jaron Lanier

Microsoft demonstrated HoloLens last week at an event held for the launch of Windows 10, the latest version of its computer operating system. The demo showed an experimental game, called Project X-Ray, in which a player sees robots burst from a wall, and can shoot them using a handheld accessory.

The HoloLens videos show virtual objects generated on the fly and added to the video footage. So they are not a completely faithful representation of what a person sees through the headset. The field of view in a HoloLens headset, for instance, is actually relatively narrow, meaning virtual objects only appear directly in front of the wearer.

Speaking of Lanier’s project, Jeremy Bailenson, founding director of Stanford University’s Virtual Human Interaction Lab, says the key problem with creating virtual experiences for more than one person is tracking movement. “The critical aspect for multi-person virtual reality is that you have to track everybody’s movements very accurately in their own scene,” he says.

Bailenson believes that some form of virtual reality will prove especially useful for communications. And he says Microsoft is doing very good work with both HoloLens and its research. “Jaron is the guy to do this,” he adds. “He’s been thinking about this for decades.”

Hear more about augmented reality from the experts at the EmTech Digital Conference, March 26-27, 2018 in San Francisco.

Learn more and register
More from Connectivity

What it means to be constantly connected with each other and vast sources of information.

Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

    Bimonthly digital/PDF edition

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special interest publications

    Discount to MIT Technology Review events

    Special discounts to select partner offerings

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning print magazine, unlimited online access plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

  • Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.