Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Connectivity

Microsoft Researchers Are Working on Multi-Person Virtual Reality

Researchers in Jaron Lanier’s lab at Microsoft are exploring ways for people to share the experience of mixed reality.

Numerous companies are developing virtual reality products, but it’s not yet clear what the killer apps will be.

undefined
In an example of shared mixed reality, two students see a visual effect as they shake hands.

Augmented or mixed reality, which renders virtual images in a view of the real world, can be spectacular to experience. But it may be even more fun when you bring a friend.

Researchers in the Microsoft lab of Jaron Lanier are experimenting with multi-person augmented reality, and the results of their work could help shape the way the technology is commercialized and used. Lanier was a pioneer of virtual and augmented, or mixed, reality in the 1980s.

Microsoft is testing a commercial augmented reality product, called HoloLens. Lanier stresses that his work is separate from HoloLens and does not reflect how that product will develop. Still, multi-person mixed reality is a long-standing challenge for those interested in the technology. Beyond gaming, there is hope that virtual and augmented reality could prove useful for communications, collaboration, and for new ways of accessing and handling information.

One of Lanier’s students, Andrea Stevenson Won, wears a headset developed in his lab at Microsoft.

Lanier’s project is called Comradre (and pronounced “comradery”). A video produced by Lanier’s lab shows several projects developed by student interns in which more than one person interacts with the same virtual object or phenomena. These headsets were made using smartphones and laptop computers, and the tracking of a person’s head movements are performed using external sensors. And most of the footage was shot through one of the headsets.

The video shows a number of experimental applications developed for the platform. Andrea Stevenson Won of Stanford University is testing the social implications of the technology with a system that highlights physical interactions with virtual effects. Judith Amores and Xavie Benavides from the MIT Media Lab created a way for real-world blocks to correspond with virtual objects and animations, offering a way for children to play in mixed reality, even if not everyone is wearing a headset.

Other students in the group created more utilitarian uses. Andrzej Banburski of the Perimeter Institute for Advanced Physics, for example, developed a tool for visualizing mathematical equations. And Kishore Rathinavel, from the University of North Carolina, Chapel Hill, created a system that allows sound waves to be detected and then shown visually. In one scene from the group’s video, the sound produced by Lanier’s flute playing is shown in mixed reality. Two other students, Victor Mateevitsi, at the University of Illinois at Chicago, and Gheric Speiginer of Georgia Tech, were also part of the group.

Jaron Lanier

Both augmented reality and virtual reality—which provides a completely immersive experience—are currently hot areas of research and commercial development at several major technology companies. Facebook, Google, and Sony are all developing headsets and software for virtual reality. Meanwhile, a company called Magic Leap has raised more than $500 million in funding to develop a convincing form of augmented reality using holographic technology (see “Breakthrough Technologies 2015: Magic Leap”).

Microsoft demonstrated HoloLens last week at an event held for the launch of Windows 10, the latest version of its computer operating system. The demo showed an experimental game, called Project X-Ray, in which a player sees robots burst from a wall, and can shoot them using a handheld accessory.

The HoloLens videos show virtual objects generated on the fly and added to the video footage. So they are not a completely faithful representation of what a person sees through the headset. The field of view in a HoloLens headset, for instance, is actually relatively narrow, meaning virtual objects only appear directly in front of the wearer.

Speaking of Lanier’s project, Jeremy Bailenson, founding director of Stanford University’s Virtual Human Interaction Lab, says the key problem with creating virtual experiences for more than one person is tracking movement. “The critical aspect for multi-person virtual reality is that you have to track everybody’s movements very accurately in their own scene,” he says.

Bailenson believes that some form of virtual reality will prove especially useful for communications. And he says Microsoft is doing very good work with both HoloLens and its research. “Jaron is the guy to do this,” he adds. “He’s been thinking about this for decades.”

Be the leader your company needs. Implement ethical AI.
Join us at EmTech Digital 2019.

Register now
More from Connectivity

What it means to be constantly connected with each other and vast sources of information.

Want more award-winning journalism? Subscribe to Insider Online Only.
  • Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.