Intelligent Machines

Augmented Reality Study Projects Life-Sized People into Other Rooms

A Microsoft Research study uses augmented reality to project a life-size person into a room with you, perched in an empty seat.

Nothing beats talking to another person face-to-face, but a group of researchers are considering whether a life-size projection of a person who appears to be sitting across from you in an actual chair might be a close second.

Some of the equipment used to make Room2Room work: a Kinect depth-sensing camera and a digital projector.

Room2Room, a project from Microsoft Research, does just this: it uses Kinect depth cameras and digital projectors to capture the image of a person in 3-D in one room and project a life-sized version of that person in real time onto a piece of furniture in another room, where someone else is actually hanging out, and vice versa. Each person can then see a digital image of the other with the correct perspective, look at the other person from different viewpoints, and interact accordingly, the researchers say.

A paper on the work will be presented at the Computer-Supported Cooperative Work and Social Computing conference in San Francisco at the end of February.

Augmented reality—the idea of combining digital images with real life—has been around for years, but it’s only recently that the technology has started to catch on. Microsoft is among the companies trying to popularize AR by testing out its HoloLens headset, which it envisions as a tool for work and play, and secretive Florida-based startup Magic Leap is also working on a head-worn device that has similar aims (see “Breakthrough Technologies 2015: Magic Leap”).

Room2Room, an augmented-reality telepresence project from Microsoft Research, projects a life-sized digital version of a person onto a piece of furniture.

To make Room2Room work, the researchers took advantage of an existing Microsoft Research augmented-reality project called RoomAlive, which uses Kinect depth-sensing cameras and digital projectors to create a room-sized augmented-reality gaming arena. Instead of setting up just one room with this hardware, though, they set up two similar ones so they could scan a person sitting in each room and project him into the other one.

A video I was shown illustrates how it looks, with one man sitting on a chair, while another man is projected into a chair across from him (Room2Room places the projected image of a person into an open space, like a chair if the person is captured in a seated position).

In order to get a sense for how well people could communicate this way, seven pairs of study participants built three-dimensional shapes with blocks via augmented reality. One person sat in front of a table in one room with the actual blocks, while their partner sat in another, giving instructions for what kind of shape to construct. Each person was projected into the other’s room so they could work together.

Researchers found that while putting this kind of puzzle together took only about four minutes when people were face to face, it took about seven minutes using the augmented-reality system and nine via Skype video chat.

There are still plenty of other issues to solve before something like Room2Room is likely to show up in boardrooms or living rooms. While the depth-sensing and projection hardware needed to make it work is widely available, it’s bulky and can be a pain to set up. Also, it doesn’t produce very high-resolution images, says Tomislav Pejsa, who worked on Room2Room while an intern at Microsoft Research and was the lead author of the paper. The low resolution meant it could be hard to tell where a projected person’s gaze was aimed.

Tobias Höllerer, a professor at the University of California, Santa Barbara, who studies augmented reality, says the resolution could easily be improved, and expects we will be using systems similar to Room2Room in the coming years. The growing popularity of virtual reality, spurred by the upcoming release of some consumer-geared headsets from companies like Oculus, could help push this kind of augmented-reality technology along, too, he thinks.

“If you think about it, it took like 50, 60 years to get from the first demos of video telephony to where we are with Skype and everything else,” he says. “These are the beginnings of more immersive conferencing.” 

Get stories like this before anyone else with First Look.

Subscribe today
Already a Premium subscriber? Log in.

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.