Skip to Content

With SideBySide, Projections Interact

Disney Research and Carnegie Mellon team up to create a novel interactive system that uses both visible and invisible projections.
October 27, 2011

Researchers have developed a new device called SideBySide, which allows two projections to interact with one another. Drawing apparent inspiration from a range of sources–the device is reminiscent in some respects of Microsoft Kinect, in other respects of smart phone technology–the device was developed by Disney Research and Carnegie Mellon, which are both located in Pittsburgh. One of the most interesting and important features of the system is that it allows ad-hoc interaction on any surface; you don’t need to wire up an environment first in preparation.

As Disney Research explains on its site, the system projects both visible and invisible (near-infrared) light. Users maneuver the visible projections–two boxers duking it out, for instance. Meanwhile, the infrared projections contain markers that help the system realize when the projections are overlapping. These invisible projections also enable communication between the devices. (“Fiducial markers” is the term of art here, for those who enjoy the jargon of imaging technology.)

The researchers put together a series of games that use the device, including a boxing game, a King Kong-like game where a gorilla and airplane do battle, and a game where players collaborate to knock bricks off of a platform. (None of the games has me eager to chuck my Xbox and race out for a SideBySide instead, but this is just a prototype, after all.) The researchers also developed basic non-gaming applications: a 3-D viewer that enables two people to collaborate in manipulating a 3-D model, and a basic file-transfer application, vaguely reminiscent of the iPhone/Android app Bump.

“Smartphones have made it possible for us to communicate, play games and retrieve information from the Web wherever we might be, but our interaction with the devices remains a largely solitary, single user experience,” Karl D.D. Willis, who is affiliated with both Carnegie Mellon and Disney Research, said in a release. “Now that handheld projectors have become a reality, we finally have a technology that allows us to create a new way for people to interact in the real world.” In other words, SideBySide offers a sort of bridge between the virtual world of the smart phone and the actual world of our four-walled rooms.

The researchers point to several limitations in their paper on the system, published recently. Projection brightness remains an issue, especially with picoprojectors–get more than 100 cm away from the wall, and the icons can be difficult to see. If sunlight, which contains infrared light of its own, streams in, it can interfere with the IR markers embedded in the images. Currently, the team behind the system runs it on an Apple MacBook Pro–but if this device were to be commercialized, users would probably want it on their phones. The authors say that they believe SideBySide could run “sufficiently” on current mobile devices, but it doesn’t look as though they’ve tested it yet.

Despite the limitations, the researchers have taken significant strides here. “Enabling separate handheld projection devices to interact together is an important step towards truly seamless interaction across the greater ubiquitous computing landscape,” write the authors in their paper. “We envision a day when digital content can traverse the boundaries of individual screens for fluid interaction between devices, people, and the physical environment.”

SideBySide points to that future–somewhat dimly, and in the form of a giant gorilla swiping at an airplane.

Keep Reading

Most Popular

DeepMind’s cofounder: Generative AI is just a phase. What’s next is interactive AI.

“This is a profound moment in the history of technology,” says Mustafa Suleyman.

What to know about this autumn’s covid vaccines

New variants will pose a challenge, but early signs suggest the shots will still boost antibody responses.

Human-plus-AI solutions mitigate security threats

With the right human oversight, emerging technologies like artificial intelligence can help keep business and customer data secure

Next slide, please: A brief history of the corporate presentation

From million-dollar slide shows to Steve Jobs’s introduction of the iPhone, a bit of show business never hurt plain old business.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.