With SideBySide, Projections Interact

Researchers have developed a new device called SideBySide, which allows two projections to interact with one another. Drawing apparent inspiration from a range of sources–the device is reminiscent in some respects of Microsoft Kinect, in other respects of smart phone technology–the device was developed by Disney Research and Carnegie Mellon, which are both located in Pittsburgh. One of the most interesting and important features of the system is that it allows ad-hoc interaction on any surface; you don’t need to wire up an environment first in preparation.

As Disney Research explains on its site, the system projects both visible and invisible (near-infrared) light. Users maneuver the visible projections–two boxers duking it out, for instance. Meanwhile, the infrared projections contain markers that help the system realize when the projections are overlapping. These invisible projections also enable communication between the devices. (“Fiducial markers” is the term of art here, for those who enjoy the jargon of imaging technology.)

The researchers put together a series of games that use the device, including a boxing game, a King Kong-like game where a gorilla and airplane do battle, and a game where players collaborate to knock bricks off of a platform. (None of the games has me eager to chuck my Xbox and race out for a SideBySide instead, but this is just a prototype, after all.) The researchers also developed basic non-gaming applications: a 3-D viewer that enables two people to collaborate in manipulating a 3-D model, and a basic file-transfer application, vaguely reminiscent of the iPhone/Android app Bump.

“Smartphones have made it possible for us to communicate, play games and retrieve information from the Web wherever we might be, but our interaction with the devices remains a largely solitary, single user experience,” Karl D.D. Willis, who is affiliated with both Carnegie Mellon and Disney Research, said in a release. “Now that handheld projectors have become a reality, we finally have a technology that allows us to create a new way for people to interact in the real world.” In other words, SideBySide offers a sort of bridge between the virtual world of the smart phone and the actual world of our four-walled rooms.

The researchers point to several limitations in their paper on the system, published recently. Projection brightness remains an issue, especially with picoprojectors–get more than 100 cm away from the wall, and the icons can be difficult to see. If sunlight, which contains infrared light of its own, streams in, it can interfere with the IR markers embedded in the images. Currently, the team behind the system runs it on an Apple MacBook Pro–but if this device were to be commercialized, users would probably want it on their phones. The authors say that they believe SideBySide could run “sufficiently” on current mobile devices, but it doesn’t look as though they’ve tested it yet.

Despite the limitations, the researchers have taken significant strides here. “Enabling separate handheld projection devices to interact together is an important step towards truly seamless interaction across the greater ubiquitous computing landscape,” write the authors in their paper. “We envision a day when digital content can traverse the boundaries of individual screens for fluid interaction between devices, people, and the physical environment.”

SideBySide points to that future–somewhat dimly, and in the form of a giant gorilla swiping at an airplane.