Skip to Content
Uncategorized

With SideBySide, Projections Interact

Disney Research and Carnegie Mellon team up to create a novel interactive system that uses both visible and invisible projections.
October 27, 2011

Researchers have developed a new device called SideBySide, which allows two projections to interact with one another. Drawing apparent inspiration from a range of sources–the device is reminiscent in some respects of Microsoft Kinect, in other respects of smart phone technology–the device was developed by Disney Research and Carnegie Mellon, which are both located in Pittsburgh. One of the most interesting and important features of the system is that it allows ad-hoc interaction on any surface; you don’t need to wire up an environment first in preparation.

As Disney Research explains on its site, the system projects both visible and invisible (near-infrared) light. Users maneuver the visible projections–two boxers duking it out, for instance. Meanwhile, the infrared projections contain markers that help the system realize when the projections are overlapping. These invisible projections also enable communication between the devices. (“Fiducial markers” is the term of art here, for those who enjoy the jargon of imaging technology.)

The researchers put together a series of games that use the device, including a boxing game, a King Kong-like game where a gorilla and airplane do battle, and a game where players collaborate to knock bricks off of a platform. (None of the games has me eager to chuck my Xbox and race out for a SideBySide instead, but this is just a prototype, after all.) The researchers also developed basic non-gaming applications: a 3-D viewer that enables two people to collaborate in manipulating a 3-D model, and a basic file-transfer application, vaguely reminiscent of the iPhone/Android app Bump.

“Smartphones have made it possible for us to communicate, play games and retrieve information from the Web wherever we might be, but our interaction with the devices remains a largely solitary, single user experience,” Karl D.D. Willis, who is affiliated with both Carnegie Mellon and Disney Research, said in a release. “Now that handheld projectors have become a reality, we finally have a technology that allows us to create a new way for people to interact in the real world.” In other words, SideBySide offers a sort of bridge between the virtual world of the smart phone and the actual world of our four-walled rooms.

The researchers point to several limitations in their paper on the system, published recently. Projection brightness remains an issue, especially with picoprojectors–get more than 100 cm away from the wall, and the icons can be difficult to see. If sunlight, which contains infrared light of its own, streams in, it can interfere with the IR markers embedded in the images. Currently, the team behind the system runs it on an Apple MacBook Pro–but if this device were to be commercialized, users would probably want it on their phones. The authors say that they believe SideBySide could run “sufficiently” on current mobile devices, but it doesn’t look as though they’ve tested it yet.

Despite the limitations, the researchers have taken significant strides here. “Enabling separate handheld projection devices to interact together is an important step towards truly seamless interaction across the greater ubiquitous computing landscape,” write the authors in their paper. “We envision a day when digital content can traverse the boundaries of individual screens for fluid interaction between devices, people, and the physical environment.”

SideBySide points to that future–somewhat dimly, and in the form of a giant gorilla swiping at an airplane.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.