One of the most popular videos is of a 3-D interactive puppet. “It’s fun, it’s intuitive, and it’s something that would be really hard to do without this inexpensive, off-the-shelf component. As you bring down the barriers, people have room to get creative.”
MIT’s Fritz is quick to note that three-dimensional, natural user interface computing using gestural recognition and depth sensors has been in play in the research community for years. The Kinect is a breakthrough device in terms of packaging and implementing these technologies for consumers. The more familiar users become with it, the more likely they are to translate it to spheres beyond gaming.
“The keyboard and the mouse aren’t going anywhere, but there is a lot of space for something more, and I think people are ready for that,” Fritz says.
But any effort to translate gestures to the screen inevitably bumps into the fact that we’re still three-dimensional beings trying to interact with a two-dimensional world. Most Kinect games solve this problem by matching us with an onscreen avatar who imitates our movements. Whether we’re dancing, playing volleyball, or whitewater rafting, the characters on the screen perform a stylized version of our movements offscreen.
One solution could be to use light projectors to create virtual objects in real space that we can interact with. Microsoft Research has already taken steps in this direction with Mobile Surface, a projector-based multitouch environment.