Skip to Content

Better Tools for Kinect Hacking

Academics and hobbyists are already hacking the Kinect—and doing so is about to get even easier.
March 2, 2011

A new generation of tools will soon make it even easier to tinker with the Kinect, the $150 Microsoft device that lets people control video games by gesturing.

Only Kinect: The Open Exhibits project creates free software modules for students, educators, museums, and other nonprofits so that they can use the Microsoft Kinect to create interactive exhibits such as the one above. It allows the user to navigate a virtual-reality version of the pre-Columbian Mayan city Uxmal.

Since the Kinect’s release in November, Microsoft has sold upwards of 8 million units and perhaps inadvertently started a frenzy of hacking, as academics and hobbyists have adapted the sensors available in the Kinect for projects ranging from robotics to unorthodox displays. The Kinect is so popular for such projects because it offers an inexpensive way to set up gestural controls. Unlike the Nintendo Wii, the Kinect can “see” users’ gestures without requiring a remote or special clothing. And it’s designed to work with USB, the protocol that’s commonly used to connect devices to computers.

Hackers figured out how to communicate with the device just days after its release, but recently companies—including Microsoft—have given their activities more official support.

To work with the Kinect, the hacker has to be able to get information from the device—such as the depth map that the Kinect constructs to be able to “see” what’s going on in a room. The hacker also needs to be able to define what should be done with the information the device picks up—in other words, what gestures to recognize. Software is needed for both of these roles.  Last week, Microsoft announced that it will release a noncommercial-software development kit for Windows, to make it easy for academics and researchers to get deep into the Kinect. The kit will offer access to the device’s audio system, direct control of the main sensor, and system-wide application programming interfaces. The company plans to follow that with a commercial version of the kit.

PrimeSense, a company based in Tel Aviv, Israel, which provided some of the technology that Microsoft licensed for the Kinect, has also been supporting hackers. It sells the PrimeSensor Reference Design, a device that’s very similar to the Kinect, and offers open-source drivers for it.

Finally, SoftKinetic, a company based in Brussels, Belgium, which specializes in 3-D gesture-recognition software, will offer a free version of its middleware; this will be available for download in mid-March. This software can take depth-sensing input from any device, including the Kinect, and it can be used to program a device to respond to gestures.

Kinect hackers already rely a great deal on shared information. “I don’t believe I actually hacked the Kinect,” says one tinkerer, Jim Spadacinni. He is owner and creative director of Ideum, a company that develops interactive exhibits for science museums and other institutions. Spadacinni is also the principal investigator for Open Exhibits, a project funded by the National Science Foundation, which is aimed at developing free open-source software to help put together exhibits. He has recently been investigating the Kinect’s possibilities on this front.

Open Exhibits has added to the base of knowledge about the Kinect by releasing a module that lets people write gesture-based applications for Kinect in Flash. Still, Spadacinni gives credit for that to people like Hector Martin, who released the open-source Open Kinect driver that many hackers have been using to extract and manipulate data from the device.

The ease with which people are now able to program for 3-D input “is an absolute turning point,” says Sean Kean, organizer of the New York City OpenKinect Meetup Group. “We will see a revolution.” Kean says that what’s being done with the Kinect is lowering the barriers that individuals face in experimenting with robotics and other advanced applications. The technology doesn’t just enable standing in front of a device and gesturing—it can be used for any sort of 3-D input, such as guiding a robot’s movements.

Indeed, hackers have used the technology to create a robot that can latch onto and follow a person, gesture-controlled interfaces for browsing news articles, and input for a 3-D display system.

All that was possible before, but not at such a low cost, says Garratt Gallagher, a robotics engineer who works at MIT’s Computer Science and Artificial Intelligence Laboratory and has created several Kinect hacks.  “Now,” Gallagher says, “we’ve gotten a giant step closer to all our science-fiction fantasies.”

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.