Skip to Content

Microsoft Makes It Easier to Hack the Kinect

Developers get a new software kit to create Kinect applications, and a number of them did so at a 24-hour “code camp.”
June 16, 2011

Today, Microsoft launched Kinect for Windows, a software development kit that makes it easier for programmers to dream up new uses for its gesture-sensing hardware. Microsoft also organized a 24-hour “code camp,” at which hackers, academics, and hobbyists gave the software a try.

Lifts off: Oregon State University students test software written using Kinect for Windows. Alex Wiggins operates a remote-control toy helicopter while teammates Ruma Paul (left) and Fabio Matsui (right) look on.

At the end of the camp, one group of hackers presented a game that would allow a traveling parent to interact with a child back at home. The child controls a character in a maze by moving in front of a Kinect, while the parent controls a character through accelerometers in a Windows 7 smart phone. The two can play cooperatively and talk to each other over the phone and the Kinect’s microphones. Another project developed during the camp lets users control a toy helicopter with gestures. A third gives Kinect users control of a virtual orchestra, including the ability to swell sound levels by lifting their hands. A fourth group created virtual light sabers.

Shortly after the Kinect launched late last year, hackers started enthusiastically coopting it for all sorts of purposes, to take advantage of its sophisticated depth-sensing capabilities. They were able to do it because they found a way to pull raw data from the device. A community of developers devoted to figuring out how to process that data also sprang up. Kinect for Windows gives outside developers access to software that Microsoft uses to process Kinect data, and also interfaces smoothly with other Microsoft products, such as the Windows 7 and Windows 7 Phone operating systems.

The Kinect, which retails for $150, includes sensors that hackers say rival or exceed those in pieces of hardware that sell for thousands. The device contains depth sensors, a camera, and an array of microphones. Hobbyists have used the Kinect to do everything from helping robots navigate to controlling unmanned flying drones with body movements.

Anoop Gupta, a distinguished scientist at Microsoft Research, stresses that Microsoft’s official offering, which is free to download and works on Windows 7, takes advantage of the company’s experience in creating the Kinect in the first place. “We believe we have the deepest insight into the technology,” Gupta says. In particular, the company is giving access to the algorithms that it uses to process information from the Kinect’s sensors.

The new software could expand the appeal Kinect hacking holds for less experienced programmers. Not only does it continue to allow access to raw data streams from the camera or microphones, but it also offers access to Kinect data that’s already been cleaned up by Microsoft’s algorithms. Gupta says, for example, that Kinect for Windows can automatically perform skeletal tracking, which lets developers reference body parts such as “left hand” in their code. The software also includes access to advanced audio capabilities, such as pinpointing the source of a voice using data from the microphones. Kinect for Windows includes integration with Windows 7 features. For example, data from the Kinect can be pulled into the speech-recognition application programming interface in Windows.

Developers can program with Kinect for Windows using any of Microsoft’s .NET languages, including C# and Visual Basic. Gupta also notes that Microsoft has designed the software so that projects will remain stable even if the company releases a different version of the Kinect.

“Today is a huge day for hackers and for Microsoft,” says Phillip Torrone, creative director at the open-source hardware company Adafruit Industries and co-founder of the Open Kinect project, which offered a $3,000 bounty to the first person who developed software that gave other programmers access to the Kinect’s functionality. “The hackers won the race to show how amazing ‘unlocked’ and freed hardware can be, and Microsoft won by ultimately embracing and celebrating all the talented makers, hackers, and artists who have created with the hacked Kinect.”

Torrone believes that the hacking community that has developed around the Kinect in the months since its launch has taught Microsoft a lesson. He says, “I think Microsoft was shocked and amazed by how quickly the open-source community created so many projects within weeks of the Kinect launch. The response was so overwhelming that Microsoft couldn’t beat us, so they joined us.”

Some hackers, however, are more skeptical. Garrat Gallagher, a robotics engineer who works at MIT’s Computer Science and Artificial Intelligence Laboratory, has created numerous hacks using the Kinect, including a robot called the Bilibot. He says Microsoft’s software development kit is “a step in the right direction” but is still “woefully behind the curve.” In particular, he says, “the terms of use are still pretty restrictive.” He’d like to see Microsoft release the software as an open-source project as well.

Gupta says Microsoft eventually plans to launch software that allows developers to build commercial products using the Kinect.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

The problem with plug-in hybrids? Their drivers.

Plug-in hybrids are often sold as a transition to EVs, but new data from Europe shows we’re still underestimating the emissions they produce.

Google DeepMind’s new generative model makes Super Mario–like games from scratch

Genie learns how to control games by watching hours and hours of video. It could help train next-gen robots too.

How scientists traced a mysterious covid case back to six toilets

When wastewater surveillance turns into a hunt for a single infected individual, the ethics get tricky.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.