We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Intelligent Machines

Microsoft Makes It Easier to Hack the Kinect

Developers get a new software kit to create Kinect applications, and a number of them did so at a 24-hour “code camp.”

Today, Microsoft launched Kinect for Windows, a software development kit that makes it easier for programmers to dream up new uses for its gesture-sensing hardware. Microsoft also organized a 24-hour “code camp,” at which hackers, academics, and hobbyists gave the software a try.

Lifts off: Oregon State University students test software written using Kinect for Windows. Alex Wiggins operates a remote-control toy helicopter while teammates Ruma Paul (left) and Fabio Matsui (right) look on.

At the end of the camp, one group of hackers presented a game that would allow a traveling parent to interact with a child back at home. The child controls a character in a maze by moving in front of a Kinect, while the parent controls a character through accelerometers in a Windows 7 smart phone. The two can play cooperatively and talk to each other over the phone and the Kinect’s microphones. Another project developed during the camp lets users control a toy helicopter with gestures. A third gives Kinect users control of a virtual orchestra, including the ability to swell sound levels by lifting their hands. A fourth group created virtual light sabers.

Shortly after the Kinect launched late last year, hackers started enthusiastically coopting it for all sorts of purposes, to take advantage of its sophisticated depth-sensing capabilities. They were able to do it because they found a way to pull raw data from the device. A community of developers devoted to figuring out how to process that data also sprang up. Kinect for Windows gives outside developers access to software that Microsoft uses to process Kinect data, and also interfaces smoothly with other Microsoft products, such as the Windows 7 and Windows 7 Phone operating systems.

The Kinect, which retails for $150, includes sensors that hackers say rival or exceed those in pieces of hardware that sell for thousands. The device contains depth sensors, a camera, and an array of microphones. Hobbyists have used the Kinect to do everything from helping robots navigate to controlling unmanned flying drones with body movements.

Anoop Gupta, a distinguished scientist at Microsoft Research, stresses that Microsoft’s official offering, which is free to download and works on Windows 7, takes advantage of the company’s experience in creating the Kinect in the first place. “We believe we have the deepest insight into the technology,” Gupta says. In particular, the company is giving access to the algorithms that it uses to process information from the Kinect’s sensors.

The new software could expand the appeal Kinect hacking holds for less experienced programmers. Not only does it continue to allow access to raw data streams from the camera or microphones, but it also offers access to Kinect data that’s already been cleaned up by Microsoft’s algorithms. Gupta says, for example, that Kinect for Windows can automatically perform skeletal tracking, which lets developers reference body parts such as “left hand” in their code. The software also includes access to advanced audio capabilities, such as pinpointing the source of a voice using data from the microphones. Kinect for Windows includes integration with Windows 7 features. For example, data from the Kinect can be pulled into the speech-recognition application programming interface in Windows.

Developers can program with Kinect for Windows using any of Microsoft’s .NET languages, including C# and Visual Basic. Gupta also notes that Microsoft has designed the software so that projects will remain stable even if the company releases a different version of the Kinect.

“Today is a huge day for hackers and for Microsoft,” says Phillip Torrone, creative director at the open-source hardware company Adafruit Industries and co-founder of the Open Kinect project, which offered a $3,000 bounty to the first person who developed software that gave other programmers access to the Kinect’s functionality. “The hackers won the race to show how amazing ‘unlocked’ and freed hardware can be, and Microsoft won by ultimately embracing and celebrating all the talented makers, hackers, and artists who have created with the hacked Kinect.”

Torrone believes that the hacking community that has developed around the Kinect in the months since its launch has taught Microsoft a lesson. He says, “I think Microsoft was shocked and amazed by how quickly the open-source community created so many projects within weeks of the Kinect launch. The response was so overwhelming that Microsoft couldn’t beat us, so they joined us.”

Some hackers, however, are more skeptical. Garrat Gallagher, a robotics engineer who works at MIT’s Computer Science and Artificial Intelligence Laboratory, has created numerous hacks using the Kinect, including a robot called the Bilibot. He says Microsoft’s software development kit is “a step in the right direction” but is still “woefully behind the curve.” In particular, he says, “the terms of use are still pretty restrictive.” He’d like to see Microsoft release the software as an open-source project as well.

Gupta says Microsoft eventually plans to launch software that allows developers to build commercial products using the Kinect.

Learn from the humans leading the way in intelligent machines at EmTech Next. Register Today!
June 11-12, 2019
Cambridge, MA

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Print + All Access Digital.
  • Print + All Access Digital {! insider.prices.print_digital !}*

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivery each weekday to your inbox

    The MIT Technology Review App

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.