Microsoft Makes It Easier to Hack the Kinect
Developers get a new software kit to create Kinect applications, and a number of them did so at a 24-hour “code camp.”
Today, Microsoft launched Kinect for Windows, a software development kit that makes it easier for programmers to dream up new uses for its gesture-sensing hardware. Microsoft also organized a 24-hour “code camp,” at which hackers, academics, and hobbyists gave the software a try.
At the end of the camp, one group of hackers presented a game that would allow a traveling parent to interact with a child back at home. The child controls a character in a maze by moving in front of a Kinect, while the parent controls a character through accelerometers in a Windows 7 smart phone. The two can play cooperatively and talk to each other over the phone and the Kinect’s microphones. Another project developed during the camp lets users control a toy helicopter with gestures. A third gives Kinect users control of a virtual orchestra, including the ability to swell sound levels by lifting their hands. A fourth group created virtual light sabers.
Shortly after the Kinect launched late last year, hackers started enthusiastically coopting it for all sorts of purposes, to take advantage of its sophisticated depth-sensing capabilities. They were able to do it because they found a way to pull raw data from the device. A community of developers devoted to figuring out how to process that data also sprang up. Kinect for Windows gives outside developers access to software that Microsoft uses to process Kinect data, and also interfaces smoothly with other Microsoft products, such as the Windows 7 and Windows 7 Phone operating systems.
The Kinect, which retails for $150, includes sensors that hackers say rival or exceed those in pieces of hardware that sell for thousands. The device contains depth sensors, a camera, and an array of microphones. Hobbyists have used the Kinect to do everything from helping robots navigate to controlling unmanned flying drones with body movements.
Anoop Gupta, a distinguished scientist at Microsoft Research, stresses that Microsoft’s official offering, which is free to download and works on Windows 7, takes advantage of the company’s experience in creating the Kinect in the first place. “We believe we have the deepest insight into the technology,” Gupta says. In particular, the company is giving access to the algorithms that it uses to process information from the Kinect’s sensors.
The new software could expand the appeal Kinect hacking holds for less experienced programmers. Not only does it continue to allow access to raw data streams from the camera or microphones, but it also offers access to Kinect data that’s already been cleaned up by Microsoft’s algorithms. Gupta says, for example, that Kinect for Windows can automatically perform skeletal tracking, which lets developers reference body parts such as “left hand” in their code. The software also includes access to advanced audio capabilities, such as pinpointing the source of a voice using data from the microphones. Kinect for Windows includes integration with Windows 7 features. For example, data from the Kinect can be pulled into the speech-recognition application programming interface in Windows.
Developers can program with Kinect for Windows using any of Microsoft’s .NET languages, including C# and Visual Basic. Gupta also notes that Microsoft has designed the software so that projects will remain stable even if the company releases a different version of the Kinect.
“Today is a huge day for hackers and for Microsoft,” says Phillip Torrone, creative director at the open-source hardware company Adafruit Industries and co-founder of the Open Kinect project, which offered a $3,000 bounty to the first person who developed software that gave other programmers access to the Kinect’s functionality. “The hackers won the race to show how amazing ‘unlocked’ and freed hardware can be, and Microsoft won by ultimately embracing and celebrating all the talented makers, hackers, and artists who have created with the hacked Kinect.”
Torrone believes that the hacking community that has developed around the Kinect in the months since its launch has taught Microsoft a lesson. He says, “I think Microsoft was shocked and amazed by how quickly the open-source community created so many projects within weeks of the Kinect launch. The response was so overwhelming that Microsoft couldn’t beat us, so they joined us.”
Gupta says Microsoft eventually plans to launch software that allows developers to build commercial products using the Kinect.
Keep up with the latest in detecting computer hackers at EmTech MIT.
Discover where tech, business, and culture converge.
September 11-14, 2018
MIT Media Lab