Skip to Content

What Will Hackers Do with the New Kinect?

Upgraded robot vision will be just one of the uses for the new version of Microsoft’s gesture control camera.

Microsoft announced a new version of the Xbox One today, and with it an improved and essentially reinvented version of Kinect, the company’s body- and gesture-control sensor. That bodes well for Xbox gamers, but also for the community of hackers that have found so many original uses for the first Kinect, from robot vision to 3-D doodling (see “Hackers Take the Kinect to New Levels”). It seems likely that a new wave of Kinect hacking activity will begin as soon as the new device becomes available.

The new Kinect sensor can track finger movements and even a person’s pulse rate (Credit: Microsoft).

Reports from media sources like Wired, which got early access to the new Xbox and Kinect, say the new Kinect is significantly more accurate and capable of dealing with cluttered space, in part thanks to a fundamental change in how it works (more on which below). Microsoft says that the Kinect can now track the movement of fingers and facial features, not just limbs. It can keep tracking a person who is very close to other objects—for example, responding to hand gestures when a person is sitting on a couch. If a person sits down with the Xbox controller in view of the new Kinect, it will turn on the Xbox console. Kinect can also use its infrared light and camera to monitor a person’s pulse, by tracking the waves of blood in his or her face (a technique that works for any skin tone).

Those new capabilities will probably help existing Kinect hacks get better. Robots that use Kinect for 3-D vision, such as the PR2 that is used by many researchers (it’s used in this towel folding project), may see an instant vision upgrade, perhaps allowing them to handle more complex tasks (and laundry). The improved sensor may could also allow new hacks not possible before. One category could be adaptations that use Kinect for fine control of computers using hand and finger gestures, in a way similar to the Leap Motion controllers (see “Leaping into the Gesture Control Era”). Microsoft hasn’t yet announced that the Windows compatible version of Kinect (see “Microsoft’s Plan to Bring About the Gesture Control Era”) is to be upgraded to match the Xbox one, but it seems inevitable.

The new Kinect is like the old one in that it uses infrared light and an infrared camera to sense motion and depth, but in a fundamentally different way. The original Kinect projects an infrared pattern onto a room and looks for distortions in that pattern to figure out the shape of objects and people and how they are moving (see “Gestural Interfaces”). The new Kinect uses a more advanced technique called “time of flight,” the same principle underlying radar and sonar. An infrared light flashes rapidly to illuminate whatever is in front of the sensor, and the pixels of an infrared camera watch for each pulse to reflect back. The time it takes for light to reach a particular pixel reveals how far away the object it bounced off is.

Keep Reading

Most Popular

Geoffrey Hinton tells us why he’s now scared of the tech he helped build

“I have suddenly switched my views on whether these things are going to be more intelligent than us.”

Meet the people who use Notion to plan their whole lives

The workplace tool’s appeal extends far beyond organizing work projects. Many users find it’s just as useful for managing their free time.

Learning to code isn’t enough

Historically, learn-to-code efforts have provided opportunities for the few, but new efforts are aiming to be inclusive.

Deep learning pioneer Geoffrey Hinton has quit Google

Hinton will be speaking at EmTech Digital on Wednesday.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.