Skip to Content

What Will Hackers Do with the New Kinect?

Upgraded robot vision will be just one of the uses for the new version of Microsoft’s gesture control camera.

Microsoft announced a new version of the Xbox One today, and with it an improved and essentially reinvented version of Kinect, the company’s body- and gesture-control sensor. That bodes well for Xbox gamers, but also for the community of hackers that have found so many original uses for the first Kinect, from robot vision to 3-D doodling (see “Hackers Take the Kinect to New Levels”). It seems likely that a new wave of Kinect hacking activity will begin as soon as the new device becomes available.

The new Kinect sensor can track finger movements and even a person’s pulse rate (Credit: Microsoft).

Reports from media sources like Wired, which got early access to the new Xbox and Kinect, say the new Kinect is significantly more accurate and capable of dealing with cluttered space, in part thanks to a fundamental change in how it works (more on which below). Microsoft says that the Kinect can now track the movement of fingers and facial features, not just limbs. It can keep tracking a person who is very close to other objects—for example, responding to hand gestures when a person is sitting on a couch. If a person sits down with the Xbox controller in view of the new Kinect, it will turn on the Xbox console. Kinect can also use its infrared light and camera to monitor a person’s pulse, by tracking the waves of blood in his or her face (a technique that works for any skin tone).

Those new capabilities will probably help existing Kinect hacks get better. Robots that use Kinect for 3-D vision, such as the PR2 that is used by many researchers (it’s used in this towel folding project), may see an instant vision upgrade, perhaps allowing them to handle more complex tasks (and laundry). The improved sensor may could also allow new hacks not possible before. One category could be adaptations that use Kinect for fine control of computers using hand and finger gestures, in a way similar to the Leap Motion controllers (see “Leaping into the Gesture Control Era”). Microsoft hasn’t yet announced that the Windows compatible version of Kinect (see “Microsoft’s Plan to Bring About the Gesture Control Era”) is to be upgraded to match the Xbox one, but it seems inevitable.

The new Kinect is like the old one in that it uses infrared light and an infrared camera to sense motion and depth, but in a fundamentally different way. The original Kinect projects an infrared pattern onto a room and looks for distortions in that pattern to figure out the shape of objects and people and how they are moving (see “Gestural Interfaces”). The new Kinect uses a more advanced technique called “time of flight,” the same principle underlying radar and sonar. An infrared light flashes rapidly to illuminate whatever is in front of the sensor, and the pixels of an infrared camera watch for each pulse to reflect back. The time it takes for light to reach a particular pixel reveals how far away the object it bounced off is.

Keep Reading

Most Popular

A Roomba recorded a woman on the toilet. How did screenshots end up on Facebook?

Robot vacuum companies say your images are safe, but a sprawling global supply chain for data from our devices creates risk.

A startup says it’s begun releasing particles into the atmosphere, in an effort to tweak the climate

Make Sunsets is already attempting to earn revenue for geoengineering, a move likely to provoke widespread criticism.

10 Breakthrough Technologies 2023

Every year, we pick the 10 technologies that matter the most right now. We look for advances that will have a big impact on our lives and break down why they matter.

These exclusive satellite images show that Saudi Arabia’s sci-fi megacity is well underway

Weirdly, any recent work on The Line doesn’t show up on Google Maps. But we got the images anyway.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.