We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Tom Simonite

A View from Tom Simonite

What Will Hackers Do with the New Kinect?

Upgraded robot vision will be just one of the uses for the new version of Microsoft’s gesture control camera.

  • May 21, 2013

Microsoft announced a new version of the Xbox One today, and with it an improved and essentially reinvented version of Kinect, the company’s body- and gesture-control sensor. That bodes well for Xbox gamers, but also for the community of hackers that have found so many original uses for the first Kinect, from robot vision to 3-D doodling (see “Hackers Take the Kinect to New Levels”). It seems likely that a new wave of Kinect hacking activity will begin as soon as the new device becomes available.

The new Kinect sensor can track finger movements and even a person’s pulse rate (Credit: Microsoft).

Reports from media sources like Wired, which got early access to the new Xbox and Kinect, say the new Kinect is significantly more accurate and capable of dealing with cluttered space, in part thanks to a fundamental change in how it works (more on which below). Microsoft says that the Kinect can now track the movement of fingers and facial features, not just limbs. It can keep tracking a person who is very close to other objects—for example, responding to hand gestures when a person is sitting on a couch. If a person sits down with the Xbox controller in view of the new Kinect, it will turn on the Xbox console. Kinect can also use its infrared light and camera to monitor a person’s pulse, by tracking the waves of blood in his or her face (a technique that works for any skin tone).

Those new capabilities will probably help existing Kinect hacks get better. Robots that use Kinect for 3-D vision, such as the PR2 that is used by many researchers (it’s used in this towel folding project), may see an instant vision upgrade, perhaps allowing them to handle more complex tasks (and laundry). The improved sensor may could also allow new hacks not possible before. One category could be adaptations that use Kinect for fine control of computers using hand and finger gestures, in a way similar to the Leap Motion controllers (see “Leaping into the Gesture Control Era”). Microsoft hasn’t yet announced that the Windows compatible version of Kinect (see “Microsoft’s Plan to Bring About the Gesture Control Era”) is to be upgraded to match the Xbox one, but it seems inevitable.

The new Kinect is like the old one in that it uses infrared light and an infrared camera to sense motion and depth, but in a fundamentally different way. The original Kinect projects an infrared pattern onto a room and looks for distortions in that pattern to figure out the shape of objects and people and how they are moving (see “Gestural Interfaces”). The new Kinect uses a more advanced technique called “time of flight,” the same principle underlying radar and sonar. An infrared light flashes rapidly to illuminate whatever is in front of the sensor, and the pixels of an infrared camera watch for each pulse to reflect back. The time it takes for light to reach a particular pixel reveals how far away the object it bounced off is.

Tech Obsessive?
Become an Insider to get the story behind the story — and before anyone else.

Subscribe today
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Print Subscription.
  • Print Subscription {! insider.prices.print_only !}*

    {! insider.display.menuOptionsLabel !}

    Six print issues per year plus The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Print magazine (6 bi-monthly issues)

    The Download: newsletter delivered daily

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.