Tom Simonite

A View from Tom Simonite

What Will Hackers Do with the New Kinect?

Upgraded robot vision will be just one of the uses for the new version of Microsoft’s gesture control camera.

  • May 21, 2013

Microsoft announced a new version of the Xbox One today, and with it an improved and essentially reinvented version of Kinect, the company’s body- and gesture-control sensor. That bodes well for Xbox gamers, but also for the community of hackers that have found so many original uses for the first Kinect, from robot vision to 3-D doodling (see “Hackers Take the Kinect to New Levels”). It seems likely that a new wave of Kinect hacking activity will begin as soon as the new device becomes available.

The new Kinect sensor can track finger movements and even a person’s pulse rate (Credit: Microsoft).

Reports from media sources like Wired, which got early access to the new Xbox and Kinect, say the new Kinect is significantly more accurate and capable of dealing with cluttered space, in part thanks to a fundamental change in how it works (more on which below). Microsoft says that the Kinect can now track the movement of fingers and facial features, not just limbs. It can keep tracking a person who is very close to other objects—for example, responding to hand gestures when a person is sitting on a couch. If a person sits down with the Xbox controller in view of the new Kinect, it will turn on the Xbox console. Kinect can also use its infrared light and camera to monitor a person’s pulse, by tracking the waves of blood in his or her face (a technique that works for any skin tone).

Those new capabilities will probably help existing Kinect hacks get better. Robots that use Kinect for 3-D vision, such as the PR2 that is used by many researchers (it’s used in this towel folding project), may see an instant vision upgrade, perhaps allowing them to handle more complex tasks (and laundry). The improved sensor may could also allow new hacks not possible before. One category could be adaptations that use Kinect for fine control of computers using hand and finger gestures, in a way similar to the Leap Motion controllers (see “Leaping into the Gesture Control Era”). Microsoft hasn’t yet announced that the Windows compatible version of Kinect (see “Microsoft’s Plan to Bring About the Gesture Control Era”) is to be upgraded to match the Xbox one, but it seems inevitable.

The new Kinect is like the old one in that it uses infrared light and an infrared camera to sense motion and depth, but in a fundamentally different way. The original Kinect projects an infrared pattern onto a room and looks for distortions in that pattern to figure out the shape of objects and people and how they are moving (see “Gestural Interfaces”). The new Kinect uses a more advanced technique called “time of flight,” the same principle underlying radar and sonar. An infrared light flashes rapidly to illuminate whatever is in front of the sensor, and the pixels of an infrared camera watch for each pulse to reflect back. The time it takes for light to reach a particular pixel reveals how far away the object it bounced off is.

Tech Obsessive?
Become an Insider to get the story behind the story — and before anyone else.

Subscribe today

Uh oh–you've read all of your free articles for this month.

Insider Premium

$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe and become an Insider.

  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look. Exclusive early access to stories.

    Insider Conversations. Join in and ask questions as our editors talk to innovators from around the world.

  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning magazine and daily delivery of The Download, our newsletter of what’s important in technology and innovation.

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.