We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Intelligent Machines

Hold Your Hand Up to Play Some Music

Via an ordinary webcam, a startup puts simple gestural control in the palm of your hand.

  • by Rachel Metz
  • April 3, 2012
  • Helping hand: Flutter lets you play or pause music in iTunes or Spotify by holding up an open palm.

When Navneet Dalal listens to music on his computer, he doesn’t want any keyboard taps or mouse-button clicks to interrupt the experience. He’d rather just hold up an open palm to play or pause the tunes.

Dalal is cofounder of Flutter, a startup that offers simple software that lets users control ordinary computer apps with such simple gestures. Flutter emerges as gesture controls are becoming increasingly popular on consumer devices: Microsoft’s Kinect motion sensor brought it to the Xbox, and the company now sells a Kinect for Windows setup that encourages users to build applications using the technology. Samsung, meanwhile, has added gesture-control technology to some of its TVs.

Unlike existing gesture-control software that is only available on certain devices or that requires special hardware, Flutter harnesses a regular webcam. Anyone can download the free software, which was released in a public “alpha” testing phase last month (it’s currently available just for Macs). Flutter users can control the play and pause functions on iTunes and Spotify by holding up a flat, open palm.

If Dalal and his cofounder, Mehul Nariyawala, have their way, this will just be the beginning. With billions of laptops, smart phones, and tablets already featuring built-in cameras, and more on the way, Flutter envisions a not-so-distant future where you’ll be able to use gestures to control all sorts of applications and games. So far, the idea is catching on. Over 20,000 people have downloaded the software since it was released in March.

Flutter doesn’t use your computer’s webcam to look for movements; instead, it determines if an open palm is present or not. The program’s algorithms process the image information and determine what action to take. Again, those actions are currently limited to pausing or playing music or videos in iTunes and music in Spotify, but Flutter plans to enable more gestures and the ability to control more video content.

The company thinks gesture controls should be simple, and they want the act of controlling an application or game to feel natural. “We treat this webcam as the eye of the machine. And our thought process is that machines and humans should interact the way two humans with eyes should interact,” Nariyawala says.

While the Kinect requires time to calibrate when you start it up and is sensitive to sunlight, Flutter doesn’t need calibration time, Dalal says, and will work in any lighting conditions except complete darkness. Its software will also ignore background movement, such as someone walking behind you, he says.

But as the “alpha” designation suggests, it’s still quite finicky. I tried out Flutter on a MacBook Pro and found that while it could definitely play and pause music on command, it often interpreted my hand moving away from the screen postgesture as an additional gesture, and would turn the music back on after I had paused it (or the reverse). Nariyawala says they’ve solved many issues with the software, but are still working on eliminating these false positives.

Flutter may have to win over some skeptics, too. Sidhant Gupta, a graduate student and researcher at the University of Washington who builds and designs interaction and sensing systems, says the software doesn’t look much different from other efforts he’s seen from Toshiba and the Israeli startup eyeSight. But he does think it might work as part of a more powerful gesture-recognition system if combined with a Kinect-like 3-D depth sensor and sonic sensor.

Dalal and Nariyawala have plenty of additions in the works. In the coming months, they intend to make it possible for Flutter to control Netflix and YouTube videos and roll out additional gesture controls such as skipping forward or back and turning volume up or down.

They showed me a demo of a two-player video game they built. Players use their hands to intercept blue shooting stars that fly across the screen, blowing them up in a shower of red and yellow. The game is simple, but it required no more startup time than the average computer application and seemed to accurately interpret its creators’ gestures.

In the future, the company envisions charging for some advanced features. Flutter may try to make money by letting app developers use its software, with Flutter taking a cut of the resulting revenue. The company might also license its software to device makers so it can be included on new gadgets. For now, though, Dalal and Nariyawala are offering the software for free as they work out the kinks.

“We want it to be as smooth and seamless as possible,” Dalal says.

Learn from the humans leading the way in intelligent machines at EmTech Next. Register Today!
June 11-12, 2019
Cambridge, MA

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Print + All Access Digital.
  • Print + All Access Digital {! insider.prices.print_digital !}*

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivery each weekday to your inbox

    The MIT Technology Review App

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.