Intelligent Machines

Get to Grips with Virtual Objects Using This Stripped-Down Glove

A new controller puts sensors on your fingers so you can type without a keyboard and manipulate digital objects.

Dominant ways of interacting with computers do not provide an intuitive way to work with three-dimensional virtual objects.

Keyboards, mice, and touch screens all have their uses but have distinct limitations. They’re not particularly good for interacting with 3-D objects, such as those found in virtual reality or design programs. A company called Gest thinks its device, which slides onto the back of your hands and fingers to track their movements, provides a more natural and useful alternative.

A device that attaches sensors to your fingers and hands can track movements in high detail. Use cases include typing on any flat surface like it’s a keyboard.

The Gest controller has four soft, rubbery half-rings that clip onto your fingers, and a pad that straps onto the back of your hand. Each device contains 15 inexpensive sensors that measure motion with gyroscopes and accelerometers, much like those found in mobile phones. Thumb movements are inferred using data from the other sensors.

“The keyboard and mouse don’t make sense for things like augmented reality and virtual reality,” Gest cofounder Mike Pfister says. “It’s not going to succeed unless there is a really intuitive, fundamentally new input method.”

When I met Pfister in a San Francisco coffee shop, he slipped a prototype Gest controller onto each hand and began tapping on the table as if it were a keyboard. As his fingers drummed, potential words popped up on the screen. He tapped and swiped his hand to scroll through and select the correct word. “Hello, welcome to the future of interaction,” he wrote.

This prototype has 15 cheap motion sensors, similar to those inside a smartphone.

Gest doesn’t need you to set up an exact location that corresponds to each key. Instead the device uses the relative motion of your fingers to understand what the user is trying to do. It can also adapt to a person’s individual typing style over time based on the suggested words he or she selects.

The first version of Gest was focused on typing and built at a hackathon, after another of the company’s four cofounders, Sid Srikumar, read about a similar idea in the science fiction novel Accelerando, by British author Charles Stross. They raised some investment funding and participated in the Capital Factory startup incubator, in Austin, Texas.

Gest has since shifted to developing its devices as a way to use natural hand gestures to interact with virtual 3-D objects. Pfister says it could help creatives working with 3-D models or drawings. He believes it could also be a powerful interaction mechanism for the Oculus Rift, heading to market next year, or other virtual reality headsets. During the coffee shop demo, he made a fist and turned his hand to grab and rotate a model of a monkey head on the screen.

Research has shown that being able to interact with a virtual world seen through a headset like the Oculus Rift helps provide a strong sense that the virtual world is real. However, the mechanism to provide that interactivity is unclear.

This non-functional design prototype shows how the final device should look when it is released next year.

Gest will face competition from companies with a range of different controller designs, such as Leap Motion, which uses infrared and LEDs to sense hand movement, and Facebook’s Oculus division, which will next year sell a less naturalistic handheld controller that uses a combination of buttons and hand movements (see “Oculus Hand Controls Are Not Always So Handy”).

Gest plans to ship its first devices, and launch tools for software developers to integrate them into software, before the end of 2016. The company predicts a price of $200.

Cut off? Read unlimited articles today.

Become an Insider
Already an Insider? Log in.

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.