Get to Grips with Virtual Objects Using This Stripped-Down Glove

Keyboards, mice, and touch screens all have their uses but have distinct limitations. They’re not particularly good for interacting with 3-D objects, such as those found in virtual reality or design programs. A company called Gest thinks its device, which slides onto the back of your hands and fingers to track their movements, provides a more natural and useful alternative.

The Gest controller has four soft, rubbery half-rings that clip onto your fingers, and a pad that straps onto the back of your hand. Each device contains 15 inexpensive sensors that measure motion with gyroscopes and accelerometers, much like those found in mobile phones. Thumb movements are inferred using data from the other sensors.

“The keyboard and mouse don’t make sense for things like augmented reality and virtual reality,” Gest cofounder Mike Pfister says. “It’s not going to succeed unless there is a really intuitive, fundamentally new input method.”

When I met Pfister in a San Francisco coffee shop, he slipped a prototype Gest controller onto each hand and began tapping on the table as if it were a keyboard. As his fingers drummed, potential words popped up on the screen. He tapped and swiped his hand to scroll through and select the correct word. “Hello, welcome to the future of interaction,” he wrote.

Gest doesn’t need you to set up an exact location that corresponds to each key. Instead the device uses the relative motion of your fingers to understand what the user is trying to do. It can also adapt to a person’s individual typing style over time based on the suggested words he or she selects.

The first version of Gest was focused on typing and built at a hackathon, after another of the company’s four cofounders, Sid Srikumar, read about a similar idea in the science fiction novel Accelerando, by British author Charles Stross. They raised some investment funding and participated in the Capital Factory startup incubator, in Austin, Texas.

Gest has since shifted to developing its devices as a way to use natural hand gestures to interact with virtual 3-D objects. Pfister says it could help creatives working with 3-D models or drawings. He believes it could also be a powerful interaction mechanism for the Oculus Rift, heading to market next year, or other virtual reality headsets. During the coffee shop demo, he made a fist and turned his hand to grab and rotate a model of a monkey head on the screen.

Research has shown that being able to interact with a virtual world seen through a headset like the Oculus Rift helps provide a strong sense that the virtual world is real. However, the mechanism to provide that interactivity is unclear.

Gest will face competition from companies with a range of different controller designs, such as Leap Motion, which uses infrared and LEDs to sense hand movement, and Facebook’s Oculus division, which will next year sell a less naturalistic handheld controller that uses a combination of buttons and hand movements (see “Oculus Hand Controls Are Not Always So Handy”).

Gest plans to ship its first devices, and launch tools for software developers to integrate them into software, before the end of 2016. The company predicts a price of $200.