Keyboards, mice, and touch screens all have their uses but have distinct limitations. They’re not particularly good for interacting with 3-D objects, such as those found in virtual reality or design programs. A company called Gest thinks its device, which slides onto the back of your hands and fingers to track their movements, provides a more natural and useful alternative.
The Gest controller has four soft, rubbery half-rings that clip onto your fingers, and a pad that straps onto the back of your hand. Each device contains 15 inexpensive sensors that measure motion with gyroscopes and accelerometers, much like those found in mobile phones. Thumb movements are inferred using data from the other sensors.
“The keyboard and mouse don’t make sense for things like augmented reality and virtual reality,” Gest cofounder Mike Pfister says. “It’s not going to succeed unless there is a really intuitive, fundamentally new input method.”
When I met Pfister in a San Francisco coffee shop, he slipped a prototype Gest controller onto each hand and began tapping on the table as if it were a keyboard. As his fingers drummed, potential words popped up on the screen. He tapped and swiped his hand to scroll through and select the correct word. “Hello, welcome to the future of interaction,” he wrote.
Gest doesn’t need you to set up an exact location that corresponds to each key. Instead the device uses the relative motion of your fingers to understand what the user is trying to do. It can also adapt to a person’s individual typing style over time based on the suggested words he or she selects.
The first version of Gest was focused on typing and built at a hackathon, after another of the company’s four cofounders, Sid Srikumar, read about a similar idea in the science fiction novel Accelerando, by British author Charles Stross. They raised some investment funding and participated in the Capital Factory startup incubator, in Austin, Texas.
Gest has since shifted to developing its devices as a way to use natural hand gestures to interact with virtual 3-D objects. Pfister says it could help creatives working with 3-D models or drawings. He believes it could also be a powerful interaction mechanism for the Oculus Rift, heading to market next year, or other virtual reality headsets. During the coffee shop demo, he made a fist and turned his hand to grab and rotate a model of a monkey head on the screen.
Research has shown that being able to interact with a virtual world seen through a headset like the Oculus Rift helps provide a strong sense that the virtual world is real. However, the mechanism to provide that interactivity is unclear.
Gest will face competition from companies with a range of different controller designs, such as Leap Motion, which uses infrared and LEDs to sense hand movement, and Facebook’s Oculus division, which will next year sell a less naturalistic handheld controller that uses a combination of buttons and hand movements (see “Oculus Hand Controls Are Not Always So Handy”).
Gest plans to ship its first devices, and launch tools for software developers to integrate them into software, before the end of 2016. The company predicts a price of $200.
The big new idea for making self-driving cars that can go anywhere
The mainstream approach to driverless cars is slow and difficult. These startups think going all-in on AI will get there faster.
Inside Charm Industrial’s big bet on corn stalks for carbon removal
The startup used plant matter and bio-oil to sequester thousands of tons of carbon. The question now is how reliable, scalable, and economical this approach will prove.
The hype around DeepMind’s new AI model misses what’s actually cool about it
Some worry that the chatter about these tools is doing the whole field a disservice.
The dark secret behind those cute AI-generated animal images
Google Brain has revealed its own image-making AI, called Imagen. But don't expect to see anything that isn't wholesome.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.