Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Computing at Your Fingertips

Media Lab researchers explore very different applications of finger-mounted input devices.

In April at the Association for Computing Machinery’s Computer-­Human Interaction conference, the major conference for computer interface design, two research groups from the MIT Media Lab presented prototypes of input devices that—though both worn on the finger—serve very different purposes.

A thumbnail-mounted trackpad lets users control electronic devices when their hands are full.

One device, from Pattie Maes’s Fluid Interfaces group, is a text-to-speech converter for blind users. Worn like a ring, it has a built-in camera, and it uses audio or tactile feedback to help users scan their fingers along lines of text.

This story is part of the July/August 2015 Issue of the MIT News Magazine
See the rest of the issue
Subscribe

The other device, a joint project between Chris Schmandt’s Living Mobile group and Joe Paradiso’s Responsive Environment group, was inspired by a cosmetic product—colorful stickers applied to fingernails, which are particularly popular in Asia. It’s a thumbnail-mounted track pad that would let users control their portable devices when their hands are full—to answer the phone or scroll through a recipe while cooking, for instance. It could also augment other input devices, allowing someone typing a text message, say, to toggle quickly between symbol sets.

The heart of the reading-assistance device is a novel algorithm that extracts visual information from text. Every time the user positions his or her finger at the start of a new line, the algorithm estimates the baseline of the text. It then tracks each word that slides past the camera. When it recognizes that a word is positioned near the center of the camera’s field of view—which reduces distortion—it isolates that word.

The baseline estimate also lets the algorithm realign the word, compensating for distortion caused by oddball camera angles, before passing it to open-source software that recognizes the characters and translates recognized words into synthesized speech.

“Anyone who needs help with reading can benefit from this,” says Roy Shilkrot, a graduate student in Maes’s group and one of the device’s designers. “We got many e-mails and requests from organizations, but also just parents of children with dyslexia, for instance.”

To build the nail-mounted device, the researchers needed to pack touch sensors, a battery, an antenna, and three separate chips—a microcontroller, a Bluetooth radio chip, and a sensor controller—into a space no larger than a thumbnail. “The hardest part was probably the antenna design,” says Artem ­Dementyev, a graduate student in Paradiso’s group. “You have to put the antenna far enough away from the chips so that it doesn’t interfere with them.”

Get stories like this before anyone else with First Look.

Subscribe today
Already a Premium subscriber? Log in.
Next in MIT News
Want more award-winning journalism? Subscribe to Insider Online Only.
  • Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.