Computing at Your Fingertips

Media Lab researchers explore very different applications of finger-mounted input devices.

In April at the Association for Computing Machinery’s Computer-­Human Interaction conference, the major conference for computer interface design, two research groups from the MIT Media Lab presented prototypes of input devices that—though both worn on the finger—serve very different purposes.

A thumbnail-mounted trackpad lets users control electronic devices when their hands are full.

One device, from Pattie Maes’s Fluid Interfaces group, is a text-to-speech converter for blind users. Worn like a ring, it has a built-in camera, and it uses audio or tactile feedback to help users scan their fingers along lines of text.

The other device, a joint project between Chris Schmandt’s Living Mobile group and Joe Paradiso’s Responsive Environment group, was inspired by a cosmetic product—colorful stickers applied to fingernails, which are particularly popular in Asia. It’s a thumbnail-mounted track pad that would let users control their portable devices when their hands are full—to answer the phone or scroll through a recipe while cooking, for instance. It could also augment other input devices, allowing someone typing a text message, say, to toggle quickly between symbol sets.

The heart of the reading-assistance device is a novel algorithm that extracts visual information from text. Every time the user positions his or her finger at the start of a new line, the algorithm estimates the baseline of the text. It then tracks each word that slides past the camera. When it recognizes that a word is positioned near the center of the camera’s field of view—which reduces distortion—it isolates that word.

The baseline estimate also lets the algorithm realign the word, compensating for distortion caused by oddball camera angles, before passing it to open-source software that recognizes the characters and translates recognized words into synthesized speech.

“Anyone who needs help with reading can benefit from this,” says Roy Shilkrot, a graduate student in Maes’s group and one of the device’s designers. “We got many e-mails and requests from organizations, but also just parents of children with dyslexia, for instance.”

To build the nail-mounted device, the researchers needed to pack touch sensors, a battery, an antenna, and three separate chips—a microcontroller, a Bluetooth radio chip, and a sensor controller—into a space no larger than a thumbnail. “The hardest part was probably the antenna design,” says Artem ­Dementyev, a graduate student in Paradiso’s group. “You have to put the antenna far enough away from the chips so that it doesn’t interfere with them.”

Tech Obsessive?
Become an Insider to get the story behind the story — and before anyone else.
Subscribe today

Uh oh–you've read all five of your free articles for this month.

Insider Premium

$179.95/yr US PRICE

Want more award-winning journalism? Subscribe to Insider Plus.

  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

You've read of free articles this month.