Skip to Content
77 Mass Ave

Computing at Your Fingertips

Media Lab researchers explore very different applications of finger-mounted input devices.

In April at the Association for Computing Machinery’s Computer-­Human Interaction conference, the major conference for computer interface design, two research groups from the MIT Media Lab presented prototypes of input devices that—though both worn on the finger—serve very different purposes.

A thumbnail-mounted trackpad lets users control electronic devices when their hands are full.

One device, from Pattie Maes’s Fluid Interfaces group, is a text-to-speech converter for blind users. Worn like a ring, it has a built-in camera, and it uses audio or tactile feedback to help users scan their fingers along lines of text.

The other device, a joint project between Chris Schmandt’s Living Mobile group and Joe Paradiso’s Responsive Environment group, was inspired by a cosmetic product—colorful stickers applied to fingernails, which are particularly popular in Asia. It’s a thumbnail-mounted track pad that would let users control their portable devices when their hands are full—to answer the phone or scroll through a recipe while cooking, for instance. It could also augment other input devices, allowing someone typing a text message, say, to toggle quickly between symbol sets.

The heart of the reading-assistance device is a novel algorithm that extracts visual information from text. Every time the user positions his or her finger at the start of a new line, the algorithm estimates the baseline of the text. It then tracks each word that slides past the camera. When it recognizes that a word is positioned near the center of the camera’s field of view—which reduces distortion—it isolates that word.

The baseline estimate also lets the algorithm realign the word, compensating for distortion caused by oddball camera angles, before passing it to open-source software that recognizes the characters and translates recognized words into synthesized speech.

“Anyone who needs help with reading can benefit from this,” says Roy Shilkrot, a graduate student in Maes’s group and one of the device’s designers. “We got many e-mails and requests from organizations, but also just parents of children with dyslexia, for instance.”

To build the nail-mounted device, the researchers needed to pack touch sensors, a battery, an antenna, and three separate chips—a microcontroller, a Bluetooth radio chip, and a sensor controller—into a space no larger than a thumbnail. “The hardest part was probably the antenna design,” says Artem ­Dementyev, a graduate student in Paradiso’s group. “You have to put the antenna far enough away from the chips so that it doesn’t interfere with them.”

Keep Reading

Most Popular

computation concept
computation concept

How AI is reinventing what computers are

Three key ways artificial intelligence is changing what it means to compute.

still from Embodied Intelligence video
still from Embodied Intelligence video

These weird virtual creatures evolve their bodies to solve problems

They show how intelligence and body plans are closely linked—and could unlock AI for robots.

We reviewed three at-home covid tests. The results were mixed.

Over-the-counter coronavirus tests are finally available in the US. Some are more accurate and easier to use than others.

conceptual illustration showing various women's faces being scanned
conceptual illustration showing various women's faces being scanned

A horrifying new AI app swaps women into porn videos with a click

Deepfake researchers have long feared the day this would arrive.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.