Skip to Content

Digital Lip Reader

September 1, 2003

Speech recognition is a long-promised technology that’s finally beginning to deliver. But today’s best systems tend to fail when the speaker is in a noisy spot. To fix this problem, researchers are adding lip-reading to the mix.

While people rely on mouth shapes all the time to interpret speech, lip reading is no simple task for a computer. For one thing, each shape can correspond to several specific sounds. To make matters worse, mouth movements begin as much as 120 milliseconds before a sound is uttered. Humans can use other cues such as sentence context and facial expressions to overcome these difficulties, but until recently, computers lacked the processing power to do so.

Now groups at Intel, IBM, and other institutions are modifying language-processing programs to link each vocal sound to several possible mouth movements, allowing the software to make a best guess about what’s being uttered. In tests in noisy environments, adding visual information boosted speech recognition accuracy from 20 percent to 75 percent, says Ara Nefian, a senior researcher at Intel Research in Santa Clara, CA.

Initially, this is likely to be most useful to doctors and others working in noisy locations who need better accuracy from office dictation software. With this audience in mind, IBM is building a tiny camera into the boom microphone that comes with existing speech recognition software. Further down the road, researchers envision the day when your car dashboard might have a camera peering at your lips for voice-actuated controls, or your cell phone might watch what you say.

Keep Reading

Most Popular

A Roomba recorded a woman on the toilet. How did screenshots end up on Facebook?

Robot vacuum companies say your images are safe, but a sprawling global supply chain for data from our devices creates risk.

A startup says it’s begun releasing particles into the atmosphere, in an effort to tweak the climate

Make Sunsets is already attempting to earn revenue for geoengineering, a move likely to provoke widespread criticism.

10 Breakthrough Technologies 2023

Every year, we pick the 10 technologies that matter the most right now. We look for advances that will have a big impact on our lives and break down why they matter.

These exclusive satellite images show that Saudi Arabia’s sci-fi megacity is well underway

Weirdly, any recent work on The Line doesn’t show up on Google Maps. But we got the images anyway.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.