Skip to Content

AI Shouldn’t Believe Everything It Hears

A new trick can fool voice-recognition systems into totally mishearing what a recording says.
mr. tech

Artificial intelligence can accurately identify objects in an image or recognize words uttered by a human, but its algorithms don’t work the same way as the human brain—and that means that they can be spoofed in ways that humans can’t.

New Scientist reports that researchers from Bar-Ilan University in Israel and Facebook’s AI team have shown that it’s possible to subtly tweak audio clips so that a human understands them as normal but a voice-recognition AI hears something totally different. The approach works by adding a quiet layer of noise to a sound clip that contains distinctive patterns a neural network will associate with other words.

The team applied its new algorithm, called Houdini, to a series of sound clips, which it then ran through Google Voice to have them transcribed. An example of an original sound clip read:

Her bearing was graceful and animated she led her son by the hand and before her walked two maids with wax lights and silver candlesticks.

When that original was passed through Google Voice it was transcribed as:

The bearing was graceful an animated she let her son by the hand and before he walks two maids with wax lights and silver candlesticks.

But the hijacked version, which via listening tests was confirmed to be indistinguishable to human ears from to the original, was transcribed as:

Mary was grateful then admitted she let her son before the walks to Mays would like slice furnace filter count six.

The team’s efforts can also be applied to other machine-learning algorithms. Tweaking images of people, it’s possible to confuse an algorithm designed to spot a human pose into thinking that a person is actually assuming a different stance, as in the image above. And by adding noise to an image of a road scene, the team was able to fool an AI algorithm usually used in autonomous-car applications for classifying features like roads and signs to instead see ... a minion. Those image-based results are similar to research published last year by researchers from the machine learning outfits OpenAI and Google Brain.

These so-called adversarial examples may seem like a strange area of research, but they can be used to stress-test machine-learning algorithms. More worrying, they could also be used nefariously, to trick AIs into seeing or hearing things that aren’t really there—convincing autonomous cars to see fake traffic on a road, or a smart speaker to hear false commands, for example. Of course, actually implementing such attacks in the wild is rather different from running them in a lab, not least because injecting the data is tricky.

What’s perhaps most interesting about all this is that finding a way to protect AIs from these kinds of tricks is actually quite difficult. As we’ve explained in the past, we don’t truly understand the inner workings of deep neural networks, and that means that we don’t know why they’re receptive to such subtle features in a voice clip or image. Until we do, adversarial examples will remain, well, adversarial for AI algorithms.

(Read more: “The Dark Secret at the Heart of AI,” “Machine Vision’s Achilles’ Heel Revealed by Google Brain Researchers”)

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.