Computer Vision Algorithms Are Still Way Too Easy to Trick
AI image recognition has made some stunning advances, but as new research shows, the systems can still be tripped up by examples that would never fool a person.
Labsix, a group of MIT students who recently tricked an image classifier developed by Google to think a 3-D-printed turtle was a rifle, released a paper on Wednesday that details a different technique that could fool systems even faster. This time, however, they managed to trick a “black box,” where they only had partial information on how the system was making decisions.
The team’s new algorithm starts with an image it wants to use to trick another system—in the example from their paper, it’s a dog—and then starts altering pixels to make the image look more like the source image; in this case, skiers. As it works, the “adversarial” algorithm challenges the image recognition system with versions of the picture that quickly move into territory any human would recognize as skiers (check out the gif, above). But all the while, the algorithm maintains just the right combination of sabotaged pixels to make the system think it’s looking at a dog.
The researchers tested their method on Google’s Cloud Vision API—a good test case in part because Google has not published anything about how the computer vision software works, or even all the labels the system uses to classify images. The team says that they’ve only tried foiling Google’s system so far, but that their technique should work on other image recognition systems as well.
There are plenty of researchers working on countering adversarial examples like this, but for safety-critical uses, such as autonomous vehicles, artificial intelligence won’t be trusted until adversarial attacks are impossible, or at least much more difficult, to pull off.
Deep Dive
Artificial intelligence
DeepMind’s cofounder: Generative AI is just a phase. What’s next is interactive AI.
“This is a profound moment in the history of technology,” says Mustafa Suleyman.
Deepfakes of Chinese influencers are livestreaming 24/7
With just a few minutes of sample video and $1,000, brands never have to stop selling their products.
AI hype is built on high test scores. Those tests are flawed.
With hopes and fears about the technology running wild, it's time to agree on what it can and can't do.
You need to talk to your kid about AI. Here are 6 things you should say.
As children start back at school this week, it’s not just ChatGPT you need to be thinking about.
Stay connected
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.