Skip to Content
Artificial intelligence

A new tool helps us understand what an AI is actually thinking

March 7, 2018

Google researchers developed a way to peer inside the minds of deep-learning systems, and the results are delightfully weird.

What they did: The team built a tool that combines several techniques to provide people with a clearer idea of how neural networks make decisions. Applied to image classification, it lets a person visualize how the network develops its understanding of what is, for instance, a kitten or a Labrador. The visualizations, above, are ... strange.

Why it matters: Deep learning is powerful—but opaque. That’s a problem if you want it to, say, drive a car for you. So being able to visualize decisions behind image recognition could help reveal why an autonomous vehicle has made a serious error. Plus, humans tend to want to know why a decision was made, even if it was correct.

But: Not everyone thinks machines needs to explain themselves. In a recent debate, Yann Lecunn, who leads Facebook’s AI research, argued that we should simply focus on their behavior. After all, we can’t always explain the decisions humans make either.

Deep Dive

Artificial intelligence

Why Meta’s latest large language model survived only three days online

Galactica was supposed to help scientists. Instead, it mindlessly spat out biased and incorrect nonsense.

DeepMind’s game-playing AI has beaten a 50-year-old record in computer science

The new version of AlphaZero discovered a faster way to do matrix multiplication, a core problem in computing that affects thousands of everyday computer tasks.

A bot that watched 70,000 hours of Minecraft could unlock AI’s next big thing

Online videos are a vast and untapped source of training data—and OpenAI says it has a new way to use it.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.