Skip to Content
Artificial intelligence

There’s a new way to have robots learn from their mistakes

February 28, 2018

By thinking of every incorrect action in one task as a way to do part of a different one, we can give AI the gift of hindsight.

Background: When humans mess up, they can learn several things: that an approach to a task didn’t work, but also that the method they just tried might be helpful for some other job. But when robots try to master tasks by themselves, they typically only learn  by getting a reward for every step of a job they do correctly.

Useful mistakes: IEEE Spectrum report that OpenAI, a nonprofit research company, released free software called Hindsight Experience Replay (HER) that lets an AI’s “failures” become successes. It does that by looking at how every attempt at one task could be applied to others. (The software also includes virtual environments where AIs can practice things like picking up objects or holding a pen.)

More realistic robo-training: HER doesn’t give robots rewards for getting a step of a task right—it only hands them out if the entire thing is done properly. That’s closer to how robots will learn in real life, but it usually slows training right down. Still, because every failed attempt can also get used for another job, that’s less of a problem in OpenAI’s system.

Deep Dive

Artificial intelligence

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

Google DeepMind’s new generative model makes Super Mario–like games from scratch

Genie learns how to control games by watching hours and hours of video. It could help train next-gen robots too.

What’s next for generative video

OpenAI's Sora has raised the bar for AI moviemaking. Here are four things to bear in mind as we wrap our heads around what's coming.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.