Skip to Content
Artificial intelligence

This robot learns to pick up mugs by first learning a theory of mugness

March 19, 2019

For all of the recent progress in machine intelligence, robots still struggle to adapt relatively simple tasks to new situations. Take, for example, picking up a mug and hanging it on a mug rack; even small changes in a mug’s shape, size, color, and orientation can throw a robot off.

In a new paper, researchers at MIT are now proposing a new technique for helping robots generalize their learning with relatively little data. They do so by training a neural network to extract just a few key points from a mug or other object that needs to be picked up and placed, giving the robot a visual road map for how to grasp and orient it. During testing, they found that the bot only needed three key points for a mug—one on the center of its side, one on the bottom, and one on the handle—and six key points for a shoe.

Unlike previous techniques that require hundreds or even thousands of examples for a robot to learn to pick up a mug it has never seen before, this approach requires only a few dozen. The researchers were able to train the neural network on 60 scenes of mugs and 60 scenes of shoes to reach a similar level of performance. When the system initially failed to pick up high heels because there were none in the data set, they quickly fixed the problem by adding a few labeled scenes of high heels to the data.

The team hopes to use the approach to tackle bigger tasks next, like unloading a dishwasher or wiping down a kitchen counter.

This story originally appeared in our AI newsletter The Algorithm. To have it directly delivered to your inbox, sign up here for free.

Deep Dive

Artificial intelligence

The inside story of how ChatGPT was built from the people who made it

Exclusive conversations that take us behind the scenes of a cultural phenomenon.

ChatGPT is about to revolutionize the economy. We need to decide what that looks like.

New large language models will transform many jobs. Whether they will lead to widespread prosperity or not is up to us.

GPT-4 is bigger and better than ChatGPT—but OpenAI won’t say why

We got a first look at the much-anticipated big new language model from OpenAI. But this time how it works is even more deeply under wraps.

Google just launched Bard, its answer to ChatGPT—and it wants you to make it better

Under pressure from its rivals, Google is updating the way we look for information by introducing a sidekick to its search engine.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.