Skip to Content
77 Mass Ave

Robotic sorting gets a grip

Robots learn to pick up items and classify them.
April 25, 2018
melanie gonick/MIT

Engineers from MIT and Princeton University have developed a robotic system that may one day lend a hand in unpacking groceries and other picking and sorting tasks, from organizing products in a warehouse to clearing debris from a disaster zone.

The team’s design is based on two general operations: picking (the act of successfully grasping an object) and perceiving (the ability to recognize and classify an object, once grasped).   

The researchers engineered a standard industrial robotic arm outfitted with a custom gripper and suction cup. They then trained the robotic arm to pick novel objects from a cluttered bin, using any one of four main grasping behaviors: suctioning onto an object, either vertically or from the side; gripping the object vertically like a claw; or, for objects that lie flush against a wall, gripping vertically and then using a flexible spatula to slide between the object and the wall.

The team showed the robot images of bins cluttered with objects, captured from the robot’s vantage point. They then showed the robot which objects were graspable—with which of the four main grasping behaviors—and which were not, marking each example as a success or failure. They did this for hundreds of examples, and over time, the researchers built up a library of picking successes and failures. They incorporated this library into a deep neural network—a multilayered algorithm that enables the robot to match the current problem it faces with a successful outcome from the past.

The robotic sorting arm at work.
melanie gonick/MIT

The researchers also developed a perception system, enabling the robot to classify an object once it’s been successfully grasped. They assembled a library of product images, taken from online sources such as retailer websites, and labeled each image with the correct identification—for instance, duct tape versus masking tape. Then they developed another learning algorithm to relate the pixels in a given image to the correct label for a given object.

At the Amazon Robotics Challenge in Japan last July, the team’s two-ton robot faced off against 15 other bots in a competition to pick and stow objects from a cluttered bin. The robot stowed all 20 objects in the allotted time, earning first place in the stowing category.

“This [system] can be applied to warehouse sorting but also may be used to pick things from your kitchen cabinet or clear debris after an accident. There are many situations where picking technologies could have an impact,” says mechanical engineering professor Alberto Rodriguez. He and his colleagues at MIT and Prince-ton will present a paper detailing their system at the IEEE International Conference on Robotics and Automation in May.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.