Skip to Content
77 Mass Ave

Robotic sorting gets a grip

Robots learn to pick up items and classify them.
April 25, 2018
melanie gonick/MIT

Engineers from MIT and Princeton University have developed a robotic system that may one day lend a hand in unpacking groceries and other picking and sorting tasks, from organizing products in a warehouse to clearing debris from a disaster zone.

The team’s design is based on two general operations: picking (the act of successfully grasping an object) and perceiving (the ability to recognize and classify an object, once grasped).   

The researchers engineered a standard industrial robotic arm outfitted with a custom gripper and suction cup. They then trained the robotic arm to pick novel objects from a cluttered bin, using any one of four main grasping behaviors: suctioning onto an object, either vertically or from the side; gripping the object vertically like a claw; or, for objects that lie flush against a wall, gripping vertically and then using a flexible spatula to slide between the object and the wall.

The team showed the robot images of bins cluttered with objects, captured from the robot’s vantage point. They then showed the robot which objects were graspable—with which of the four main grasping behaviors—and which were not, marking each example as a success or failure. They did this for hundreds of examples, and over time, the researchers built up a library of picking successes and failures. They incorporated this library into a deep neural network—a multilayered algorithm that enables the robot to match the current problem it faces with a successful outcome from the past.

The robotic sorting arm at work.
melanie gonick/MIT

The researchers also developed a perception system, enabling the robot to classify an object once it’s been successfully grasped. They assembled a library of product images, taken from online sources such as retailer websites, and labeled each image with the correct identification—for instance, duct tape versus masking tape. Then they developed another learning algorithm to relate the pixels in a given image to the correct label for a given object.

At the Amazon Robotics Challenge in Japan last July, the team’s two-ton robot faced off against 15 other bots in a competition to pick and stow objects from a cluttered bin. The robot stowed all 20 objects in the allotted time, earning first place in the stowing category.

“This [system] can be applied to warehouse sorting but also may be used to pick things from your kitchen cabinet or clear debris after an accident. There are many situations where picking technologies could have an impact,” says mechanical engineering professor Alberto Rodriguez. He and his colleagues at MIT and Prince-ton will present a paper detailing their system at the IEEE International Conference on Robotics and Automation in May.

Keep Reading

Most Popular

conceptual illustration showing various women's faces being scanned
conceptual illustration showing various women's faces being scanned

A horrifying new AI app swaps women into porn videos with a click

Deepfake researchers have long feared the day this would arrive.

A view of clouds illuminated by sunlight
A view of clouds illuminated by sunlight

We can’t afford to stop solar geoengineering research

It is the wrong time to take this strategy for combating climate change off the table.

Death and Jeff Bezos
Death and Jeff Bezos

Meet Altos Labs, Silicon Valley’s latest wild bet on living forever

Funders of a deep-pocketed new "rejuvenation" startup are said to include Jeff Bezos and Yuri Milner.

new GPT3 is a good student
new GPT3 is a good student

The new version of GPT-3 is much better behaved (and should be less toxic)

OpenAI has trained its flagship language model to follow instructions, making it spit out less unwanted text—but there's still a way to go.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.