Skip to Content
77 Mass Ave

Robotic sorting gets a grip

Robots learn to pick up items and classify them.
April 25, 2018
melanie gonick/MIT

Engineers from MIT and Princeton University have developed a robotic system that may one day lend a hand in unpacking groceries and other picking and sorting tasks, from organizing products in a warehouse to clearing debris from a disaster zone.

The team’s design is based on two general operations: picking (the act of successfully grasping an object) and perceiving (the ability to recognize and classify an object, once grasped).   

The researchers engineered a standard industrial robotic arm outfitted with a custom gripper and suction cup. They then trained the robotic arm to pick novel objects from a cluttered bin, using any one of four main grasping behaviors: suctioning onto an object, either vertically or from the side; gripping the object vertically like a claw; or, for objects that lie flush against a wall, gripping vertically and then using a flexible spatula to slide between the object and the wall.

The team showed the robot images of bins cluttered with objects, captured from the robot’s vantage point. They then showed the robot which objects were graspable—with which of the four main grasping behaviors—and which were not, marking each example as a success or failure. They did this for hundreds of examples, and over time, the researchers built up a library of picking successes and failures. They incorporated this library into a deep neural network—a multilayered algorithm that enables the robot to match the current problem it faces with a successful outcome from the past.

The robotic sorting arm at work.
melanie gonick/MIT

The researchers also developed a perception system, enabling the robot to classify an object once it’s been successfully grasped. They assembled a library of product images, taken from online sources such as retailer websites, and labeled each image with the correct identification—for instance, duct tape versus masking tape. Then they developed another learning algorithm to relate the pixels in a given image to the correct label for a given object.

At the Amazon Robotics Challenge in Japan last July, the team’s two-ton robot faced off against 15 other bots in a competition to pick and stow objects from a cluttered bin. The robot stowed all 20 objects in the allotted time, earning first place in the stowing category.

“This [system] can be applied to warehouse sorting but also may be used to pick things from your kitchen cabinet or clear debris after an accident. There are many situations where picking technologies could have an impact,” says mechanical engineering professor Alberto Rodriguez. He and his colleagues at MIT and Prince-ton will present a paper detailing their system at the IEEE International Conference on Robotics and Automation in May.

Keep Reading

Most Popular

This startup wants to copy you into an embryo for organ harvesting

With plans to create realistic synthetic embryos, grown in jars, Renewal Bio is on a journey to the horizon of science and ethics.

VR is as good as psychedelics at helping people reach transcendence

On key metrics, a VR experience elicited a response indistinguishable from subjects who took medium doses of LSD or magic mushrooms.

This nanoparticle could be the key to a universal covid vaccine

Ending the covid pandemic might well require a vaccine that protects against any new strains. Researchers may have found a strategy that will work.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.