Intelligent Machines

A Robot with Its Head in the Cloud Tackles Warehouse Picking

A dexterous new robotic gripper can manipulate unfamiliar objects, and it shares what it learns with a hive mind in the cloud.

RightHand’s robot grabs items from a bin and places them on a conveyor belt.

Hidden inside a busy industrial building in Somerville, Massachusetts, a robot arm spends its day picking up seemingly random objects—bottles of shampoo, onions, cans of shaving foam—from a conveyor belt that goes in a circle about 10 meters in diameter.

The odd-looking setup is a test bed for a system that could take on many of the mundane picking tasks currently done by hand in warehouses and fulfillment centers. And it shows how advances in robotic hardware, computer vision, and teleoperation, along with the ability for machines to learn collaboratively via the cloud, may transform warehouse fulfillment in coming years.

The new robotic picking platform, which uses a combination of a hybrid gripper and machine learning, and which was developed by a startup called RightHand Robotics, can handle a wide variety of objects faster and more reliably than existing systems.

The company launched its platform, called RightPick, at a supply chain industry event earlier this month. It is targeting fulfillment for the pharmaceutical, electronics, grocery, and apparel industries.

When I visited RightHand Robotics early this year, the company’s cofounders, Yaro Tenzer and Leif Jentoft, showed me several prototypes they had developed. Besides the conveyor-belt scenario, these included a setup designed to match that of a company that sends packages of cosmetics tailored to individual customers. The company’s system could pick a customer’s items from several bins attached to a circular carousel. They also showed me a system learning to grasp a particular object by trying, over and over again, to move items piled up in one bin to another bin.

Picking different types of objects piled into a bin may sound simple, but it remains a huge challenge for robots, especially if the objects are unfamiliar. Humans are able to guess how an occluded object looks and feels, and we apply years of grasping experience to the task. Fulfillment centers typically handle a range of products, making them difficult to automate. Amazon, for example, has only been able to automate parts of its centers so far.

RightHand’s system grabs objects using a compliant fingered hand with a suction cup at its center. A camera is  embedded in the hand to help figure out which appendage  to use and how to grasp the item. The company employs machine learning to refine its control algorithm over time, and the tricks learned by one robot are fed back to a cloud server so that they can be shared with others. It is also possible for RightHand’s engineers to log into a system remotely to solve problems, or to help a company train the robot to pick a new object.

It is difficult to gauge the reliability and speed of such a system, or to tell how it might deal with any number of awkward new objects, but it appeared capable of picking up common objects you might find in a grocery store about as fast as a person could.

Ken Goldberg, a professor at UC Berkeley and an expert on robot vision, manipulation, and learning, says it remains very difficult for robots to rummage for items in a cluttered bin. He says he is impressed by the hybrid  gripper and adds that applying machine learning via the cloud, so that every robot deployed by the company gets smarter over time, makes a lot of sense. “This is a clever mechanism,” Goldberg says. “These guys are smart.”

When will robots take over warehouse picking jobs?

Tell us what you think.

At the start of this month, RightHand received $8 million in Series A  funding. The company’s early investors include Playground Global. This Palo Alto incubator and venture fund was created by Andy Rubin, who led the creation of Google’s Android smartphone operating system and who later managed the company’s foray into robotics with the acquisition of a number of startups working on various robot technologies. 

Tenzer and Jentoft both studied in Harvard’s Biorobotics Lab, and early company employees come from robotics labs at Yale and MIT.

Over the past year or so, the company has been working with a number of large logistics companies and retailers to prove the reliability of its system. “When we saw the tech and the progress they’ve made on the business side, we got really excited,” says Mark Valdez, a partner at Playground Global. “There’s an opportunity to build a virtuous cycle and a network effect for some of these software-defined hardware products.”

Besides Amazon, many other companies are trying to develop robots capable of grasping a range of objects from a disordered pile. “This is a major frontier for robotics right now,” says Goldberg of UC Berkeley.

Weren't able to make it to EmTech Digital?
We've got you covered.

Watch videos here

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly home delivery and unlimited 24/7 access to MIT Technology Review’s website.

    The Download. Our daily newsletter of what's important in technology and innovation.

    Access to the Magazine archive. Over 24,000 articles going back to 1899 at your fingertips.

    Special Discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.