We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Intelligent Machines

A Robot with Its Head in the Cloud Tackles Warehouse Picking

A dexterous new robotic gripper can manipulate unfamiliar objects, and it shares what it learns with a hive mind in the cloud.

RightHand’s robot grabs items from a bin and places them on a conveyor belt.

Hidden inside a busy industrial building in Somerville, Massachusetts, a robot arm spends its day picking up seemingly random objects—bottles of shampoo, onions, cans of shaving foam—from a conveyor belt that goes in a circle about 10 meters in diameter.

The odd-looking setup is a test bed for a system that could take on many of the mundane picking tasks currently done by hand in warehouses and fulfillment centers. And it shows how advances in robotic hardware, computer vision, and teleoperation, along with the ability for machines to learn collaboratively via the cloud, may transform warehouse fulfillment in coming years.

The new robotic picking platform, which uses a combination of a hybrid gripper and machine learning, and which was developed by a startup called RightHand Robotics, can handle a wide variety of objects faster and more reliably than existing systems.

The company launched its platform, called RightPick, at a supply chain industry event earlier this month. It is targeting fulfillment for the pharmaceutical, electronics, grocery, and apparel industries.

When I visited RightHand Robotics early this year, the company’s cofounders, Yaro Tenzer and Leif Jentoft, showed me several prototypes they had developed. Besides the conveyor-belt scenario, these included a setup designed to match that of a company that sends packages of cosmetics tailored to individual customers. The company’s system could pick a customer’s items from several bins attached to a circular carousel. They also showed me a system learning to grasp a particular object by trying, over and over again, to move items piled up in one bin to another bin.

Picking different types of objects piled into a bin may sound simple, but it remains a huge challenge for robots, especially if the objects are unfamiliar. Humans are able to guess how an occluded object looks and feels, and we apply years of grasping experience to the task. Fulfillment centers typically handle a range of products, making them difficult to automate. Amazon, for example, has only been able to automate parts of its centers so far.

RightHand’s system grabs objects using a compliant fingered hand with a suction cup at its center. A camera is  embedded in the hand to help figure out which appendage  to use and how to grasp the item. The company employs machine learning to refine its control algorithm over time, and the tricks learned by one robot are fed back to a cloud server so that they can be shared with others. It is also possible for RightHand’s engineers to log into a system remotely to solve problems, or to help a company train the robot to pick a new object.

It is difficult to gauge the reliability and speed of such a system, or to tell how it might deal with any number of awkward new objects, but it appeared capable of picking up common objects you might find in a grocery store about as fast as a person could.

Ken Goldberg, a professor at UC Berkeley and an expert on robot vision, manipulation, and learning, says it remains very difficult for robots to rummage for items in a cluttered bin. He says he is impressed by the hybrid  gripper and adds that applying machine learning via the cloud, so that every robot deployed by the company gets smarter over time, makes a lot of sense. “This is a clever mechanism,” Goldberg says. “These guys are smart.”

When will robots take over warehouse picking jobs?

Tell us what you think.

At the start of this month, RightHand received $8 million in Series A  funding. The company’s early investors include Playground Global. This Palo Alto incubator and venture fund was created by Andy Rubin, who led the creation of Google’s Android smartphone operating system and who later managed the company’s foray into robotics with the acquisition of a number of startups working on various robot technologies. 

Tenzer and Jentoft both studied in Harvard’s Biorobotics Lab, and early company employees come from robotics labs at Yale and MIT.

Over the past year or so, the company has been working with a number of large logistics companies and retailers to prove the reliability of its system. “When we saw the tech and the progress they’ve made on the business side, we got really excited,” says Mark Valdez, a partner at Playground Global. “There’s an opportunity to build a virtuous cycle and a network effect for some of these software-defined hardware products.”

Besides Amazon, many other companies are trying to develop robots capable of grasping a range of objects from a disordered pile. “This is a major frontier for robotics right now,” says Goldberg of UC Berkeley.

Learn from the humans leading the way in robotics at EmTech Next. Register Today!
June 11-12, 2019
Cambridge, MA

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to All Access Digital.
  • All Access Digital {! insider.prices.digital !}*

    {! insider.display.menuOptionsLabel !}

    The digital magazine, plus unlimited site access, our online archive, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    Digital magazine (6 bi-monthly issues)

    Access to entire PDF magazine archive dating back to 1899

    The Download: newsletter delivery each weekday to your inbox

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.