Hidden inside a busy industrial building in Somerville, Massachusetts, a robot arm spends its day picking up seemingly random objects—bottles of shampoo, onions, cans of shaving foam—from a conveyor belt that goes in a circle about 10 meters in diameter.
The odd-looking setup is a test bed for a system that could take on many of the mundane picking tasks currently done by hand in warehouses and fulfillment centers. And it shows how advances in robotic hardware, computer vision, and teleoperation, along with the ability for machines to learn collaboratively via the cloud, may transform warehouse fulfillment in coming years.
The new robotic picking platform, which uses a combination of a hybrid gripper and machine learning, and which was developed by a startup called RightHand Robotics, can handle a wide variety of objects faster and more reliably than existing systems.
The company launched its platform, called RightPick, at a supply chain industry event earlier this month. It is targeting fulfillment for the pharmaceutical, electronics, grocery, and apparel industries.
When I visited RightHand Robotics early this year, the company’s cofounders, Yaro Tenzer and Leif Jentoft, showed me several prototypes they had developed. Besides the conveyor-belt scenario, these included a setup designed to match that of a company that sends packages of cosmetics tailored to individual customers. The company’s system could pick a customer’s items from several bins attached to a circular carousel. They also showed me a system learning to grasp a particular object by trying, over and over again, to move items piled up in one bin to another bin.
Picking different types of objects piled into a bin may sound simple, but it remains a huge challenge for robots, especially if the objects are unfamiliar. Humans are able to guess how an occluded object looks and feels, and we apply years of grasping experience to the task. Fulfillment centers typically handle a range of products, making them difficult to automate. Amazon, for example, has only been able to automate parts of its centers so far.
RightHand’s system grabs objects using a compliant fingered hand with a suction cup at its center. A camera is embedded in the hand to help figure out which appendage to use and how to grasp the item. The company employs machine learning to refine its control algorithm over time, and the tricks learned by one robot are fed back to a cloud server so that they can be shared with others. It is also possible for RightHand’s engineers to log into a system remotely to solve problems, or to help a company train the robot to pick a new object.
It is difficult to gauge the reliability and speed of such a system, or to tell how it might deal with any number of awkward new objects, but it appeared capable of picking up common objects you might find in a grocery store about as fast as a person could.
Ken Goldberg, a professor at UC Berkeley and an expert on robot vision, manipulation, and learning, says it remains very difficult for robots to rummage for items in a cluttered bin. He says he is impressed by the hybrid gripper and adds that applying machine learning via the cloud, so that every robot deployed by the company gets smarter over time, makes a lot of sense. “This is a clever mechanism,” Goldberg says. “These guys are smart.”
At the start of this month, RightHand received $8 million in Series A funding. The company’s early investors include Playground Global. This Palo Alto incubator and venture fund was created by Andy Rubin, who led the creation of Google’s Android smartphone operating system and who later managed the company’s foray into robotics with the acquisition of a number of startups working on various robot technologies.
Tenzer and Jentoft both studied in Harvard’s Biorobotics Lab, and early company employees come from robotics labs at Yale and MIT.
Over the past year or so, the company has been working with a number of large logistics companies and retailers to prove the reliability of its system. “When we saw the tech and the progress they’ve made on the business side, we got really excited,” says Mark Valdez, a partner at Playground Global. “There’s an opportunity to build a virtuous cycle and a network effect for some of these software-defined hardware products.”
Besides Amazon, many other companies are trying to develop robots capable of grasping a range of objects from a disordered pile. “This is a major frontier for robotics right now,” says Goldberg of UC Berkeley.
This new data poisoning tool lets artists fight back against generative AI
The tool, called Nightshade, messes up training data in ways that could cause serious damage to image-generating AI models.
The Biggest Questions: What is death?
New neuroscience is challenging our understanding of the dying process—bringing opportunities for the living.
Rogue superintelligence and merging with machines: Inside the mind of OpenAI’s chief scientist
An exclusive conversation with Ilya Sutskever on his fears for the future of AI and why they’ve made him change the focus of his life’s work.
How to fix the internet
If we want online discourse to improve, we need to move beyond the big platforms.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.