Skip to Content
Artificial intelligence

This Factory Robot Learns a New Job Overnight

The world’s largest industrial robot maker, Fanuc, is developing robots that use reinforcement learning to figure out how to do things.
March 18, 2016

Inside a modest-looking office building in Tokyo lives an unusually clever industrial robot made by the Japanese company Fanuc. Give the robot a task, like picking widgets out of one box and putting them into another container, and it will spend the night figuring out how to do it. Come morning, the machine should have mastered the job as well as if it had been programmed by an expert.

Fanuc demonstrates a robot trained through reinforcement learning at the International Robot Exhibition in Tokyo in December.

Industrial robots are capable of extreme precision and speed, but they normally need to be programmed very carefully in order to do something like grasp an object. This is difficult and time-consuming, and it means that such robots can usually work only in tightly controlled environments.

Fanuc’s robot uses a technique known as deep reinforcement learning to train itself, over time, how to learn a new task. It tries picking up objects while capturing video footage of the process. Each time it succeeds or fails, it remembers how the object looked, knowledge that is used to refine a deep learning model, or a large neural network, that controls its action. Deep learning has proved to be a powerful approach in pattern recognition over the past few years.

“After eight hours or so it gets to 90 percent accuracy or above, which is almost the same as if an expert were to program it,” explains Shohei Hido, chief research officer at Preferred Networks, a Tokyo-based company specializing in machine learning. “It works overnight; the next morning it is tuned.”

Robotics researchers are testing reinforcement learning as a way to simplify and speed up the programming of robots that do factory work. Earlier this month, Google published details of its own research on using reinforcement learning to teach robots how to grasp objects.

The Fanuc robot was programmed by Preferred Networks. Fanuc, the world’s largest maker of industrial robots, invested $7.3 million in Preferred Networks in August last year. The companies demonstrated the learning robot at the International Robot Exhibition in Tokyo last December.

One of the big potential benefits of the learning approach, Hido says, is that it can be accelerated if several robots work in parallel and then share what they have learned. So eight robots working together for one hour can perform the same learning as one machine going for eight hours. “Our project is oriented to distributed learning,” Hido says. “You can imagine hundreds of factory robots sharing information.”

This form of distributed learning, sometimes called “cloud robotics,” is shaping up to be a big trend both in research and industry (see “10 Breakthrough Technologies 2016: Robots That Teach Each Other”).

“Fanuc is well placed to think about this,” says Ken Goldberg, a professor of robotics at the University of California, Berkeley, because it installs so many machines in factories around the world. He adds that cloud robotics will most likely reshape the way robots are used in the coming years.

Goldberg and colleagues (including several researchers at Google) are in fact taking this a step further by teaching robots how certain movements may be used to grasp not just specific objects but certain shapes. A paper on this work will appear at the IEEE International Conference on Robotics and Automation in May.

However, Goldberg notes, applying machine learning to robotics is challenging because controlling behavior is more complex than, say, recognizing objects in images. “Deep learning has made enormous progress in pattern recognition,” Goldberg says. “The challenge with robotics is that you’re doing something beyond that. You need to be able to generate the appropriate actions for a huge range of inputs.”

Fanuc may not be the only company developing robots that use machine learning. In 2014, the Swiss robot maker ABB invested in another AI startup called Vicarious. The fruits of that investment have yet to appear, however.

Deep Dive

Artificial intelligence

conceptual illustration showing various women's faces being scanned
conceptual illustration showing various women's faces being scanned

A horrifying new AI app swaps women into porn videos with a click

Deepfake researchers have long feared the day this would arrive.

storm front
storm front

DeepMind’s AI predicts almost exactly when and where it’s going to rain

The firm worked with UK weather forecasters to create a model that was better at making short term predictions than existing systems.

People are hiring out their faces to become deepfake-style marketing clones

AI-powered characters based on real people can star in thousands of videos and say anything, in any language.

Tentacle of Octopus
Tentacle of Octopus

What an octopus’s mind can teach us about AI’s ultimate mystery

Machine consciousness has been debated since Turing—and dismissed for being unscientific. Yet it still clouds our thinking about AIs like GPT-3.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.