An unusual fleet of self-driving cars will take to the road in coming months. Unlike most automated vehicles, which are programmed to deal with the situations they may encounter, these cars will have taught themselves, in simulation, how to handle tricky scenarios safely.
The cars will learn to navigate busy intersections, crowded highways, and packed rotaries using reinforcement learning, an approach inspired by the way animals learn to associate a reward with the behavior that led to it.
Mobileye, an Israeli company that provides vehicle safety systems to many carmakers, announced at CES in Las Vegas last week that it will test the approach on the road, in collaboration with the German automaker BMW and the chip company Intel, in the second half of this year.
In reinforcement learning, a computer is not hand-coded, or given specific examples to learn from; instead, it experiments, altering its own programming in light of the behavior that most reliably leads to a certain result. In the case of automated driving, the goal might be entering a rotary or merging into traffic safely and smoothly. The technique has proved an effective way of training computers to do things that are hard to capture with code, such as learning how to achieve superhuman skill at Atari video games and the board game Go.
James Maddox, director of the American Center for Mobility, a nonprofit that works with companies to develop and establish standards for connected and automated technologies, says interacting with human drivers will be a key challenge for self-driving cars. Such systems “need to learn not just from a vehicle’s experience but from other drivers as well,” he says. Mobileye is also developing a platform that would let different carmakers share the data collected by their automated cars. Ready access to that information may prove important for the technology’s progress, Maddox says.
Automated driving technology was the focus of a flurry of announcements and demonstrations at this year’s CES. Toyota showed off a self-driving concept car featuring a virtual assistant. The chipmaker Nvidia presented a powerful new system on a chip that it created for automated driving. The automotive parts maker Delphi demonstrated a self-driving Audi that it developed in collaboration with Mobileye.
Mobileye has been working on its learning system for some time. Speaking in December at an AI conference held in Barcelona, Spain, Shai Shalev-Shwartz, vice president of technology at the company, explained that reinforcement learning offers a way to equip self-driving vehicles with a range of subtler driving skills. He showed a demonstration of one situation that his company is tackling using the technique. In a simulation, at the point where two virtual highways intersect, a handful of cars simultaneously merged in opposite directions.
“We need to balance between defensive and aggressive behavior,” Shalev-Shwartz said. “If we are too defensive, we will not make progress; if we are too aggressive, we might hit other cars. We need to negotiate with other drivers. We cannot [just] follow the rules—we need to know the rules of breaking the rules.”