A View from Emerging Technology from the arXiv
How Future Cars Will Predict Your Driving Maneuvers Before You Make Them
Prototype car safety systems are learning to watch the driver, as well as the road ahead, to predict future maneuvers more than three seconds before a driver makes them.
Buy a new car these days and the chances are that it will be fitted with an array of driver-assistance technologies. These can match the speed of a car ahead, manage lane changing safely, and even apply the brakes to help prevent a collision. So an interesting question is how much better these safety systems can become before the inevitable occurs and the car takes over completely.
Today we get a partial answer thanks to the work of Ashesh Jain at Cornell University and a few pals, who have developed a system that can predict a human driver’s next maneuver some three seconds before he or she makes it. This information, they say, can then be used to identify and prevent potential accidents.
The approach is straightforward in theory. Jain and co point out that a comprehensive knowledge of the driving environment, both inside and outside the car, can be used to make a pretty good guess at the driver’s immediate intentions. For example, drivers usually check the lanes next to them before making lane changes. So monitoring driver head movements helps predict whether the driver intends to change lanes in the next few seconds.
Equally, GPS and street map information show when a car is approaching an intersection where a left or right turn become an option. And speed is an important indicator too since drivers usually slow down before making a turn.
But this data is disparate. Head movement data is entirely different to speed data or map data. The difficulty is in combining and analyzing these information streams in a way that allows good predictions to be made.
The solution that Jain and co have come up with is to analyze these streams together using advanced artificial intelligence algorithms that learn to spot the telltale signs that a maneuver is impending.
The first part of their work was to gather the data needed to train their machine. Jain and co fitted a car with one camera to monitor the driver and another to monitor the road head. They also used GPS data along with street maps and a speed logger to keep track of the vehicle’s velocity.
They then gathered data from 10 different drivers who clocked up over 1,000 miles of freeway and city driving over two months. They annotated the data by hand to indicate what maneuvers were occurring. In total they identified 700 “events”: some 300 lane changes, 130 turns, and almost 300 randomly chosen instances of driving straight.
They used this data to train a number of different number-crunching machines to identify the conditions under which a driver would turn left or right, change lanes to the left of right, or just keep on driving in a straight line.
The results make for interesting reading. The best performing algorithm was able to correctly determine a future maneuver most of the time—some 90 percent of is predictions were correct. And on average it was able to make its prediction 3.5 seconds before the maneuver actually occurred.
Of course, there is more work ahead. One potential problem is to determine how well the algorithm works in the most hazardous driving conditions, particularly at night or when visibility is poor because of snow or rain storms or when the sun is low in the sky.
Accidents are more likely under these conditions so a predictive algorithm could have the most benefit. But how the system performs under these conditions is not clear.
Another question is what to do with this information once it has been gathered. How can it be used to prevent accidents? Again, it is not clear how car makers can exploit the data.
Nevertheless, the new approach could herald an interesting change in the approach to driver safety. Accurate prediction of driver maneuvers will surely help to make cars safer in the near future. Unless, of course, the self-driving car technology makes humans—and the technology to predict what they will do in future—redundant more quickly than anybody expects.
Ref: http://arxiv.org/abs/1601.00740 : Brain4Cars: Car That Knows Before You Do via Sensory-Fusion Deep Learning Architecture