Skip to Content

Toyota Tests Backseat-Driver Software That Could Take Control in Dangerous Moments

Cameras that watch where people are looking allow cars to judge when the driver is likely to miss a dangerous situation.
March 7, 2017
This modified Lexus is used by Toyota to test autonomous driving software.

Making a turn across oncoming traffic is one of the most dangerous maneuvers drivers undertake every day. Researchers at Toyota think it’s one of the situations in which a software guardian angel built into your car could save lives.

In trials at private testing grounds in the U.S., left turns are one of the first scenarios Toyota has used to test the concept of a system it has dubbed “Guardian,” which judges whether a human is about to make a dangerous mistake.

Radar and other sensors on the outside of the car monitor what’s happening around the vehicle, while cameras inside track the driver’s head movements and gaze. Software uses the sensor data to estimate when a person needs help spotting or avoiding a hazardous situation.

So far Toyota is just testing the ability of software to understand the hazards around a car and whether a person has spotted them, but the company plans to eventually make Guardian capable of taking action if a person doesn’t look ready to do so already.

“Imagine going through an intersection and you’re going to get T-boned—the right thing for the car to do is accelerate you out of it,” says Ryan Eustice, VP of autonomous driving at Toyota Research Institute, which was established in 2015 to work on robotics and automated driving (see “Toyota’s Billion Dollar Bet”). The group first said it would start developing Guardian last year (see “Toyota Joins the Race for Self-Driving Cars with an Invisible Co-Pilot”).

Eustice argues that the Guardian effort could have a widespread impact on public safety earlier than cars that fully remove driving duties from humans. Toyota is working on such technology, along with competitors like Alphabet, Ford, and Uber. But despite high-profile testing programs on public roads, Eustice and his counterparts at other companies say that truly driverless vehicles are still some years from serving the public and will initially be limited to certain routes or locales.

“We see an opportunity to deploy it sooner and more widely,” says Eustice of the backseat-driver approach. That’s because unlike full autonomy, it won’t be reliant on hyper-detailed maps and could be easily packaged into a conventional vehicle sold to consumers, he says. However, he declines to predict how soon Guardian might be ready for commercialization.

Steven Shladover, a researcher at the University of California, Berkeley, says the claim that Guardian could save lives sooner than fully autonomous vehicles makes sense. “If the driver has a 99 percent chance of detecting hazards and the automation system also has a 99 percent chance of detecting hazards, that gives the combination of the driver and system a 99.99 percent chance,” he says. “But this is much simpler and easier than designing a fully automated system that could reach that 99.99 percent level by itself.”

Getting the relationship between Guardian and humans right will be critical, though. Any mistakes it makes, such as intervening or sending a warning when a person has correctly interpreted a situation, would undermine a person’s trust in the system and could even lead to new kinds of accidents, says Shladover.

Eustice says Toyota is well aware of those challenges. “There will have to be a lot of studies to understand human acceptance,” he says. One idea he’s considering is enabling the system to talk with a driver about incidents on the road.

“If the car does intervene, it will be important for the car to explain to you why it did that, or to later say, ‘Hey, I didn’t intervene back there, but that was actually a close call,’” he says.

Keep Reading

Most Popular

10 Breakthrough Technologies 2024

Every year, we look for promising technologies poised to have a real impact on the world. Here are the advances that we think matter most right now.

Scientists are finding signals of long covid in blood. They could lead to new treatments.

Faults in a certain part of the immune system might be at the root of some long covid cases, new research suggests.

AI for everything: 10 Breakthrough Technologies 2024

Generative AI tools like ChatGPT reached mass adoption in record time, and reset the course of an entire industry.

What’s next for AI in 2024

Our writers look at the four hot trends to watch out for this year

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.