Intelligent Machines

Nvidia Lets You Peer Inside the Black Box of Its Self-Driving AI

In a step toward making AI more accountable, Nvidia has developed a neural network for autonomous driving that highlights what it’s focusing on.

Nvidia has developed a self-driving AI that shows you how it works.

As we explained in our latest cover story, “The Dark Secret at the Heart of AI,” some of the most powerful machine-learning techniques available result in software that is almost completely opaque, even to the engineers that build it. Approaches that provide some clues as to how an AI works will, therefore, be hugely important for building trust in a technology that looks set to revolutionize everything from medicine to manufacturing.

Nvidia provides chips that are ideal for deep learning, an especially powerful machine-learning technique (see “10 Breakthrough Technologies 2013: Deep Learning”).

Nvidia's neural network software highlights the areas it's focusing on as it makes driving decisions.

The chip maker has also been developing systems that demonstrate how an automaker might apply deep learning to autonomous driving. This includes a car that is controlled entirely by a deep-learning algorithm. Amazingly, the vehicle’s computer isn’t given any rules to follow—it simply matches input from several video cameras to the behavior of a human driver, and figures out for itself how it should drive. The only catch is that the system is so complex that it’s difficult to untangle how it actually works.

But Nvidia is working to open this black box. It has developed a way to visually highlight what the system is paying attention to. As explained in a recently published paper, the neural network architecture developed by Nvidia’s researchers is designed so that it can highlight the areas of a video picture that contribute most strongly to the behavior of the car’s deep neural network. Remarkably, the results show that the network is focusing on the edges of roads, lane markings, and parked cars—just the sort of things that a good human driver would want to pay attention to.

“What’s revolutionary about this is that we never directly told the network to care about these things,” Urs Muller, Nvidia’s chief architect for self-driving cars, wrote in a blog post.

It isn’t a complete explanation of how the neural network reasons, but it’s a good start. As Muller says: “I can’t explain everything I need the car to do, but I can show it, and now it can show me what it learned.”

This sort of approach could become increasingly important as deep learning is applied to just about any problem involving large quantities of data, including critical areas like medicine, finance, and military intelligence.

A handful of academic researchers are exploring the issue as well. For example, Jeff Clune at the University of Wyoming and Carlos Guestrin at the University of Washington (and Apple) have found ways of highlighting the parts of images that classification systems are picking up on. And Tommi Jaakola and Regina Barzilay at MIT are developing ways to provide snippets of text that help explain a conclusion drawn from large quantities of written data.

The Defense Advanced Projects Research Agency (DARPA), which does long-term research for the U.S. military, is funding several similar research efforts through a program it calls Explainable Artificial Intelligence (XAI).

Beyond the technical specifics, though, it’s fascinating to consider how this compares to human intelligence. We do all sorts of things we can’t explain fully, and the explanations we concoct are often only approximations, or “stories” about what’s going on. Given the opacity of today’s increasingly complex machine-learning methods, we may someday be forced to accept such explanations from AI, too.

(Sources: Nvidia, “The Dark Secret at the Heart of AI”, “The U.S. Military Wants Its Autonomous Machines to Explain Themselves”)

Hear more about AI at EmTech MIT 2017.

Register now
Nvidia's neural network software highlights the areas it's focusing on as it makes driving decisions.

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.