AI program gets really good at navigation by developing a brain-like GPS system

An AI program trained to navigate through a virtual maze has unexpectedly developed an architecture that resembles the neural “GPS system” found inside a brain. The AI was then able to find its way around the maze with unprecedented skill.

The discovery comes from DeepMind, a UK company owned by Alphabet and dedicated to advancing general artificial intelligence.

The work, published in the journal Nature, hints at how artificial neural networks, which are themselves inspired by biology, might be used to explore aspects of the brain that remain mysterious. But this idea should be treated with some caution, since there is much we do not know about how the brain works, and since the functioning of artificial neural networks is also often hard to explain.

Researchers at DeepMind set out to train an artificial neural network to mimic path integration, a method animals use to calculate their movement through a space. The researchers trained a neural network with a feedback loop to navigate a maze by feeding it examples of the routes taken by mice traversing a real maze.

The team found that the neural network developed something similar to the “grid cells” found in a biological brain. These cells, arranged in a triangular grid, seem to provide a way for an animal to position itself in physical space. Grid cells were first identified in 2005, and the scientists who found them were awarded a Nobel Prize for their discovery in 2014.

The DeepMind researchers used the trained network to navigate through unfamiliar mazes by adding reinforcement learning to their approach. They found that the newly trained network could navigate far more effectively than any previous AI system, and that it explored its space more like a real animal.

Neural networks can be used to do many useful things, but until now they have not proved especially good at navigation.

“This study is a compelling demonstration that deep learning can be of value for tasks that depend not just on perceptual abilities but also on higher cognitive functions—in this case, spatial navigation,” says Francesco Savelli, a neuroscientist at Johns Hopkins University who studies grid cells, and who wrote about the research in a related Nature paper.

The research suggests that grid cells  play a fundamental role in how animals—including humans—find their way around the world. This discovery might eventually have significant practical benefits, like helping robots navigate through unfamiliar buildings more easily.

“Our work is building artificial general intelligence, and we think navigation is a fundamental piece of that,” says Andrea Banino, one of the DeepMind team members.

His colleague Dharshan Kumaran says the next step is to get the AI agents to learn more complex navigation skills. “We are thinking of more challenging environments,” he says.

DeepMind has previously demonstrated some remarkable progress in machine learning, including programs capable of learning how to play video games, as well as board games like Go and chess, with superhuman skill. These achievements also relied on training very large, or deep, artificial neural networks.

According to Demis Hassabis, cofounder and CEO of DeepMind, AI research may reveal new thing about the brain. “The human brain is the only existence proof we have that the sort of general intelligence we’re trying to build is even possible,” he said in statement. “We believe that this inspiration should be a two-way street, with insights also flowing back from AI research to shed light on open questions in neuroscience.”

It isn’t clear, however, how far neural networks, which are very simplified representations of biology, will take us in explaining the brain. Several neuroscientists contacted by MIT Technology Review note that the workings of a deep neural network aren’t that much more interpretable than the functioning of a biological brain.