She used reinforcement learning to better understand problem solving in both the human brain and AI systems
Kimberly Stachenfeld, a researcher at DeepMind, helped to develop a theory of the human brain region called the hippocampus, which is responsible for spatial memory and navigation. Now she’s taking her groundbreaking neuroscience work and using it to better understand artificial intelligence.
Earlier theories of the hippocampus focused on its key role in representing the past and one’s current situation, in particular one’s location in space. But Stachenfeld wanted to explain how it may also link the present to the future, by representing the current situation in terms of what it predicts about upcoming events. Using insights from an area of AI called reinforcement learning, which is based on trial and error, Stachenfeld proposed that the hippocampus uses a similar mechanism to make associations between a person’s present state (like being in one’s garage) and a desirable future state (like getting to work on time).
Stachenfeld and her team’s theory better explains how the hippocampus might play a role as a prediction system to help the brain quickly evaluate choices, like getting into a car and heading to work versus staying at home and watching TV on a weekday morning.
Now Stachenfeld is taking what she knows about the brain and aims to use it to improve AI. For instance, AI systems can efficiently learn how to achieve simple tasks—like locating the sugar in your cabinet.
But such systems are no match for the human brain, which can learn many things at once by grouping tasks together, and remembers incidental details while learning a task, which might be useful to recall while learning some other, related task. For example, we learn stirring and mixing are fundamentally similar concepts, and we can reuse similar behaviors to perform them.
If Stachenfeld can figure out how the brain does this, she believes she can will help train AI systems orders of magnitudes faster without the need for as much data.
Raluca Ada Popa
Her computer security method could protect data, even when attackers break in
Raluca Ada Popa found a fix for one of cybersecurity’s most fundamental challenges: securing computer systems without relying on firewalls to keep hackers out.
Popa’s breakthrough work started with practical database management systems that could work on encrypted data. Though encrypting data had worked for simple messaging applications like WhatsApp, it was too sluggish for systems that needed to also run calculations on the data, like databases and web applications. But Popa found a way to make computation on encrypted data practical. Today, her encryption systems work with a range of applications and provide a level of protection that firewalls cannot: even if attackers break in, they have no way to decipher the data.
Popa says her techniques allow systems to operate as if they’ve been blindfolded. They’re able to compute on data without actually seeing it—which is opening the cybersecurity field to a host of new applications. A more recent innovation of hers, Helen, can be used by hospitals to share and aggregate patient records without compromising confidentiality. Another of her systems, Opaque, secures hardware systems against potentially compromised software and is now used by such companies as IBM.
He’s making artificial intelligence better by having it play poker
Noam Brown was never very good at poker. But an artificially intelligent program he created became the first to beat the world’s top players in no-limit Texas Hold’em, the game’s most popular variant.
In recent years, machines have defeated humans in checkers, chess, and Go—known as “perfect information” games, where both players know the exact state of play at any given point. Imperfect information games like poker, where hidden cards introduce strategies like bluffing, add another level of complexity.
“When you introduce hidden information, all these past techniques just fall apart,” Brown says.
Most strategic interactions in the real world, after all, involve some form of hidden information. In the long run, Brown envisions his research leading to automated solutions to situations that are similar to hidden-information games—from managing traffic, to predicting the performance of markets, to conducting national security negotiations.
Brown’s creation, known as Libratus, is essentially three AI systems in one. The first developed a strategy for poker by playing against itself over trillions of hands during several months of training. Another refined that strategy in real time during games with humans, and a third reviewed the hands played at the end of each day of competition to identify weaknesses, like predictable betting patterns, that opponents might exploit.
In January 2017, Libratus defeated four of the world’s top players head-to-head over 120,000 hands in 20 days at a Pittsburgh casino. Because the bot didn’t learn to play by mimicking humans, it used tactics that human players typically don’t employ. Some of those strategies, like dramatically upping the ante of small pots, have begun to change how the pros play poker.
She uses data science to detect disinformation and organized harassment campaigns
Researchers have been refining methods to detect fake accounts on social media for many years. But methods created to sniff out individual bots can fail to detect more sophisticated forms of manipulation—such as state-sponsored disinformation or harassment campaigns spanning thousands of accounts over many years.
Camille François, the chief innovation officer at Graphika, says the public needs better data and models to address online manipulation without inadvertently silencing genuine voices.
François and her team use machine learning to map out online communities and the ways information flows through networks. They apply data science and investigative methods to these maps to find the telltale signatures of coordinated disinformation campaigns. Last year, François and colleagues at Oxford used this approach to help the US Senate Select Committee on Intelligence better understand Russian activities during and after the 2016 presidential election.
François says that some of her biggest breakthroughs have come from interviewing troll farm defectors and victims to understand the inner workings of troll farms. “This work is two parts technology, one part sociology,” she says. “The techniques are always evolving, and we have to stay one step ahead."
His probes could revolutionize brain treatments
Guosong Hong invented a tool for probing the brain and retina down to the resolution of individual neurons. It’s essentially a mesh-like electrode that’s small and flexible enough to be coiled into a needle and injected into the precise region researchers want to study. Brain electrodes are already being used to treat a number of conditions such as Parkinson’s disease, but these are large, rigid objects that need to be implanted by means of extensive surgery. A few weeks after these electrodes are implanted, scar tissue begins to build up, rendering them less effective over time.
The electrode Hong invented can seamlessly integrate with neural tissues without eliciting attacks from the immune system. This allows researchers to safely and reliably record live animals’ neuronal activities for nearly a year.
This tool could be applied in many areas. It could help scientists understand complex neurological processes such as the aging of the brain. It could be used to treat neurological diseases such as Alzheimer’s and epilepsy. It could help restore function in paralyzed people’s limbs. It also holds the potential for treating eye diseases such as glaucoma, if injected into the eye.
Hong envisions building interfaces between the brain and computers using this mesh, or even enabling direct brain-to-brain communication. He believes the mesh is one step further toward a world where “everyone can freely share his or her thoughts without barrier.”
Making CRISPR more flexible to treat brain disease
The gene-editing technology CRISPR has revolutionized our ability to alter DNA. Patrick Hsu is expanding its reach to RNA—the molecule responsible for translating DNA’s blueprints into proteins—and using it to tackle brain disease.
As a child, Hsu, who leads a lab at the Salk Institute for Biological Studies in California, watched the onset of dementia in his grandfather. “He would get into my bed in the middle of the night, disoriented, not knowing where he was,” he says. “It really made me think, how can I help?”
As a graduate student at Harvard University, he worked with CRISPR inventor Feng Zhang, building some of the technology’s foundational components. But he came to realize that manipulating RNA might be a more flexible technique than making permanent, and sometimes unintended, changes to the genetic code.
So after starting his own lab at Salk, Hsu developed a computer program to trawl publicly available genome data for novel proteins. He discovered a family of highly efficient and selective CRISPR enzymes targeting RNA.
Hsu is provided a tantalizing glimpse into how his technology could one day treat brain disease. He has shown that when it is applied to human neuron cells grown in the lab, it can correct RNA processing errors responsible for fronto-temporal dementia, a neurodegenerative disorder similar to Alzheimer’s that leads to a gradual decline in cognitive function.
She taught an AI to design AI chips
Azalia Mirhoseini, a research scientist at Google Brain, is using artificial intelligence itself to make better chips for artificial intelligence.
Many microchips that are used for AI weren’t specifically built for it. Most are repurposed from hardware used in video and gaming. As a result, these older, human-engineered designs leave much to be desired in terms of energy efficiency, cost, and functionality.
Mirhoseini’s system—which trained itself using trial and error, based on the AI concept of reinforcement learning—can produce chip designs in just a few hours. (The world’s top experts need several weeks.) Her AI-designed methods allow for chips that are as good as or better than those designed by human engineers: they’re faster and more energy efficient, and their total internal wire length, and therefore cost, is much lower.
Reinforcement learning is one of AI’s most promising frameworks. Software that uses it essentially teaches itself how to accomplish a task, rather than being programmed, step by step, by a human. Now, Mirhoseini says, “it’s time to use machine learning and AI to develop better computers and close the loop.”
Using AI to make cities more responsive to their residents
Liang Xu and his team have developed an AI platform that is transforming how cities across China improve public health, reduce crime, and increase efficiency in public management. Xu’s team works closely with municipal agencies in China, which provide access to troves of data such as tens of millions of health records and customs border-crossing records. After crunching all these data, stripped of identifying details, and going through other forms of training, the platform, called PADIA, is then integrated into these agencies’ computer systems.
In the cities of Chongqing and Shenzhen, this platform is helping public health authorities predict flu outbreaks with an accuracy of over 90%. A local government agency in Shenzhen has also used the software to reduce the time it takes to process documents by 95%. In several provinces, it has detected health care fraud to the tune of nearly 1 billion yuan ($150 million).
Government use of AI is stirring up debates in many countries. Xu is aware of pitfalls such as privacy breaches and job losses. But he’s also optimistic about AI’s potential to bring modern education and health care to areas that have traditionally been left out. He points out that teachers in rural areas could find answers to questions using AI’s vast knowledge base, and community health centers that lack trained staff members to interpret medical scans could use AI algorithms to help diagnose serious illnesses.