Recent breakthroughs in how accurately software can recognize images and speech came thanks to additional computing power behind a technique known as deep learning. Microsoft now reports progress on an idea that could put even greater muscle behind the technique. A practical way to power up deep learning software even more could lead to further significant advances in the intelligence of machines.
Deep learning software learns to make sense of data using rough simulations of biological neurons (see “10 Breakthrough Technologies 2013: Deep Learning”). One priority for companies such as Google, Microsoft, and Facebook investing in the technology is finding ways to train larger networks of neurons with larger collections of training data, by running the software on more powerful computers.
Using graphics processors, known as GPUs, has proven to be one of the best ways of doing that. But their price and high electricity consumption makes GPUs costly even for large companies. “It’s very expensive and challenging to build, maintain, and scale out your own training platform,” says Eric Chung, a researcher at Microsoft. Systems of GPUs used for deep learning are generally “small to medium” compared to the scale of the groups of the computers that work together to power online services, he says.
Chung is part of a project investigating a possible route to running deep learning at much greater scale. The idea is to use FPGAs, field-programmable gate arrays, chips that can be reconfigured to implement any design and that can be very power-efficient. Microsoft began using FPGAs to power parts of its Bing search engine last year, and reported it was testing their use to power the virtual neurons of deep learning in February. Chung says that the research has now advanced to using some of the most powerful FPGAs available, and that it looks like a practical way to deliver a major boost to the power of deep learning. Microsoft is using FPGAs made by Altera, a company that chip maker Intel bought in June for $17 billion, citing the potential for such chips to make corporate data centers more powerful.
Even at what Chung called the “prototyping” stage, the team found a nearly tenfold increase in the performance of a neural network attempting to identify images, compared to conventional computers without GPUs. “It could be a game-changer if we eventually manage to deploy FPGAs widely at scale, which will provide an aggregate capability that exceeds what’s possible today,” he says.
Using FPGAs does come with drawbacks, for example the work that has to be done to program them to do the work at hand. But Chung predicts the technique will allow training of neural networks of “unprecedented size and quality.”
That might help lead to improvements in things like software that can describe the content of images (see “Google Software Describes What It Sees in Images”), or understand language and show a form of common sense (“Teaching Machines to Understand Us”). Microsoft’s latest results on using FPGAs were presented at the Hot Chips conference on advances in processor performance in Cupertino, California, on Tuesday.
The gene-edited pig heart given to a dying patient was infected with a pig virus
The first transplant of a genetically-modified pig heart into a human may have ended prematurely because of a well-known—and avoidable—risk.
Meta has built a massive new language AI—and it’s giving it away for free
Facebook’s parent company is inviting researchers to pore over and pick apart the flaws in its version of GPT-3
Saudi Arabia plans to spend $1 billion a year discovering treatments to slow aging
The oil kingdom fears that its population is aging at an accelerated rate and hopes to test drugs to reverse the problem. First up might be the diabetes drug metformin.
The dark secret behind those cute AI-generated animal images
Google Brain has revealed its own image-making AI, called Imagen. But don't expect to see anything that isn't wholesome.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.