With Its New Photo Filter, Facebook Announces Its Plan to Have AI Invade Your Phone

Facebook has announced a new AI-powered filter for your photos—a modest move, but one that signals the company’s grand ambitions to bake artificial intelligence into every aspect of its business.

The new software uses deep neural networks to transform photographs and live video into images that resemble famous artworks. But unlike other products that achieve a similar result by farming out the calculations to algorithms running on the cloud, Facebook’s offering will actually perform all the computation on your phone.

That’s a necessity in order to apply the filters in real-time—the lag introduced by sending data across the Internet would render a cloud-based approach useless. But it’s also a massive engineering headache. Facebook engineers say that it was made possible by building two products: a lightweight AI processing system called Caffe2Go, and a specially optimized set of models that create desired artistic effects without wasting computational effort.

Essentially, the team spent a long time running its artistic AI models—not themselves a new concept—to work out where it could cut corners. That let the team shrink down the size of the model by a factor of a hundred. The result is a piece of AI software that can run on a smartphone to provide convincing results, while also remaining fast and unaffected by the reliability of a data connection.

It’s certainly not the first company to shoehorn AI into a software running on a handset. Google’s translation tool and Apple’s facial recognition system do, too. But the new Facebook software is impressive because it replicates a process that’s usually so resource-intensive.

In fact the way in which Facebook has approached the problem shows that, really, the image filter isn’t the big news here. Instead, with Caffe2Go it appears to have figured out a path toward introducing on-device AI into far more products. That might include gesture or image recognition in the near future, and it’s not difficult to imagine a similar approach being used to power speech recognition or sound synthesis.

Indeed, in an enthusiastic post about the company’s AI ambitions published today, Mike Schroepfer, the chief technology officer at Facebook, expounds about how machine learning research is reaching all corners of the organization. The company is using computer vision tools to optimize Internet access, say, and speech recognition to create more realistic VR avatars.

He also points out some numbers that quantify the rapid pace of AI adoption. It’s created a data-sharing tool called AutoML, for instance, that allows all of its machine learning modes—of which it apparently tests more than 300,000 each month—to automatically update each other with new learnings. And it’s now running twice as many AI experiments per month than it was six months ago.

There are still some major hurdles facing AI systems, of course—among them providing a genuine understanding of language, the ability to make sense of how the real world looks, and the capacity to predict how real physical situations play out. Schroepfer acknowledges those as the next big challenges for AI research. It’ll be a while before those kinds of smarts make it to your phone—but Facebook is certainly eyeing the possibility.

(Read more: Facebook, Facebook Code, “AI’s Language Problem,” “Apple Lags Behind Google and Facebook on AI,” “Apple Rolls Out Privacy-Sensitive Artificial Intelligence”)