Skip to Content

With Its New Photo Filter, Facebook Announces Its Plan to Have AI Invade Your Phone

The tool is fun, but it’s also a mission statement for the social networking giant.
November 8, 2016

Facebook has announced a new AI-powered filter for your photos—a modest move, but one that signals the company’s grand ambitions to bake artificial intelligence into every aspect of its business.

The new software uses deep neural networks to transform photographs and live video into images that resemble famous artworks. But unlike other products that achieve a similar result by farming out the calculations to algorithms running on the cloud, Facebook’s offering will actually perform all the computation on your phone.

That’s a necessity in order to apply the filters in real-time—the lag introduced by sending data across the Internet would render a cloud-based approach useless. But it’s also a massive engineering headache. Facebook engineers say that it was made possible by building two products: a lightweight AI processing system called Caffe2Go, and a specially optimized set of models that create desired artistic effects without wasting computational effort.

Essentially, the team spent a long time running its artistic AI models—not themselves a new concept—to work out where it could cut corners. That let the team shrink down the size of the model by a factor of a hundred. The result is a piece of AI software that can run on a smartphone to provide convincing results, while also remaining fast and unaffected by the reliability of a data connection.

It’s certainly not the first company to shoehorn AI into a software running on a handset. Google’s translation tool and Apple’s facial recognition system do, too. But the new Facebook software is impressive because it replicates a process that’s usually so resource-intensive.

In fact the way in which Facebook has approached the problem shows that, really, the image filter isn’t the big news here. Instead, with Caffe2Go it appears to have figured out a path toward introducing on-device AI into far more products. That might include gesture or image recognition in the near future, and it’s not difficult to imagine a similar approach being used to power speech recognition or sound synthesis.

Indeed, in an enthusiastic post about the company’s AI ambitions published today, Mike Schroepfer, the chief technology officer at Facebook, expounds about how machine learning research is reaching all corners of the organization. The company is using computer vision tools to optimize Internet access, say, and speech recognition to create more realistic VR avatars.

He also points out some numbers that quantify the rapid pace of AI adoption. It’s created a data-sharing tool called AutoML, for instance, that allows all of its machine learning modes—of which it apparently tests more than 300,000 each month—to automatically update each other with new learnings. And it’s now running twice as many AI experiments per month than it was six months ago.

There are still some major hurdles facing AI systems, of course—among them providing a genuine understanding of language, the ability to make sense of how the real world looks, and the capacity to predict how real physical situations play out. Schroepfer acknowledges those as the next big challenges for AI research. It’ll be a while before those kinds of smarts make it to your phone—but Facebook is certainly eyeing the possibility.

(Read more: Facebook, Facebook Code, “AI’s Language Problem,” “Apple Lags Behind Google and Facebook on AI,” “Apple Rolls Out Privacy-Sensitive Artificial Intelligence”)

Keep Reading

Most Popular

A Roomba recorded a woman on the toilet. How did screenshots end up on Facebook?

Robot vacuum companies say your images are safe, but a sprawling global supply chain for data from our devices creates risk.

A startup says it’s begun releasing particles into the atmosphere, in an effort to tweak the climate

Make Sunsets is already attempting to earn revenue for geoengineering, a move likely to provoke widespread criticism.

10 Breakthrough Technologies 2023

Every year, we pick the 10 technologies that matter the most right now. We look for advances that will have a big impact on our lives and break down why they matter.

These exclusive satellite images show that Saudi Arabia’s sci-fi megacity is well underway

Weirdly, any recent work on The Line doesn’t show up on Google Maps. But we got the images anyway.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.