Skip to Content

With Its New Photo Filter, Facebook Announces Its Plan to Have AI Invade Your Phone

The tool is fun, but it’s also a mission statement for the social networking giant.
November 8, 2016

Facebook has announced a new AI-powered filter for your photos—a modest move, but one that signals the company’s grand ambitions to bake artificial intelligence into every aspect of its business.

The new software uses deep neural networks to transform photographs and live video into images that resemble famous artworks. But unlike other products that achieve a similar result by farming out the calculations to algorithms running on the cloud, Facebook’s offering will actually perform all the computation on your phone.

That’s a necessity in order to apply the filters in real-time—the lag introduced by sending data across the Internet would render a cloud-based approach useless. But it’s also a massive engineering headache. Facebook engineers say that it was made possible by building two products: a lightweight AI processing system called Caffe2Go, and a specially optimized set of models that create desired artistic effects without wasting computational effort.

Essentially, the team spent a long time running its artistic AI models—not themselves a new concept—to work out where it could cut corners. That let the team shrink down the size of the model by a factor of a hundred. The result is a piece of AI software that can run on a smartphone to provide convincing results, while also remaining fast and unaffected by the reliability of a data connection.

It’s certainly not the first company to shoehorn AI into a software running on a handset. Google’s translation tool and Apple’s facial recognition system do, too. But the new Facebook software is impressive because it replicates a process that’s usually so resource-intensive.

In fact the way in which Facebook has approached the problem shows that, really, the image filter isn’t the big news here. Instead, with Caffe2Go it appears to have figured out a path toward introducing on-device AI into far more products. That might include gesture or image recognition in the near future, and it’s not difficult to imagine a similar approach being used to power speech recognition or sound synthesis.

Indeed, in an enthusiastic post about the company’s AI ambitions published today, Mike Schroepfer, the chief technology officer at Facebook, expounds about how machine learning research is reaching all corners of the organization. The company is using computer vision tools to optimize Internet access, say, and speech recognition to create more realistic VR avatars.

He also points out some numbers that quantify the rapid pace of AI adoption. It’s created a data-sharing tool called AutoML, for instance, that allows all of its machine learning modes—of which it apparently tests more than 300,000 each month—to automatically update each other with new learnings. And it’s now running twice as many AI experiments per month than it was six months ago.

There are still some major hurdles facing AI systems, of course—among them providing a genuine understanding of language, the ability to make sense of how the real world looks, and the capacity to predict how real physical situations play out. Schroepfer acknowledges those as the next big challenges for AI research. It’ll be a while before those kinds of smarts make it to your phone—but Facebook is certainly eyeing the possibility.

(Read more: Facebook, Facebook Code, “AI’s Language Problem,” “Apple Lags Behind Google and Facebook on AI,” “Apple Rolls Out Privacy-Sensitive Artificial Intelligence”)

Keep Reading

Most Popular

DeepMind’s cofounder: Generative AI is just a phase. What’s next is interactive AI.

“This is a profound moment in the history of technology,” says Mustafa Suleyman.

What to know about this autumn’s covid vaccines

New variants will pose a challenge, but early signs suggest the shots will still boost antibody responses.

Human-plus-AI solutions mitigate security threats

With the right human oversight, emerging technologies like artificial intelligence can help keep business and customer data secure

Next slide, please: A brief history of the corporate presentation

From million-dollar slide shows to Steve Jobs’s introduction of the iPhone, a bit of show business never hurt plain old business.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.