Skip to Content

With Its New Photo Filter, Facebook Announces Its Plan to Have AI Invade Your Phone

The tool is fun, but it’s also a mission statement for the social networking giant.
November 8, 2016

Facebook has announced a new AI-powered filter for your photos—a modest move, but one that signals the company’s grand ambitions to bake artificial intelligence into every aspect of its business.

The new software uses deep neural networks to transform photographs and live video into images that resemble famous artworks. But unlike other products that achieve a similar result by farming out the calculations to algorithms running on the cloud, Facebook’s offering will actually perform all the computation on your phone.

That’s a necessity in order to apply the filters in real-time—the lag introduced by sending data across the Internet would render a cloud-based approach useless. But it’s also a massive engineering headache. Facebook engineers say that it was made possible by building two products: a lightweight AI processing system called Caffe2Go, and a specially optimized set of models that create desired artistic effects without wasting computational effort.

Essentially, the team spent a long time running its artistic AI models—not themselves a new concept—to work out where it could cut corners. That let the team shrink down the size of the model by a factor of a hundred. The result is a piece of AI software that can run on a smartphone to provide convincing results, while also remaining fast and unaffected by the reliability of a data connection.

It’s certainly not the first company to shoehorn AI into a software running on a handset. Google’s translation tool and Apple’s facial recognition system do, too. But the new Facebook software is impressive because it replicates a process that’s usually so resource-intensive.

In fact the way in which Facebook has approached the problem shows that, really, the image filter isn’t the big news here. Instead, with Caffe2Go it appears to have figured out a path toward introducing on-device AI into far more products. That might include gesture or image recognition in the near future, and it’s not difficult to imagine a similar approach being used to power speech recognition or sound synthesis.

Indeed, in an enthusiastic post about the company’s AI ambitions published today, Mike Schroepfer, the chief technology officer at Facebook, expounds about how machine learning research is reaching all corners of the organization. The company is using computer vision tools to optimize Internet access, say, and speech recognition to create more realistic VR avatars.

He also points out some numbers that quantify the rapid pace of AI adoption. It’s created a data-sharing tool called AutoML, for instance, that allows all of its machine learning modes—of which it apparently tests more than 300,000 each month—to automatically update each other with new learnings. And it’s now running twice as many AI experiments per month than it was six months ago.

There are still some major hurdles facing AI systems, of course—among them providing a genuine understanding of language, the ability to make sense of how the real world looks, and the capacity to predict how real physical situations play out. Schroepfer acknowledges those as the next big challenges for AI research. It’ll be a while before those kinds of smarts make it to your phone—but Facebook is certainly eyeing the possibility.

(Read more: Facebook, Facebook Code, “AI’s Language Problem,” “Apple Lags Behind Google and Facebook on AI,” “Apple Rolls Out Privacy-Sensitive Artificial Intelligence”)

Keep Reading

Most Popular

AGI is just chatter for now concept
AGI is just chatter for now concept

The hype around DeepMind’s new AI model misses what’s actually cool about it

Some worry that the chatter about these tools is doing the whole field a disservice.

Workers disinfect the street outside Shijiazhuang Railway Station
Workers disinfect the street outside Shijiazhuang Railway Station

Why China is still obsessed with disinfecting everything

Most public health bodies dealing with covid have long since moved on from the idea of surface transmission. China’s didn’t—and that helps it control the narrative about the disease’s origins and danger.

Europe's AI Act concept
Europe's AI Act concept

A quick guide to the most important AI law you’ve never heard of

The European Union is planning new legislation aimed at curbing the worst harms associated with artificial intelligence.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.