Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Connectivity

The Next Generation of Emoji Will Be Based on Your Facial Expressions

There’s no faking your feelings with these social icons.

An app called Polygram uses AI to automatically capture your responses to friends’ photos and videos.

A new app is trying to make it simpler to help you react to photos and videos that your friends post online—it’s using AI to capture your facial expressions and automatically translate them into a range of emoji faces.

Polygram, which is free and available only for the iPhone for now, is a social app that lets you share things like photos, videos, and messages. Unlike on, say, Facebook, though, where you have a small range of pre-set reactions to choose from beyond clicking a little thumbs-up icon, Polygram uses a neural network that runs locally on the phone to figure out if you’re smiling, frowning, bored, embarrassed, surprised, and more.

Marcin Kmiec, one of Polygram’s cofounders, says the app’s AI works by capturing your face with the front-facing camera on the phone and analyzing sequences of images as quickly as possible, rather than just looking at specific points on the face like your pupils and nose. This is done directly on the phone, using the iPhone’s graphics processing unit, he says.

Subscribe to Weekend Reads
Our guide to stories in the archives that put technology in perspective.
Manage your newsletter preferences

When you look at a post in the app (for now the posts seem to consist of a suspicious amount of luxury vacation spots, fancy cars, and women in tight clothing), you see a small yellow emoji on the bottom of the display, its expression changing along with your real one. There’s a slight delay—20 milliseconds, which is just barely noticeable—between what you’re expressing on your face and what shows up in the app. The app records your response (or responses, if your expression changes a few times) in a little log of emoji on the side of the screen, along with those of others who’ve already looked at the same post.

The app is clearly meant to appeal to those who really care about how they’re perceived on social media: users can see a tally of the emoji reactions to each photo or video they post to the app, as well as details about who looked at the post, how long they looked at it, and where they’re located. This might be helpful for some mega-users, but could turn off those who are more wary about how their activity is tracked, even when it’s anonymized.

And, as many app makers know, it’s hard to succeed in social media; for every Instagram or Snapchat there are countless ones that fail to catch on. (Remember Secret? Or Path? Or Yik Yak? Or Google+?) Polygram’s founders say they’re concentrating on using the technology in their own app for now, but they also think it could be useful in other kinds of apps, like telemedicine, where it could be used to gauge a patient’s reaction to a doctor or nurse, for instance. Eventually, they say, they may release software tools that let other developers come up with their own applications for the technology.

Become an MIT Technology Review Insider for in-depth analysis and unparalleled perspective.

Subscribe today
An app called Polygram uses AI to automatically capture your responses to friends’ photos and videos.

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Connectivity

What it means to be constantly connected with each other and vast sources of information.

Want more award-winning journalism? Subscribe to Insider Basic.
  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning magazine and daily delivery of The Download, our newsletter of what’s important in technology and innovation.

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.