Skip to Content
MIT News magazine

Delight or Frustration? Tough Call.

Media Lab research could help computers—and people—interpret facial expressions

Your computer doesn’t know if you’re happy or frustrated. But before long, it may do a better job of telling the difference than your friends do.

Though most people don’t realize it, a recent MIT study found that people who are frustrated often assume an expression that looks like a smile. Computers programmed with information from this research do better at differentiating smiles of delight and frustration than human observers do. The findings could pave the way for computers that assess the emotional states of their users. They could also lead to teaching tools for people who have difficulty interpreting expressions, such as those with autism.

“The goal is to help people with face-to-face communication,” says Ehsan Hoque, a graduate student in the Media Lab’s Affective Computing Group, who worked with Professor Rosalind Picard, SM ’86, ScD ’91, and graduate student Daniel McDuff on the study.

Subjects were asked to feign both delight and frustration. They were also asked to fill out a form designed to cause genuine frustration and to watch a video of a cute baby, designed to evoke delight. A webcam recorded their expressions.

When pretending to be frustrated, Hoque says, 90 percent of subjects did not smile. But when genuinely frustrated—after filling out a long online form, only to have everything disappear after they pressed “Submit”—90 percent did make a face that resembled a smile. Still photos showed little difference between the expressions, but video analysis revealed a crucial distinction: typically, happy smiles built up gradually, while frustrated smiles appeared quickly but faded fast. Though people may not know exactly what cues they are responding to, timing has a lot to do with how they interpret expressions. “Getting the timing right is very crucial if you want to be perceived as sincere and genuine with your smiles,” Hoque says.

When the MIT researchers asked a different group of people to interpret still images of these real responses, they got it right only half the time. Understanding the subtleties that reveal underlying emotions is a major goal of this research, Hoque says. “People with autism are taught that a smile means someone is happy,” he says. The research, however, shows it’s not that simple.

The analysis could be useful in creating computers that respond appropriately to the moods of their users. One goal of the Affective Computing Group’s research is to “make a computer that’s more intelligent and respectful,” Hoque says—and one that knows when you’re having a bad day. 

Keep Reading

Most Popular

Workers disinfect the street outside Shijiazhuang Railway Station
Workers disinfect the street outside Shijiazhuang Railway Station

Why China is still obsessed with disinfecting everything

Most public health bodies dealing with covid have long since moved on from the idea of surface transmission. China’s didn’t—and that helps it control the narrative about the disease’s origins and danger.

individual aging affects covid outcomes concept
individual aging affects covid outcomes concept

Anti-aging drugs are being tested as a way to treat covid

Drugs that rejuvenate our immune systems and make us biologically younger could help protect us from the disease’s worst effects.

Europe's AI Act concept
Europe's AI Act concept

A quick guide to the most important AI law you’ve never heard of

The European Union is planning new legislation aimed at curbing the worst harms associated with artificial intelligence.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.