Skip to Content
MIT News magazine

Delight or Frustration? Tough Call.

Media Lab research could help computers—and people—interpret facial expressions

Your computer doesn’t know if you’re happy or frustrated. But before long, it may do a better job of telling the difference than your friends do.

Though most people don’t realize it, a recent MIT study found that people who are frustrated often assume an expression that looks like a smile. Computers programmed with information from this research do better at differentiating smiles of delight and frustration than human observers do. The findings could pave the way for computers that assess the emotional states of their users. They could also lead to teaching tools for people who have difficulty interpreting expressions, such as those with autism.

“The goal is to help people with face-to-face communication,” says Ehsan Hoque, a graduate student in the Media Lab’s Affective Computing Group, who worked with Professor Rosalind Picard, SM ’86, ScD ’91, and graduate student Daniel McDuff on the study.

Subjects were asked to feign both delight and frustration. They were also asked to fill out a form designed to cause genuine frustration and to watch a video of a cute baby, designed to evoke delight. A webcam recorded their expressions.

When pretending to be frustrated, Hoque says, 90 percent of subjects did not smile. But when genuinely frustrated—after filling out a long online form, only to have everything disappear after they pressed “Submit”—90 percent did make a face that resembled a smile. Still photos showed little difference between the expressions, but video analysis revealed a crucial distinction: typically, happy smiles built up gradually, while frustrated smiles appeared quickly but faded fast. Though people may not know exactly what cues they are responding to, timing has a lot to do with how they interpret expressions. “Getting the timing right is very crucial if you want to be perceived as sincere and genuine with your smiles,” Hoque says.

When the MIT researchers asked a different group of people to interpret still images of these real responses, they got it right only half the time. Understanding the subtleties that reveal underlying emotions is a major goal of this research, Hoque says. “People with autism are taught that a smile means someone is happy,” he says. The research, however, shows it’s not that simple.

The analysis could be useful in creating computers that respond appropriately to the moods of their users. One goal of the Affective Computing Group’s research is to “make a computer that’s more intelligent and respectful,” Hoque says—and one that knows when you’re having a bad day. 

Keep Reading

Most Popular

still from Embodied Intelligence video
still from Embodied Intelligence video

These weird virtual creatures evolve their bodies to solve problems

They show how intelligence and body plans are closely linked—and could unlock AI for robots.

pig kidney transplant surgery
pig kidney transplant surgery

Surgeons have successfully tested a pig’s kidney in a human patient

The test, in a brain-dead patient, was very short but represents a milestone in the long quest to use animal organs in human transplants.

conceptual illustration showing various women's faces being scanned
conceptual illustration showing various women's faces being scanned

A horrifying new AI app swaps women into porn videos with a click

Deepfake researchers have long feared the day this would arrive.

thermal image of young woman wearing mask
thermal image of young woman wearing mask

The covid tech that is intimately tied to China’s surveillance state

Heat-sensing cameras and face recognition systems may help fight covid-19—but they also make us complicit in the high-tech oppression of Uyghurs.

Stay connected

Illustration by Rose WongIllustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.