Delight or Frustration? Tough Call.

Your computer doesn’t know if you’re happy or frustrated. But before long, it may do a better job of telling the difference than your friends do.
Though most people don’t realize it, a recent MIT study found that people who are frustrated often assume an expression that looks like a smile. Computers programmed with information from this research do better at differentiating smiles of delight and frustration than human observers do. The findings could pave the way for computers that assess the emotional states of their users. They could also lead to teaching tools for people who have difficulty interpreting expressions, such as those with autism.
“The goal is to help people with face-to-face communication,” says Ehsan Hoque, a graduate student in the Media Lab’s Affective Computing Group, who worked with Professor Rosalind Picard, SM ’86, ScD ’91, and graduate student Daniel McDuff on the study.
Subjects were asked to feign both delight and frustration. They were also asked to fill out a form designed to cause genuine frustration and to watch a video of a cute baby, designed to evoke delight. A webcam recorded their expressions.
When pretending to be frustrated, Hoque says, 90 percent of subjects did not smile. But when genuinely frustrated—after filling out a long online form, only to have everything disappear after they pressed “Submit”—90 percent did make a face that resembled a smile. Still photos showed little difference between the expressions, but video analysis revealed a crucial distinction: typically, happy smiles built up gradually, while frustrated smiles appeared quickly but faded fast. Though people may not know exactly what cues they are responding to, timing has a lot to do with how they interpret expressions. “Getting the timing right is very crucial if you want to be perceived as sincere and genuine with your smiles,” Hoque says.
When the MIT researchers asked a different group of people to interpret still images of these real responses, they got it right only half the time. Understanding the subtleties that reveal underlying emotions is a major goal of this research, Hoque says. “People with autism are taught that a smile means someone is happy,” he says. The research, however, shows it’s not that simple.
The analysis could be useful in creating computers that respond appropriately to the moods of their users. One goal of the Affective Computing Group’s research is to “make a computer that’s more intelligent and respectful,” Hoque says—and one that knows when you’re having a bad day.
Keep Reading
Most Popular
This new data poisoning tool lets artists fight back against generative AI
The tool, called Nightshade, messes up training data in ways that could cause serious damage to image-generating AI models.
The Biggest Questions: What is death?
New neuroscience is challenging our understanding of the dying process—bringing opportunities for the living.
Rogue superintelligence and merging with machines: Inside the mind of OpenAI’s chief scientist
An exclusive conversation with Ilya Sutskever on his fears for the future of AI and why they’ve made him change the focus of his life’s work.
How to fix the internet
If we want online discourse to improve, we need to move beyond the big platforms.
Stay connected
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.