Delight or Frustration? Tough Call.

Media Lab research could help computers—and people—interpret facial expressions

  • by David L. Chandler
  • August 21, 2012
  • Who’s not happy?: A subject exhibits happiness (left) and frustration (right).

Your computer doesn’t know if you’re happy or frustrated. But before long, it may do a better job of telling the difference than your friends do.

Though most people don’t realize it, a recent MIT study found that people who are frustrated often assume an expression that looks like a smile. Computers programmed with information from this research do better at differentiating smiles of delight and frustration than human observers do. The findings could pave the way for computers that assess the emotional states of their users. They could also lead to teaching tools for people who have difficulty interpreting expressions, such as those with autism.

“The goal is to help people with face-to-face communication,” says Ehsan Hoque, a graduate student in the Media Lab’s Affective Computing Group, who worked with Professor Rosalind Picard, SM ’86, ScD ’91, and graduate student Daniel McDuff on the study.

Subjects were asked to feign both delight and frustration. They were also asked to fill out a form designed to cause genuine frustration and to watch a video of a cute baby, designed to evoke delight. A webcam recorded their expressions.

When pretending to be frustrated, Hoque says, 90 percent of subjects did not smile. But when genuinely frustrated—after filling out a long online form, only to have everything disappear after they pressed “Submit”—90 percent did make a face that resembled a smile. Still photos showed little difference between the expressions, but video analysis revealed a crucial distinction: typically, happy smiles built up gradually, while frustrated smiles appeared quickly but faded fast. Though people may not know exactly what cues they are responding to, timing has a lot to do with how they interpret expressions. “Getting the timing right is very crucial if you want to be perceived as sincere and genuine with your smiles,” Hoque says.

When the MIT researchers asked a different group of people to interpret still images of these real responses, they got it right only half the time. Understanding the subtleties that reveal underlying emotions is a major goal of this research, Hoque says. “People with autism are taught that a smile means someone is happy,” he says. The research, however, shows it’s not that simple.

The analysis could be useful in creating computers that respond appropriately to the moods of their users. One goal of the Affective Computing Group’s research is to “make a computer that’s more intelligent and respectful,” Hoque says—and one that knows when you’re having a bad day. 

Become an MIT Technology Review Insider for in-depth analysis and unparalleled perspective.

Subscribe today

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

Want more award-winning journalism? Subscribe to Insider Premium.
  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look: exclusive early access to important stories, before they’re available to anyone else

    Insider Conversations: listen in on in-depth calls between our editors and today’s thought leaders

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.