Software that recognizes and rates smiles was demonstrated recently at an exhibition in Tokyo, where attendees competed to outsmile one another.
Say cheese: Software developed by a Japanese company detects and rates smiles. It’s the latest addition to Omron Corporation’s OKAO Vision facial-recognition software suite.
The smile-checking technology is the latest addition to Omron Corporation’s OKAO Vision software suite, which detects faces in images and can determine the person’s gender and approximate age, or verify his or her identity from a database of faces. The smile software is Omron’s first foray into facial-expression detection and analysis, a field that could revolutionize how humans interact with machines, and with each other.
Advertisement
Omron, a Japanese electronics company, won’t say if it plans to commercialize its smile software, which was on show at Japan’s Combined Exhibition of Advanced Technologies. But spokesman James Seddon says that it could be used in digital cameras to help capture people’s broadest smiles, in market research and customer-service training, and even by mental-health professionals to evaluate patients. Sony uses similar technology in some of its newest digital cameras so that they snap pictures when people are smiling their best.
This story is only available to subscribers.
Don’t settle for half the story.
Get paywall-free access to technology news for the here and now.
Omron’s smile-measurement software picks up the hallmarks of a smile–such as narrowed eyes, an open mouth, creases around the mouth, and wrinkles turning downward around the eyes–and uses an algorithm to assess the extent of the smile and rate it on a percentage scale. The analysis is performed in real time and only takes about 44 milliseconds using a Pentium 4 3.2-gigahertz PC, Seddon says. The smile software works on images of faces as small as 60 pixels wide.
Omron engineers used about 10,000 images of human faces–some with spontaneous smiles, some with posed smiles, and others sporting different expressions–to train the software to evaluate smiles.
Omron’s smile-recognition technology is part of the company’s OKAO Vision software suite. The face-recognition component of the software analyzes patterns of light and dark in an image to determine if it includes a face. Because it’s based on contrast, Seddon says that the system performs well when dealing with poorly lit images.
Once OKAO Vision has registered a two-dimensional version of a face, it overlays a 3-D mask that allows the software to evaluate the face, even if the subject’s head is turned or she’s looking away from the camera.
Omron envisions the smile software being used in marketing, perhaps to evaluate consumers’ reactions to a new product or to an advertising campaign. A smile checker could also help train customer-service staff to meet Japan’s legendarily high standards, Seddon says.
“Clearly, it’s an interesting thing,” says Joseph Atick of L-1 Identity Solutions, based in Stamford, CT, which supplies identification technology, primarily for security applications. “If you can read people better, you can serve them better.”
A smile in isolation is easy to detect, but the bigger challenge is to develop systems that can recognize the concerto of facial actions that make up complex expressions like confusion, fear, and disgust. “You can also superimpose any other emotion on top of a smile,” says Rana el Kaliouby, a postdoctoral associate at MIT who is developing mind-reading machines. “You can have an angry smile, an interested smile–even a confused smile.”
Advertisement
“We’d like to be able to distinguish those,” says Jeffrey Cohn, director of the Affect Analysis Group, at the University of Pittsburgh.
Sophisticated facial-expression analysis could help mental-health professionals evaluate their patients and monitor their progress. Cohn is studying facial expressions in depressed patients and in people in physical pain.
El Kaliouby says that facial-expression analysis could help autistic people “who need help with real-time analysis of facial expressions and other social cues.”
Facial-expression analysis could also have security applications, although Atick is skeptical that a machine can reliably pick out a terrorist on its own. “That smacks of black magic, voodoo,” he says. “Tell me what my intent is standing in front of passport control. Maybe it’s a long flight and I’m annoyed, but I don’t intend to do harm.”
