What if your car knew you were getting drowsy, and could take you to a coffee shop for some caffeine? Or what if it knew you were experiencing road rage and would play your favorite song to calm you down?
These are some of the questions being asked by Affectiva, a startup spun out of the MIT Media Lab that makes emotion-sensing technology. And the company says it has amassed a significant amount of data on driver behavior through a recent program, capturing everything from road rage to impromptu road-trip singalongs.
“[We want] holistic automotive AI systems that not only look outside the vehicle [... but] also look inside the vehicle,” said Taniya Mishra, the company’s director of AI research, on stage at EmTech Digital, an event organized by MIT Technology Review. “We want AI to know what is the mental and emotional state of people inside the car.”
Affectiva is running a program that pays drivers to help train its emotion-recognition system. The company sends drivers a kit including cameras and other sensors to place within their vehicles. These record a person’s facial expressions, gestures, and tone of voice on the road. That data is then labeled by trained specialists for a range of emotions, and fed into deep neural networks.
Through this approach, the team has been able to collect a rich source of naturalistic behavior and emotional data. Drivers typically have an initial adjustment period of around one and a half days during which they are self-conscious about being watched, but they quickly get used to it.
The study has also helped the team deepen their understanding of different patterns in human emotion, such as how anger is typically expressed through both facial expressions and a raised voice while happiness is more often expressed through only facial expressions.
Emotion-recognition technology is still nascent, but it could be applied in a variety of ways beyond automotives. Mishra mentioned, for example, that it could help autistic children like her three-year-old learn to socialize and engage with their peers. It could also help humans engage in more meaningful collaborations with AI “to learn, to find enjoyment, and to do tasks together,” she said.
But such technology also raises important questions about privacy. Many attendees at EmTech Digital found the idea of having their car monitor their emotions rather “creepy.” Mishra also noted that Affectiva made a conscious decision not to use the technology for any kind of security application.
How to prevent those uses, however, remains in question as more and more players enter this space. Mishra said it would be important for the tech industry, and society, to engage in open dialogue and discussion.