You know the looks–the stare that says “I’m bored” or the smile that means “Keep talking.” But many people with autism struggle to read the silent cues that tell us how to behave in conversation. Those who miss such cues may act inappropriately–for instance, droning on when it’s time to stop talking, says Rana el Kaliouby, a postdoctoral associate at MIT’s Media Lab. With colleagues Alea Teeters, a grad student, and Professor Rosalind Picard, el Kaliouby is developing a teaching tool to help.
The prototype of the ESP, or emotional-social intelligence prosthesis, consists of a small neck-mounted camera and a belt-mounted computer. Autistic people could use the device to learn about faces by watching themselves.
During conversation, the “self-cam” films the wearer’s face. The computer analyzes eye, eyebrow, mouth, and head movements and infers what they mean. It then produces a graph indicating when the wearer appears to be concentrating, thinking, agreeing, disagreeing, or expressing interest or confusion. The user can download the videos and watch them alongside the graphs.
Recording others’ faces may also be helpful, says el Kaliouby, but uninvited cameras violate others’ privacy. She says that though it is an open question whether people with autism make the same kinds of facial expressions that others do, they may learn the relationship between faces and emotions best by starting with their own. Those who feel anxious when looking at faces may feel most comfortable watching themselves. Parents and friends can also wear self-cams and offer their videos for viewing. Future versions of the system may use two self-cams, each communicating wirelessly with the other, to provide real-time instruction. One person’s ESP might send a message–“She’s losing interest”–to another person’s, which could alert its wearer with, say, a message whispered through an earphone.
The team is now testing whether its device helps high-functioning teens with autism improve their social skills over time.
Keep Reading
Most Popular
Large language models can do jaw-dropping things. But nobody knows exactly why.
And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.
The problem with plug-in hybrids? Their drivers.
Plug-in hybrids are often sold as a transition to EVs, but new data from Europe shows we’re still underestimating the emissions they produce.
Google DeepMind’s new generative model makes Super Mario–like games from scratch
Genie learns how to control games by watching hours and hours of video. It could help train next-gen robots too.
How scientists traced a mysterious covid case back to six toilets
When wastewater surveillance turns into a hunt for a single infected individual, the ethics get tricky.
Stay connected
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.