Speech recognition has gotten sophisticated, but spoken words aren’t the only revealing noises people make. We also cough, laugh, grunt, grind teeth, breathe hard, and make other sounds that can provide clues to mood and health.
Now researchers at Cornell have built a system designed to detect body noises other than speech. The system consists of a microphone that attaches behind the user’s ear, and someday could be built into the frame of a device like Google Glass. By picking up sound waves transmitted through the skull, it can detect subtle clues about the activity or emotional state of the person wearing it—when he or she is eating, for example, or has a cold—and could make devices that track fitness or health much more accurate.
“We see ‘quantified self’ and health tracking taking off, but one unsolved problem is how to track food consumption in an automated way,” says Tanzeem Choudhury, who led the research. “This can reliably detect the onset of eating and how frequently are you eating.”
If used in enough smartphones, Choudhury’s technology might measure the health of a city. “This could be a bridge between tracking pollution and coughing and other respiratory sounds to get a better measure of how pollution is affecting the population,” she says.
Such technology also could be combined with other methods of ambient sensing in smartphones. Motorola’s latest handset, the Moto X, includes a chip that constantly listens for certain keywords (see “The Era of Ubiquitous Listening Dawns”) to determine what the phone’s owner is doing.
Rana el Kaliouby, cofounder of Affectiva, a Waltham, Massachusetts-based company that makes software that can read people’s faces to detect their emotions, says the Cornell technology could help with both mood-sensing and health. “I like their focus on nonspeech body sounds. We know from our work that these are very important and telling,” she says.
DeepMind’s cofounder: Generative AI is just a phase. What’s next is interactive AI.
“This is a profound moment in the history of technology,” says Mustafa Suleyman.
What to know about this autumn’s covid vaccines
New variants will pose a challenge, but early signs suggest the shots will still boost antibody responses.
Human-plus-AI solutions mitigate security threats
With the right human oversight, emerging technologies like artificial intelligence can help keep business and customer data secure
Next slide, please: A brief history of the corporate presentation
From million-dollar slide shows to Steve Jobs’s introduction of the iPhone, a bit of show business never hurt plain old business.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.