It’s a common complaint in any communication breakdown: “It’s not what you said, it’s how you said it.” For professor Sandy Pentland and his group at MIT’s Media Lab, the tone and pitch of a person’s voice, the length and frequency of pauses and speed of speech can reveal much about his or her mood.
Signal processing: Researchers at Cogito Health are developing mathematical models to detect vocal cues that may signal depression. The last graph represents the software’s confidence level in determining depression, from the beginning to the end of a vocal recording. In this example, the data shows a very high likelihood of depression.
While most speech recognition software concentrates on turning words and phrases into text, Pentland’s group is developing algorithms that analyze subtle cues in speech to determine whether someone is feeling awkward, anxious, disconnected or depressed.
Advertisement
Cogito Health, a company spun out of MIT based in Charlestown, MA, is building on Pentland’s research by developing voice-analysis software to screen for depression over the phone.
This story is only available to subscribers.
Don’t settle for half the story.
Get paywall-free access to technology news for the here and now.
For years, psychiatrists have recognized a characteristic pattern in the way that many people with clinical depression speak: slowly, quietly and often in a halting monotone. Company CEO Joshua Feast and his colleagues are training computers to recognize such vocal patterns in audio samples.
Feast says the software could be a valuable tool in managing patients with chronic diseases, which often lead to depression. As part of certain disease-management programs, nurses routinely call patients between visits to ask if they are taking their medication. However, symptoms of depression are more difficult for nurses to identify. Feast says voice analysis software could provide a natural and noninvasive way for nurses to screen for depression during routine phone calls. “If you’re a nurse and you’re trying to deal with a patient with long-term diabetes, it’s very hard to tell if a person is depressed,” says Feast. “We try to help nurses detect possible mood disorders in patients that have chronic disease.”
A few years ago, the pharmaceutical giant Pfizer developed voice-analysis software to detect early signs of Parkinson’s disease. Pfizer scientists designed the software to recognize tiny tremors in speech. Such tremors offered clues to help gauge patients’ response to various medications.
In much the same way, Cogito Health’s software detects specific patterns in vocal recordings. For example, the researchers have developed mathematical models to measure a speaker’s consistency in tone, fluidity of speech, level of vocal energy, and level of engagement in the conversation (for example, whether someone responds with “uh-huh’s” or with silence). “It listens to the pattern of speech, not the words,” says Pentland, a scientific advisor to the company. “By measuring those signals in the background, you can tell what’s going on.”
The company is conducting a large-scale trial of the software by collecting hundreds of routine phone conversations between nurses and patients, with consent from both parties. After performing follow-up questionnaires to see which patients are depressed, the researchers tested the software, to see if it could accurately identify these patients. “The trials are still running, but the results are very encouraging,” says Feast, who adds that the first results will be published in 2010.
Mark Clements, a professor of electrical and computer engineering at the Georgia Institute of Technology, has analyzed vocal patterns associated with clinical depression. His lab also uses vocal cues to identify deception and anger, as well as early signs of intoxication. Clements says the benefit of Cogito Health’s approach is that it could help untrained professionals detect signs of depression. “A trained listener could detect these types of things in a person’s voice, but it’s difficult to teach a novice,” he says. “But things that are hard to hear can be detected by a computer, and have correlations with various emotional and even physical states.”
Carl Marci, director of Social Neuroscience at the Massachusetts General Hospital’s Department of Psychiatry, and another a scientific advisor to the company, says such technology could help monitor a patient’s long-term progress. “As a psychiatrist, I see patients at most once a week, sometimes once a month,” says Marci. “They’re living their lives in between, and if I had access to a data stream that captured their natural conversations, I could monitor their response to a treatment.”
Advertisement
Cogito Health also plans to develop software to detect other mood disorders in at-risk populations. Next year, Feast says, the company will explore vocal patterns associated with post-traumatic stress in soldiers. “We want to enable early detection of psychological distress,” says Feast, “because it has been shown that early intervention makes an enormous difference in these post-trauma situations.”
