Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Connectivity

Customer Service Bots Are Getting Better at Detecting Your Agitation

A virtual assistant that can tell you’re frustrated can slow down and help you out.

Illustration by Oscar Bolton Green

SRI International, the Silicon Valley research lab where Apple’s virtual assistant Siri was born, is working on a new generation of virtual assistants that respond to users’ emotions.

As artificial-intelligence systems such as those from Amazon, Google, and Facebook increasingly pervade our lives, there is an ever greater need for the machines to understand not only the words we speak, but what we mean as well—and emotional cues can be valuable here (see “AI’s Language Problem”).

"[Humans] change our behavior in reaction to how whoever we are talking to is feeling or what we think they’re thinking,” says William Mark, who leads SRI International's Information and Computing Sciences Division. “We want systems to be able to do the same thing."

SRI is focused first on commercial partners for the technology, called SenSay Analytics.

The system is designed to identify emotional state based on a variety of cues, including typing patterns, speech tone, facial expressions, and body movements.

SenSay could, for example, add intelligence to a pharmacy phone assistant. It might be able to tell from a patient’s pattern of speech if he or she were becoming confused, then slow down.

The machine-learning-based technology is trained on different scenarios, depending on how it will be used. The new virtual assistants can also monitor for specific words that give away a person’s mental state.

It works via text, over the phone, or in person. If someone pauses as he or she types, it could indicate confusion. In person, the system uses a camera and computer vision to pick up on facial characteristics, gaze direction, body position, gestures, and other physical signals of how a person is feeling.

While virtual assistants are becoming more common on our personal devices and in our customer service interactions, the technology is still limited. Most people still use voice-controlled interfaces for only the simplest tasks. Amazon recognizes that and is working on injecting emotional intelligence into Alexa, the virtual assistant powering its home robot Echo (“Amazon Working on Making Alexa Recognize Your Emotions”). And earlier this year Apple acquired Emotient, a startup that built technology that can analyze facial expressions, which could end up finding its way into Siri.

Since Apple acquired Siri in 2010, SRI International has been thinking about what comes next. It recently spun off Kasisto, which makes an artificial-intelligence platform trained to complete digital tasks such as transferring money or answering customer questions.

The lab’s research also extends into the Internet of things—it’s experimenting with how to bring virtual assistants to the smart home and other connected spaces.

Mark says bots can build trust by performing well, but also by explaining what they are doing or saying. That can reassure users a bot understood them and is completing the requested task.

Hear more about artificial intelligence from the experts at the EmTech Digital Conference, March 26-27, 2018 in San Francisco.

Learn more and register
Illustration by Oscar Bolton Green

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Connectivity

What it means to be constantly connected with each other and vast sources of information.

Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

    Bimonthly digital/PDF edition

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special interest publications

    Discount to MIT Technology Review events

    Special discounts to select partner offerings

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning print magazine, unlimited online access plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

  • Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.