We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

  • mr. tech
  • Intelligent Machines

    To Build a Smarter Chatbot, First Teach It a Second Language

    Translation can help an algorithm’s overall language skills.

    From Alexa and Siri to countless chatbots and automated customer support lines, computers are gradually learning to talk. The only trouble is they are still very easily confused.

    A research team at Salesforce has come up with a clever way to improve the performance of many modern language programs—teaching an algorithm to speak another language before training it to do other tasks.

    Teaching machines to hold a coherent conversation remains one of the big outstanding challenges in AI because untangling the meaning of spoken or written text so often relies on a broader understanding of the world, or commonsense knowledge (see “AI’s Language Problem”).

    It turns out that training a machine-learning system to translate between two languages automatically teaches it useful things about the relationship and appropriate context of words. When this system is used as the foundation for another machine-learning system—one trained to hold a conversation, say, or to detect the sentiment in text—it performs far better than a system trained from scratch.

    “We’re taking machine translation data, and we’re basically teaching the model how to understand words and context,” says Richard Socher, chief scientist at Salesforce and an expert on applying machine learning and language.

    Sign up for Weekend Reads
    Stories from our archives that put technology in perspective

    The work is one example of how machine-learning advances may help improve the language skills of AI systems. Many deep-learning-based computer vision systems make use of some form of network pre-training, and Socher suggests that machine translation may offer a similar way to bootstrap natural language systems.

    Salesforce, an online platform for managing customer interactions across sales, marketing, and commerce, already offers a range of AI tools through its Einstein platform. These include a tool to automatically classify the sentiment of email or chat messages, and another to prioritize the leads a worker is pursuing based on his or her previous activity.

    Socher believes this discovery will help improve the natural language capabilities of the Einstein platform. “For chatbots and automating customer support, this is super useful,” he says.

    The Salesforce researchers trained a deep-learning system to translate between English and German. This involved feeding a large number of translated documents to a many-layered neural network, and tweaking the parameters of the network until it learned to produce a decent translation for itself. The system represents words using vectors, which is a common way to encode and parse meaning in text.

    The researchers then trained the bilingual network to do a variety of things: determine the sentiment of a piece of text; classify different types of questions; and answer questions. And they show that their pre-trained network exceeded the performance of one that hasn’t learned a second language.

    Machine translation data sets are particularly large, which helps with the machine-learning challenge. “There’s an important connection between translation and the rest of language,” says Bryan McCann, a researcher at Salesforce involved with the project. “[Translation data sets] are very general; they contain information that can be useful across the board for natural language processing.”

    Learn from the humans leading the way in deep learning at EmTech Next. Register Today!
    June 11-12, 2019
    Cambridge, MA

    Register now
    More from Intelligent Machines

    Artificial intelligence and robots are transforming how we work and live.

    Want more award-winning journalism? Subscribe to All Access Digital.
    • All Access Digital {! insider.prices.digital !}*

      {! insider.display.menuOptionsLabel !}

      The digital magazine, plus unlimited site access, our online archive, and The Download delivered to your email in-box each weekday.

      See details+

      12-month subscription

      Unlimited access to all our daily online news and feature stories

      Digital magazine (6 bi-monthly issues)

      Access to entire PDF magazine archive dating back to 1899

      The Download: newsletter delivery each weekday to your inbox

    You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.