We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

  • mr. tech
  • Intelligent Machines

    To Build a Smarter Chatbot, First Teach It a Second Language

    Translation can help an algorithm’s overall language skills.

    From Alexa and Siri to countless chatbots and automated customer support lines, computers are gradually learning to talk. The only trouble is they are still very easily confused.

    A research team at Salesforce has come up with a clever way to improve the performance of many modern language programs—teaching an algorithm to speak another language before training it to do other tasks.

    Teaching machines to hold a coherent conversation remains one of the big outstanding challenges in AI because untangling the meaning of spoken or written text so often relies on a broader understanding of the world, or commonsense knowledge (see “AI’s Language Problem”).

    It turns out that training a machine-learning system to translate between two languages automatically teaches it useful things about the relationship and appropriate context of words. When this system is used as the foundation for another machine-learning system—one trained to hold a conversation, say, or to detect the sentiment in text—it performs far better than a system trained from scratch.

    “We’re taking machine translation data, and we’re basically teaching the model how to understand words and context,” says Richard Socher, chief scientist at Salesforce and an expert on applying machine learning and language.

    Sign up for Weekend Reads
    Stories from our archives that put technology in perspective

    By signing up you agree to receive email newsletters and notifications from MIT Technology Review. You can change your preferences at any time. View our Privacy Policy for more detail.

    The work is one example of how machine-learning advances may help improve the language skills of AI systems. Many deep-learning-based computer vision systems make use of some form of network pre-training, and Socher suggests that machine translation may offer a similar way to bootstrap natural language systems.

    Salesforce, an online platform for managing customer interactions across sales, marketing, and commerce, already offers a range of AI tools through its Einstein platform. These include a tool to automatically classify the sentiment of email or chat messages, and another to prioritize the leads a worker is pursuing based on his or her previous activity.

    Socher believes this discovery will help improve the natural language capabilities of the Einstein platform. “For chatbots and automating customer support, this is super useful,” he says.

    The Salesforce researchers trained a deep-learning system to translate between English and German. This involved feeding a large number of translated documents to a many-layered neural network, and tweaking the parameters of the network until it learned to produce a decent translation for itself. The system represents words using vectors, which is a common way to encode and parse meaning in text.

    The researchers then trained the bilingual network to do a variety of things: determine the sentiment of a piece of text; classify different types of questions; and answer questions. And they show that their pre-trained network exceeded the performance of one that hasn’t learned a second language.

    Machine translation data sets are particularly large, which helps with the machine-learning challenge. “There’s an important connection between translation and the rest of language,” says Bryan McCann, a researcher at Salesforce involved with the project. “[Translation data sets] are very general; they contain information that can be useful across the board for natural language processing.”

    Keep up with the latest in deep learning at EmTech Digital.
    Don't be left behind.

    March 25-26, 2019
    San Francisco, CA

    Register now
    More from Intelligent Machines

    Artificial intelligence and robots are transforming how we work and live.

    Want more award-winning journalism? Subscribe to Insider Plus.
    • Insider Plus {! insider.prices.plus !}*

      {! insider.display.menuOptionsLabel !}

      Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

      See details+

      Print + Digital Magazine (6 bi-monthly issues)

      Unlimited online access including all articles, multimedia, and more

      The Download newsletter with top tech stories delivered daily to your inbox

      Technology Review PDF magazine archive, including articles, images, and covers dating back to 1899

      10% Discount to MIT Technology Review events and MIT Press

      Ad-free website experience

    You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.