Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Intelligent Machines

Computers Are Getting a Dose of Common Sense

A startup called MetaMind has developed a new, improved algorithm for processing language.

Machines are getting smarter in many ways, but they are still relatively bad at understanding spoken or written language.

Talking to a machine over the phone or through a chat window can be an infuriating experience. However, several research groups, including some at large technology companies like Facebook and Google, are making steady progress toward improving computers’ language skills by building upon recent advances in machine learning.

The latest advance in this area comes from a startup called MetaMind, which has published details of a system that is more accurate than other techniques at answering questions about several lines of text that tell a story. MetaMind is developing technology designed to be capable of a range of different artificial-intelligence tasks and hopes to sell it to other companies. The startup was founded by Richard Socher, a prominent machine-learning expert who earned a PhD at Stanford.

MetaMind’s approach combines two forms of memory with an advanced neural network fed large quantities of annotated text. The first is a kind of database of concepts and facts; the other is short-term, or “episodic.” When asked a question, the system, which the company calls a dynamic memory network, will search for relevant patterns in the text that it has learned from; after finding associations, it will use its episodic memory to return to the question and look for further, more abstract patterns. This enables it to answer questions that require connecting several pieces of information.

A paper related to the work, posted online today, gives the following example:

Statements fed into the system:
Jane went to the hallway.
Mary walked to the bathroom.
Sandra went to the garden.
Daniel went back to the garden.
Sandra took the milk there.
Question:
Where is the milk?
Answer:
Garden.

Because Socher’s system was trained using data sets that covered sentiment and structure, the system can answer questions about the sentiment, or emotional tone, of text, as well as basic questions about its structure. “The cool thing is that it learns that from example,” Socher says. “It wires the episodic memory for itself.”

MetaMind tested its system using a data set released by Facebook for measuring machine performance at question-and-answer tasks. The startup’s software outperformed Facebook’s own algorithms by a narrow margin.

Making computers better at understanding everyday language could have significant implications for companies such as Facebook. It could provide a much easier way for users to find or filter information, allowing them to enter requests written as normal sentences. It could also enable Facebook to glean meaning from the information its users post on their pages and those of their friends. This could offer a powerful way to recommend information, or to place ads alongside content more thoughtfully.

The work is a sign of ongoing progress toward giving machines better language skills. Much of this work now revolves around an approach known as deep learning, which involves feeding vast amounts of data into a system that performs a series of calculations to help identify abstract features in, say, an image or an audio file.

“One thing that’s promising here is that the architecture clearly separates modules for ‘episodic’ and ‘semantic’ memory,” says Noah Smith, an associate professor at Carnegie Mellon University who studies natural language processing. “That’s been a shortcoming of many neural-network-based architectures, and in my view it’s great to see a step in the direction of models that can be inspected to allow engineering of further improvements.”

Yoshua Bengio, a professor at the University of Montreal and a leading figure in the field of deep learning, describes Socher’s system as “a novel variant” of the methods championed at Facebook and Google. Bengio adds that the work is one of many recent advances that are experiment but promising. “The potential for this type of research is very important for language understanding and natural language interfaces, which is crucial and of very high value for many companies,” he says.

Others are less impressed. Robert Berwick, a professor of computational linguistics and computer science at MIT, says Socher’s method uses established techniques and offers only incremental progress. He adds that it achieves little similarity with the way episodic memory works in the human brain, and ignores important progress that has been made in linguistics.

Socher believes the biggest significance of his work is that it makes progress toward more generalizable machine intelligence. “This idea of adding memory components is something that’s in the air right now,” he says. “A lot of people are building different kinds of models, but our goal is to try to find one model that can perform lots of different tasks.”

Tech Obsessive?
Become an Insider to get the story behind the story — and before anyone else.

Subscribe today

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

    Bimonthly digital/PDF edition

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special interest publications

    Discount to MIT Technology Review events

    Special discounts to select partner offerings

    Ad-free web experience

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.