We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Brian Bergstein

A View from Brian Bergstein

I'll Take 'Massively Parallel' for $1,000, Alex

Watson, a computer program built by IBM, is about to compete on “Jeopardy.” But a bigger test will come when IBM tries to translate the machine intelligence into other domains.

  • January 13, 2011

Two years after IBM announced that it was working on a computer program that would compete on TV’s Jeopardy, the event is set to air over three days next month. IBM’s program, Watson, will have two matches against the top players the game show has ever had, Ken Jennings and Brad Rutter, with the top performer getting $1 million (IBM would donate its winnings to charity). It should make for an entertaining spectacle, but it’s not clear how meaningful Watson’s performance would be.

The set of Jeopardy.

IBM wants to show that it has dramatically advanced the state of machine intelligence since 1997, when its Deep Blue computer beat chess grandmaster Garry Kasparov. Unlike playing chess, competing on Jeopardy requires a machine to understand nuances of language, which is one of the hardest problems in computing. The Watson system won’t be connected to the Internet—it will have to analyze information it has already been fed and assess its level of confidence in whether it has come up with the right answer. (Watson also won’t listen to Alex Trebek read the “answers” for which Jeopardy players come up with questions; those will be entered into the program as text. For some insight into how Watson sorts through possible responses, you can try this interactive demonstration from the New York Times.)

Michael Littman, a machine-learning researcher at Rutgers University, says that even if Watson doesn’t win the match, merely coming close to human Jeopardy champions would be an impressive achievement, because until now computers have bested humans only at “closed-world” games that can be reduced to logical choices, like chess and backgammon. Machines have not come close to matching human skill at “open-world” games and puzzles such as crosswords, Littman says. IBM says this will have real value—that Watson possesses such linguistic and computational dexterity that it is likely to lead to useful tools in a wide variety of fields. For instance, the company says, it might become the basis of engines for answering questions in medicine, customer service, and travel applications. That has long been a goal of artificial intelligence research: to create systems that aren’t “brittle,” or good only in narrow fields.

Then again, answering Jeopardy questions well is in itself a specific field. It makes use of puns and other forms of word play in a way that other applications do not. “I’ve looked into this work a bit and it seems to be an interesting collection of special-purpose methods designed for Jeopardy questions, which tend to fall into one of a fairly small set of categories,” says Stuart Russell, an AI researcher at the University of California, Berkeley. “I don’t want to minimize the effort it took to get it working, but I am not sure it represents a significant advance for AI because the techniques do not appear to generalize easily to other forms of natural language understanding.” In other words, says Doug Lenat, who heads Cycorp—a company that has labored to infuse AI software with common-sense reasoning skills—even though Watson’s ability to understand grammar and subtleties of meaning is valuable, to succeed outside of Jeopardy it would have to be taught new sets of rules for each application. “You could do something similar for other domains, but just like in the past, the narrower the domain, the more success you’re going to have with that,” Lenat says. “This is more of a celebration of the current state of the art that natural language processing has achieved rather than an advance in that state of the art.”

Some researchers find it hard to assess Watson because IBM hasn’t revealed much of the computer science behind it. Marvin Minsky, an MIT professor and AI pioneer, says he doesn’t think it would be right to comment about the project “until IBM issues a technical report on the system, how it works, and what are its results.” Insight could come from watching the show, especially when Watson gets things wrong: its incorrect responses figure to offer clues about the program’s methods.

Lenat says he’ll be watching; he plans to have a viewing party at his company. Because even if Watson does not necessarily represent a breakthrough for AI, the attention it will get will be positive for the field, he says. “It is the 2011 analogue of what Deep Blue did in chess,” Lenat says. “It will capture human imagination.”

Couldn't make it to Cambridge? We've brought EmTech MIT to you!

Watch session videos

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Insider Plus.
  • Insider Plus {! insider.prices.plus !}*

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    What's Included

    Unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Bimonthly print magazine (6 issues per year)

    Bimonthly digital/PDF edition

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special interest publications

    Discount to MIT Technology Review events

    Special discounts to select partner offerings

    Ad-free web experience

You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.