Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

A View from Henry Lieberman

A Worthwhile Contest for Artificial Intelligence

IBM’s computer can achieve a victory even if it doesn’t beat the humans on Jeopardy.

  • February 14, 2011
The face of Watson. Credit: IBM

If IBM’s Watson machine defeats people on TV’s Jeopardy this week, does that mean that computers are smarter than humans? Maybe not. But the performance could tell us something far more interesting.

For a person, Jeopardy is a test of how much specific knowledge you know and how fast you can access it. For the computer, storing a lot of stuff and accessing it quickly are a piece of cake. What’s hard is understanding what the question is really asking for. What’s hard is the wordplay and puns in many Jeopardy questions. What’s hard is that you can’t prepare a database in advance that is certain to have the subject matter. Type a Jeopardy question to a search engine and you’re unlikely to get the answer directly.

To me, the most interesting part would be if the Jeopardy computer, like people, made essential use of commonsense knowledge about everyday life. That kind of knowledge is automatic to humans, yet often absent from a computer. Take, for example, the following Jeopardy question:

Category: Rhyme Time

Clue: It’s where Pele stores his ball.

Answer: What’s a soccer locker?

Knowing that Pele is a soccer player is the easy part. The rhyming, no problem. But how would it know that an athlete stores belongings in a locker? And how would it figure out that this particular fact was the essential one? That’s the hard part, and that’s also one of the main topics of my research.

What becomes clear from an article in AI Magazine last fall is the astonishing breadth of techniques employed. The project leader, David Ferrucci, cast a wide net, not only within IBM, but also to other researchers of widely varying (and sometimes conflicting) approaches. Watson, above all, is a feat of engineering. Not so much in introducing new techniques: it’s mainly trying to make sense of the range of existing ones and see if they will play nicely with each other. It’s healthy for the field to have somebody, once in a while, try to do that.

If successful, it may provide support for Marvin Minsky’s contention that the true secret to intelligence is to have lots of different problem-solving methods. You need good ways of figuring out which one is appropriate for which sort of problem. As Minsky says, “If you understand something in only one way, you don’t understand it at all.”

Let’s be clear: winning a particular contest is not, by itself, a scientific achievement. Science is not a contest. Science advances by learning general problem-solving principles. If it happens that scientists introduce new, general principles that enable them to win a particular contest, then a contest can serve as a public demonstration of their prowess. It’s great PR. But sometimes contests can be won by tricks or specialized techniques that don’t cause scientists to learn anything really new. It all depends on how it’s done. Scientists judge by the principles and techniques, not by the contest results. Even after the Jeopardy event, we won’t really know “who won” until all the details of how it was done are published in the scientific literature.

In the past few years, there’s been a fad for contests, “challenges,” “grand prizes,” etc. in scientific and engineering fields. I have no objection if it’s only good, clean fun between consenting adults. But on the whole, I think this fad has been detrimental to science. Contests encourage competitive attitudes and secrecy between contestants. They focus people on incremental progress in very specialized areas, for one-shot tests. Science needs exactly the opposite—collaboration between researchers, openness, a diversity of approaches and “out of the box” and long-term thinking. It needs the freedom to choose what problem to work on, rather than have it dictated by the arbitrary rules of the contest.

Contests also encourage a gambling mentality, sometimes pathological. Happy stories of the winners are trumpeted, but vast majority of losers merely get their time and money wasted. A few years ago the Defense Advanced Research Projects Agency (DARPA), which has had a glorious history of funding innovative work in artificial intelligence, became enamored of contests. It browbeat researchers into participating, turning off many creative people who refused to “gamble with the rent money.” It set the field back by years. I myself have declined countless invitations to participate in contests.

But as contests go, this one is pretty good. IBM is the only research contestant, so it isn’t pitting researchers against each other. And the IBM team has been pretty open about its methods. Though not every detail was revealed, and the team has asked certain collaborators to sign non-disclosure agreements, I assume that more details will be forthcoming in the scientific literature after the contest.

Artificial Intelligence has suffered from the perception that it is tackling an impossible problem, tilting at windmills—that human intelligence can’t be understood in detail. That attitude sometimes discourages young people from entering the AI field.

My hope would be that a Jeopardy victory now, or victories in future computer-versus-human contests, will cause the researchers who win them, like Ferrucci and his team, to be role models for young researchers, like the astronauts of years past who inspired youngsters to embark on science careers. The point isn’t to show that computers are smarter than humans. It’s to show that the intelligence that enables a person to play Jeopardy is something that’s worth studying; that it can be understood, and put to work to help people. That would be the real victory. Go Watson!

Henry Lieberman is a research scientist who works on artificial intelligence at the Media Laboratory at MIT.

Hear more about AI at EmTech MIT 2017.

Register now

Uh oh–you've read all of your free articles for this month.

Insider Premium
$179.95/yr US PRICE

More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Premium {! insider.prices.premium !}*

    {! insider.display.menuOptionsLabel !}

    Our award winning magazine, unlimited access to our story archive, special discounts to MIT Technology Review Events, and exclusive content.

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

    First Look: exclusive early access to important stories, before they’re available to anyone else

    Insider Conversations: listen in on in-depth calls between our editors and today’s thought leaders

  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus ad-free web experience, select discounts to partner offerings and MIT Technology Review events

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

    Access to the magazine PDF archive—thousands of articles going back to 1899 at your fingertips

    Special discounts to select partner offerings

    Discount to MIT Technology Review events

    Ad-free web experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning magazine and daily delivery of The Download, our newsletter of what’s important in technology and innovation.

    See details+

    What's Included

    Bimonthly magazine delivery and unlimited 24/7 access to MIT Technology Review’s website

    The Download: our daily newsletter of what's important in technology and innovation

/
You've read all of your free articles this month. This is your last free article this month. You've read of free articles this month. or  for unlimited online access.