The set of Jeopardy.

Two years after IBM announced that it was working on a computer program that would compete on TV's Jeopardy, the event is set to air over three days next month. IBM's program, Watson, will have two matches against the top players the game show has ever had, Ken Jennings and Brad Rutter, with the top performer getting $1 million (IBM would donate its winnings to charity). It should make for an entertaining spectacle, but it's not clear how meaningful Watson's performance would be.

IBM wants to show that it has dramatically advanced the state of machine intelligence since 1997, when its Deep Blue computer beat chess grandmaster Garry Kasparov. Unlike playing chess, competing on Jeopardy requires a machine to understand nuances of language, which is one of the hardest problems in computing. The Watson system won't be connected to the Internet—it will have to analyze information it has already been fed and assess its level of confidence in whether it has come up with the right answer. (Watson also won't listen to Alex Trebek read the "answers" for which Jeopardy players come up with questions; those will be entered into the program as text. For some insight into how Watson sorts through possible responses, you can try this interactive demonstration from the New York Times.)

Michael Littman, a machine-learning researcher at Rutgers University, says that even if Watson doesn't win the match, merely coming close to human Jeopardy champions would be an impressive achievement, because until now computers have bested humans only at "closed-world" games that can be reduced to logical choices, like chess and backgammon. Machines have not come close to matching human skill at "open-world" games and puzzles such as crosswords, Littman says. IBM says this will have real value—that Watson possesses such linguistic and computational dexterity that it is likely to lead to useful tools in a wide variety of fields. For instance, the company says, it might become the basis of engines for answering questions in medicine, customer service, and travel applications. That has long been a goal of artificial intelligence research: to create systems that aren't "brittle," or good only in narrow fields.

Then again, answering Jeopardy questions well is in itself a specific field. It makes use of puns and other forms of word play in a way that other applications do not. "I've looked into this work a bit and it seems to be an interesting collection of special-purpose methods designed for Jeopardy questions, which tend to fall into one of a fairly small set of categories," says Stuart Russell, an AI researcher at the University of California, Berkeley. "I don't want to minimize the effort it took to get it working, but I am not sure it represents a significant advance for AI because the techniques do not appear to generalize easily to other forms of natural language understanding." In other words, says Doug Lenat, who heads Cycorp—a company that has labored to infuse AI software with common-sense reasoning skills—even though Watson's ability to understand grammar and subtleties of meaning is valuable, to succeed outside of Jeopardy it would have to be taught new sets of rules for each application. "You could do something similar for other domains, but just like in the past, the narrower the domain, the more success you're going to have with that," Lenat says. "This is more of a celebration of the current state of the art that natural language processing has achieved rather than an advance in that state of the art."

Some researchers find it hard to assess Watson because IBM hasn't revealed much of the computer science behind it. Marvin Minsky, an MIT professor and AI pioneer, says he doesn't think it would be right to comment about the project "until IBM issues a technical report on the system, how it works, and what are its results." Insight could come from watching the show, especially when Watson gets things wrong: its incorrect responses figure to offer clues about the program's methods.

Lenat says he'll be watching; he plans to have a viewing party at his company. Because even if Watson does not necessarily represent a breakthrough for AI, the attention it will get will be positive for the field, he says. "It is the 2011 analogue of what Deep Blue did in chess," Lenat says. "It will capture human imagination."

Blogging often sounds like a great idea: sharing thoughts and expertise, becoming a part of a community, and taking the first few steps to wider recognition as a writer. But many bloggers quickly get disillusioned.

IBM's internal records show, for example, that only three percent of the company's employees have posted to a blog at all. Of those who have, 80 percent have posted only five times or fewer. Many of the people interviewed for the study say they stopped blogging--or never got started--because they didn't think anyone would read their posts.

In an effort to fix this problem, IBM researchers have been experimenting with a tool called Blog Muse, which suggests a topic for a blog post with a ready-made audience.

"We saw this disconnect between readers and writers," says Werner Geyer, a researcher at IBM's center for social software in Cambridge who was involved with the work. The writers surveyed often weren't sure how to interest readers, and many of their posts got little to no response. Readers, on the other hand, couldn't find blogs on the topics they wanted to read about.

So Geyer and his colleagues built a widget to bring these two halves of the problem closer together. Readers use the widget to suggest topics they want to read about, and they can vote in support of existing suggestions. Those suggestions then get sent to possible writers, matching topics to writers by analyzing his social network connections and areas of expertise.

The researchers found that writers were most likely to post on a topic suggested by a sizeable audience, and that audience members followed up by read posts on requested topics. Blog posts resulting from the system also received about twice as many comments, three times as many ratings, and much more traffic, says Casey Dugan, another researcher at IBM's Cambridge center.

The effort didn't substantially increase the quantity of posts however. The researchers speculate that this is because users who planned to write blog posts anyway simply chose suggested topics rather than coming up with their own.

The researchers want to do a larger, longer-term deployment of the original tool (their research was done over four weeks with 1,000 users). And they plan to present their results in April at the ACM Conference on Human Factors in Computing Systems in Atlanta, GA.

This week at the Frontiers in Optics conference in San Jose, Jeffrey Kash of IBM Research laid out his vision of the future of supercomputers.

The fastest supercomputer in the world, the Los Alamos National Laboratory's IBM Roadrunner, can perform 1,000 trillion operations per second, which computer scientists call the petaflop scale. Getting up to the next level, the exaflop scale, which is three orders of magnitude faster, will require integrating more optical components to save on power consumption, Kash said. (Laser scientists at the conference are also looking towards the exascale, as I reported on Wednesday.)

Melinda Rose of photonics.com reported on Kash's talk, which he stated represented his personal views and not those of IBM:

Because a 10x increase in performance means the machine will consume double the power, to make future supercomputers feasible to build and to operate optics will need to be more widely used he said. In 2008 a 1-petaflop computer cost $150 million to build and consumes 2.5 MW of power. Using the same technology, by 2020 a 1 exaflop machine would cost $500 million to build and consume 20 MW of power.

Kash gave a timeline that would find optics replacing electrical backplanes by 2012 and replacing electrical printed circuit boards by 2016. In 2020, optics could be directly on the chip. In a less aggressive scenario, by 2020 all off-chip communications need to be optical, he said.

But for that to happen, to get optics up to millions of units in 2010, the price needs to drop to about $1 per Gb/s, he said. Today, Gb/s processing costs about $10.

• 1 Comment
• Add a Comment
• Permalink