Tuesday, March 20, 2007

Part II: A Smarter Web

New technologies will make online search more intelligent--and may even lead to a "Web 3.0."

By John Borland

Credit: Polly Becker

Part I of the story

Standards and Critics

The next years weren't easy. Miller quickly had to become researcher, diplomat, and evangelist. The effort to build the Semantic Web has been well publicized, and Berners-Lee's name in particular has lent its success an air of near-inevitability. But its visibility has also made it the target of frequent, and often harsh, criticism.

Some argue that it's unrealistic to expect busy people and businesses to create enough metadata to make the Semantic Web work. The simple tagging used in Web 2.0 applications lets users spontaneously invent their own descriptions, which may or may not relate to anything else. Semantic Web systems require a more complicated infrastructure, in which developers order terms according to their conceptual relationships to one another and--like Dewey with his books--fit data into the resulting schema. Creating and maintaining these schemas, or even adapting preëxisting ones, is no ­trivial task. Coding a database or website with metadata in the language of a schema can itself be painstaking work. But the solution to this problem may simply be better tools for creating metadata, like the blog and social-networking sites that have made building personal websites easy. "A lot of Semantic Web researchers have realized this disconnect and are investing in more human interfaces," says David Huynh, an MIT student who has helped create several such tools.

Other critics have questioned whether the ontologies designed to translate between different data descriptions can realistically help computers understand the intricacies of even basic human concepts. Equating "post code" and "zip code" is easy enough, the critics say. But what happens when a computer stumbles on a word like "marriage," with its competing connotations of monogamy, polygamy, same-sex relationships, and civil unions? A system of interlocking computer definitions could not reliably capture the conflicting meanings of many such common words, the argument goes.

"People forget there are humans under the hood and try to treat the Web like a database instead of a social construct," says Clay Shirky, an Internet consultant and adjunct professor of interactive telecommunications at New York University.

It hasn't helped that until very recently, much of the work on the Semantic Web has been hidden inside big companies or research institutions, with few applications emerging. But that paucity of products has masked a growing amount of experimentation. Miller's W3C working group, which included researchers and technologists from across academia and industry, was responsible for setting the core standards, a process completed in early 2004. Like HP, other companies have also created software development tools based on these standards, while a growing number of independent researchers have applied them to complicated data sets.

Life scientists with vast stores of biological data have been especially interested. In a recent trial project at Massachusetts General Hospital and Harvard University, conducted in collaboration with Miller when he was still at the W3C, clinical data was encoded using Semantic Web techniques so that researchers could share it and search it more easily. The Neurocommons project is taking the same approach with genetic and biotech research papers. Funded by the scientific-data management company Teranode, the Neurocommons is again working closely with W3C, as well as with MIT's Computer Science and Artificial Intelligence Laboratory.