The Semantic Web Goes Mainstream

Continued from page 1

In addition to employing the Semantic Web standards, Twine is also using extremely advanced machine learning and natural-language processing algorithms that give it capabilities beyond anything that relies on manual tagging. The tool uses a combination of natural-language algorithms to automatically extract key concepts from collections of text, essentially automatically tagging them. According to Spivack, these algorithms adroitly handle ambiguous sets of words, determining, for example, whether J.P. Morgan is a person or a company, depending on the context. And Twine can find the subject of a text even if a keyword is never mentioned, he says, by using statistical machine learning to compare the text with data sources such as Wikipedia. "We can determine when a document is about a subject even if the subject isn't mentioned in the document," Spivack says. "So we can add new paths and new ways to get to the document" during a search.

Another technique that Twine uses is graph analysis. This idea, explains Spivack, is similar to the thinking behind the "social graph" that Mark Zuckerberg, the founder of Facebook, extols: connections between people exist in the real world, and online social-networking tools simply collect those connections and make them visible. In the same way, Spivack says, Twine helps make the connections between people and their information more accessible. When data is tagged, it essentially becomes a node in a network. The connections that each node has to other nodes (which could be other data, people, places, organizations, projects, events, et cetera) depend on their tags and the statistical relevance they have to the tags of other nodes. This is how Twine determines relevance when a person searches through his or her information. The farther away a node is, the less relevant it is to a user's search.

It's still too early to know if Twine will be successful with consumers, says Tony Shaw, president of Semantic Universe, an organization committed to raising awareness of semantic technologies in business and consumer settings. Success will not simply depend on making the technology work, but also on managing people's expectations of the technology, he says. "It's about fighting the hype problem."

Twine will open up to invited users starting today. In the next couple of months, says Spivack, the tool will accept more users, and by the summer of 2008, it should be completely open. In addition, Twine will have an open platform that allows software developers to build tools on top of it, such as visualization software so that users can see their information in different ways. "But first, we're starting with the basics," Spivack says.