Information hookup: The social search engine Aardvark helps users find other people who can answer questions for them.
Aardvark
A startup says natural language processing works best with human intelligence.
Providing answers to tricky questions has become big business online. But community question-and-answer sites can get clogged up with outdated answers, and it's fiendishly difficult to create software that can automatically understand a question and provide the best answer.
Damon Horowitz, chief technology officer and cofounder of the San Francisco-based Aardvark, will outline a different approach when he speaks at the Web 2.0 Expo in New York today. Horowitz believes that the real power of natural language processing can only be unlocked by acknowledging its limitations and filling in the gaps with human intelligence. The company launched its product to the public last month.
Aardvark has done extensive research into using artificial intelligence techniques to answer questions, but the company's focus has shifted away from training machines to respond. "We wanted to let another human being answer and have the machine do the heavy lifting of indexing everybody--the tens of thousands of people who are in your extended network and all of the things that those people know," Horowitz says.
The difficulty of having machines interpret meaning has forced many "semantic Web" companies to focus on niche areas, such as answers to questions about medicine. "There's a reason why all of our artificial intelligence systems only do so well with language processing tasks," Horowitz adds. "Language has much more to do with live interaction with another person--understanding context and forming a connection."
When a new user signs up to use Aardvark, she is asked to enter her Facebook login information and list of topics she is knowledgeable about. When a she asks a new question--for example, "What's the name of a good restaurant in Cambridge, MA?"--the system tries to find other users who can answer it.
Aardvark has to parse the question to determine its topic before hunting for users with related interests. But the system also looks for potential answerers who are connected socially. It does this by gathering data from Facebook connections, and searching Twitter messages and relevant blog posts. The result, Horowitz says, is artificial intelligence that facilitates a human connection, helping users find someone who "is going to look you in the virtual eye."
A new service mines the contents of Web pages looking for meaning and relevance.
Startup will track live events, including streaming video, auctions, and competitions.
New software distinguishes between experts and spammers, showing who can be trusted.
Aardvark may have a point.
From the point of view of a superintelligence - if it's possible to imagine that from here - it's kinda wasteful to spend time interpreting human language. Human language and communication is so inefficient.
Human beings are the ones who are involved in creating and evolving their languages, so it does make some sense to keep them involved in some areas of interpretation.
When we evolve past the human-machine dichotomy, my guess is that current human language will fall by the wayside, in favor of a more rigorous means of communication. However, we will still need to think and speak 'out of the box'.
Seems to me its a little early to give up on teaching machines to understand natural language.
Pedro
Re: Ardvaak is too pessimistic
marcalpv, I agree completely. They should spend more time "teaching" the machines to learn language than the other way around being that language is the very units that make up thoughts. If a machine can understand language then they would learn how to think...
Re: Ardvaak is too pessimistic
agreed, the problem is figuring out how to get a computer to "think" for itself.
The Human mind tends to represent things with abstractions and connections. Definitions are often loose and tied to various abstractions, which, depending on circumstances, and what actually amounts to a tremendous amount of information, can be changed or inferred.
For instance when you see a chair, any type of chair, you can identify what it is in a generic way. It doesn't depend on if you have actually seen that chair before or if someone has told you that this specific object is a chair, you can infer that it is a chair by its general form, and thus infer its function. It has four legs for support, a horizontal plane for sitting on, and often another plane roughly perpendicular to the horizontal one. The human mind can instantly recognize this as a chair simply through the connections with abstract concepts that it brings up, or vague resemblances it has to other similar objects. All of these connections come together with the word "chair".
A computer generally doesn't work like that. In a human mind, all of the connections are in parallel, and can be rapidly processed. In computers just about everything is carried out sequentially, and not only that, but analytically as well. There are pre programmed connections, but these often come in the form of indexes, which have to be searched sequentially, which takes a while. It may have a chair defined as a four legged object, but what happens when it encounters say a rocking chair? It has to make a connection between the object and the word chair, but inferred from other connections.
The problem is difficult enough when dealing with physical concepts that can be readily defined, and once you start to get into abstract concepts things get even more difficult.
Re: Ardvaak is too pessimistic
A computer works however we TELL it to work. Yes, the connections in the cortext are in parallel, but the brain's communications are also much, much slower than a computer's. The computer also has a huge advantage in storage capacity (essentially, unlimited) and the fact that a computer NEVER forgets. For a complete discussion, please see http://www.aeyec.com/#C
While at this time we cannot mimic the brain's massive electrochemical parallel processing, we can copy some elements of it. For example, instead of saving a fact as a sentence, as many AI programs do, it can be saved by linking each word of the fact to a data placekeeper (which means they are linked in parallel as well as in sequence, rather than just sequentially), then you search for words which are linked to the same placekeeper rather than searching every fact to find those with the target words.
Regarding the statement that computer data "come in the form of indexes which have to be searched sequentially": nonsequential search algorithms are used to search indexes, and they are extremely fast. No programmer would use sequential searches of indexes.
At Bing, we’re working to better understand user intent and streamlining the decision-making process for users by using that intent in simple ways. We constantly have to guard against overengineering responses to user questions – if we see someone asking for “flights to seattle” we don’t need a complex language model to know that they are likely looking for tickets to get from wherever they are to Seattle. That said, this simplistic answering model also has limitations and I agree that humans today can do a good job in helping refinements and delivering opinions. That doesn’t mean we aren’t still working on making computers better acquainted with the world in which they live. And part of that is through that elusive ‘semantic search’.
What is semantic? In essence, it’s the study of meaning or the differences between meanings of words or symbols. In application, one interpretation of that I like is being able to make associations between objects as they exist in the real world in order to give us context for conversations. I know a horse is a mammal, of phylum chordata, genus equus, etc. I know it has 4 legs, I know people ride on it (it doesn’t ride people), and that it can be used for hauling Budweiser across snowy fields. All these things impose logical constraints on both the questions I ask as well as the responses I give when asked a question. I would never ask “Why isn’t there a horse on this airplane from Dublin to berlin?”
Engines today don’t benefit from this ability. They often heavily rely on a model of indexing and classification based on how close words are to each other, what pages link to what other pages, etc. It’s kind of like trying to learn a language by memorizing the dictionary – sure, you can find the definition for a horse very quickly because you know that it starts with “h” but you won’t know how to use it.
We think there a number of ways that semantic models will evolve search. First, we think you’ll get a more natural UI. We’ll be able to deliver more “in-process” searches. Search as a ‘task’ you do is becoming outmoded. Being able to hold up a device and have it tell you about the world is a ‘search’ but it not thought of that way today.
We think access to more data and a structuring of that data will accelerate in scope and complexity. We’ll be able to collect more richly augmented data. We won’t be constrained to just a crawl, but a crawl with more associated attributes.
We’ll have to train the engines to develop models about languages using an understanding of data we crawl. We’ll work to how to technically relate concepts and how they manifest in the real world.
Finally, this semantic capability will lead to better ‘task completion ability’ with engines – something more people are doing with general purpose engines despite all our current failings. The question is how do you have a conversation with an engine? How do you have it do things for you? Even our cool task tools we have today (like detecting that you want to come to seattle) will seem embryonic – today you still have to give us too many hints.
In any case, I’m delighted to see the conversation about this heating up. As we move away from yesterday’s models of search and into ones that better reflect how the users are actually using the engines, this will only become more important.
Cheers!
Stefan Weitz
Director, Bing
"Create a new language, Maybe?"
Not being an expert; yet, it would seem that a new language is needed to pull this off successfully.I think that taking apart the languages and combining the semantics/definitions and determining its' usability by the mass of end computer users could be a step into the future for future computer/ human interaction.I know this point might be moot but it is probably the most vital step in solving this dilemna."What!?,is he crazed or just uninformed?"you may be pondering at this point.I believe that to make the future of computing able to truly encompass the humanistic experience it has to be able to first respond by its' own initiative.
Re: "Create a new language, Maybe?"
I have been developing and using such a language for some years. Unlike description logics, which take a bottom-up approach for deducibility, this language takes a top-down approach for expressivity, and is then dynamically constrained for contextual deducibility.
Manufacturing in the United States is in trouble. That's bad news not just for the country's economy but for the future of innovation.
Our list of the 50 most innovative companies, including the following:
On Twitter
Become a Fan on Facebook
Subscribe to the Feed
Gcanno
24 Comments
This is an okay start but can be corrupted fairly easy by people in the network who will spam and try to manipulate the system.
Google and all these search engines are making money off the people who contribute to the web.A better way to approach this problem is to look at why do we contribute to google's knowledge and let them make money off of us.
I see a network where people get paid very small sums of money for their contributions(Searching the Web) and knowledge, which will then compound over a lifetime and serve as a new form of retirement,savings or investment funds pooled together.
These same networks will also serve as a consumer reports and create a loosely but self serving digital village with massive financial,political and personal implications never seen before on such a scale.
Albert Einstein once described compound interest as the “greatest mathematical discovery of all time.
Reply