(Page 2 of 3)
Market watch: Entering “GM Ford” into the Wolfram Alpha engine produced tables comparing company data and stock prices, including charts plotting the price history.
Wolfram Research
VARIATIONS: Adding the word "in" changed everything. When I tweaked the search query to say "10 pounds in kilograms," the Wolfram site gave me the correct conversion: 10 pounds equals 4.536 kilograms. It also gave me the volumes (in various units) of 10 pounds of water. In a final, somewhat cheesy touch, it also told me that 10 pounds was 1.8 times the weight of Wolfram's book, A New Kind of Science. In Google's case, this revised search term produced the helpful calculated result up top: 10 pounds = 4.5359237 kilograms.
When I put in "10 lbs kgs," Alpha gave me the calculated result (the assumption was that I wanted multiplication), as it had with the full words. Google gave me metric conversion sites--the top one was a "Russian Brides Cyber Guide." (It offers both brides and metric conversions.)
When I tried "10 pds kgs," Alpha choked and didn't understand. Google helpfully asked if I meant "pounds" and gave me metric conversion sites, but not the calculated result.
SEARCH TERM: light bulb
WOLFRAM ALPHA: I was expecting some facts and figures on this ubiquitous technology but got a message saying that Wolfram Alpha "isn't sure what to do with your input."
GOOGLE: I got several links--starting with a Wikipedia entry--explaining what a light bulb is and providing some history.
VARIATIONS: When I tried "light bulb inventor," I got similar results: Alpha drew a blank, but Google gave useful links. When I tried "first light bulb," Alpha provided a table explaining that the light bulb was patented in 1878; under "people involved," it cited Thomas Edison.
SEARCH TERM: Aspirin Tylenol
WOLFRAM ALPHA: Alpha gave me molecular diagrams for aspirin and acetaminophen and lots of scientific information comparing their molecular weights, boiling points, vapor pressure, and so forth.
GOOGLE: Usefully (to nonchemists suffering from headaches), the top link was to a Wiki-answers page telling people whether they can take aspirin and Tylenol together. Other links gave information about toxicity, danger to kidneys, and the like.
SEARCH TERM: Stanford Harvard
WOLFRAM ALPHA: I got tables comparing data from the two schools: size of student bodies--broken down by full-time, part-time, undergraduate, and graduate--plus the number of undergraduate, master's, and doctoral degrees awarded, and similar data. Alpha listed Stanford's tuition as $25,000, which is incorrect, and no tuition for Harvard. As with all of Alpha's results, it gave me sources against which to check the information.
GOOGLE: Google gave me a collection of links (starting with a discussion board for students trying to make a college decision) and various news stories containing the two terms.
Wolfram Alpha can now analyze data you provide, so you can do things like map out your e-mail relationships.
comparisons not quite applicable
This is fair and interesting data, but my concern is that you're not using these two sites appropriately. From what I understand, one of the chief benefits of Wolfram Alpha is that it can produce results based on natural-language questions, whereas Google relies on more machine-based queries.
Putting in multiple-word keywords naturally favors Google, because you're not really asking a question there - you're just searching for data. In this instance, Google is most likely going to return more of the "right" kind of data that you're looking for, because they've had 10 years to finetune their algorithm in this way.
I think a more practical test would be to think about a few real-life questions you want answered, and then put in the best Google search you can think of, and compare that to simply asking Wolfram Alpha the question.
These sites/engines are tools, and it's tough to compare them by telling them to do the same thing the same way. You have to use them to their strengths.
Thanks for the article, though - interesting data. Can't wait to try it out.
Nate
Re: comparisons not quite applicable
Post a pair of queries and I'll try it out. But I think it's fair to use identical search terms -- otherwise how can you compare the two?
Anyway, I'd wager that most average users don't/won't know how to customize a query (though they might learn over time). Thanks for your comment.
Re: comparisons not quite applicable
I think what the previoues poster meant was to use natural language:
"How many pounds in a 10 kilogram?" versus "10 kilogram pound". The idea being that Google does not parse your query as anything other than keywords, while Alpha apparently does look for more meaning.
So when you use a keyword-oriented query ("10 kilograms pound") you are favoring Google.
If it's true that Alpha actually parses your query, it will ultimately yield a far more natural query than keyword orientation.
It's instinctive to us, having been warped by Google queries, to use "10 kilograms pound", but that's certainly not the way we'd ask another human and it's certainly more open to misunderstanding than "How many pounds is 10 kilograms?" or "10 kilograms is how many pounds" or even the shortened "10 kilograms as pounds".
Re: comparisons not quite applicable
Thanks for the response. You're right - identical queries ARE fair, but let's try both kinds out. You just did the Google-type queries, now let's run some natural language questions (your pick of which).
Nature:
What is the cheetah's top speed?
Are any two snowflakes alike?
What is the genetic sequence of the common fruit fly?
Commerce:
Where can I see The Soloist in Milwaukee?
How many copies of Harry Potter have been sold?
What are the 100 biggest companies in the world?
Astronomy:
How much larger than earth is jupiter?
How far away is the sun?
How many solar systems are in the Milky Way?
Thanks for the opportunity to run some of these!
Nate
Re: comparisons not quite applicable
Nate I ran those searches, see this new blog:
http://www.technologyreview.com/blog/editors/23495/
Re: comparisons not quite applicable
A search engine, and even more so a question answering system such as Alpha, is supposed to respond to an information need. However, none of your queries actually express what information you are looking for. It seems that you are just throwing random keywords at it to see what comes out. Did you even have an idea what you were hoping to obtain?
"Microsoft Apple" could mean just anything, and I as a human have no idea what you meant:
- What are the top products sold by each?
- How many employees do they have?
- Who are the companies' CEOs?
- Which name is longer?
- What funny anagrams of "Microsoft Apple" are there?
Why did you always use pairs of names? That doesn't look like a typical query at all to me. Would you ever use the queries you tested here? What for?
Alpha is a system for answering factual questions, there was not a single question in all your queries, and I wouldn't even know what questions were supposedly implied by your keyword queries.
Questions would be:
- What was the population of France in 1980?
- What was the population of France in 2000?
- What is the population growth in France between 1980 and 2000? (so that it would have to deduce answers mathematically from the data)
Obviously you may want to ask more difficult things, but keep in mind that you need to actually ASK something. Random words with no indication of what you are actually looking for are not helpful.
Google falls back on "give me a list of web pages that happen to contain the given words, irrespective of whether they have anything to do with what I'm looking for." Alpha does not.
Re: comparisons not quite applicable
Wolfram did fine on the France population questions -- see new post:
http://www.technologyreview.com/blog/editors/23495/
I think the fundamentals of Google and Wolfram are different. Google is a "Search" tool where as Wolfram is a "Lookup" tool.
If my intention is to Search for information across internet, I will use Google. If I am looking for an answer to a straight forward ques, I will use tools like ask or yahoo answers or wiki or Wolfram.
Google too has the facility of "Lookup" but you need to use keywords, as you have stated in your example. If you search "10 kg pounds" in Google it will point you to sites having those words and that can do the conversion for you. If you give "10 kg in pounds" it will answer your question with precisely. So comparing Google to Wolfram is like comparing apples to oranges.
Re: Fundamentals of this test.
The "apples and oranges" criticism is valid in that Google and Wolfram are quite different. Google mainly searches the web (but also does some computation for you, such as for metric conversion questions), while Wolfram mainly tries to calculate or summon answers from its own databases (but also points you to Wikipedia entries and source URLS).
The question is: if you want data (or a calculated answer) does Wolfram clearly do a better job--as it seeks to? My limited comparison shone a tiny bit of light on this question. Users will be the ultimate judge.
It's pointless to say that Wolfram Alpha awaits natural language questions. Many users will just use it like Google, maybe because they don't get the difference, maybe because they're in a hurry or too lazy. So it's obvious that there will be disappointments. This will not be the fault of Wolfram Research but let's face it, it will reveal how dumb a crowd is out there on the net.
Many users formulate questions with Goggle too...
Also, I think many users don't really take time or know how to formulate queries that would narrow results down to what their are looking for.
Luckily, Google is here since more than 10 years, it has a lot of stats about user behavior and past queries and it has its ranking mechanism. It learns and improves.
First, I think that Wolfram Alpha will also learn and improve with time. It's still a baby.
But anyway, any query or question will return inaccurate results if it lacks contextual information. Asking "New York Hawaii?" to a travel agent or to a football fan will give different results...
Just entering two words in a web engine could give many different answers in many different domains. With Google the results are sorted according to the users majority and many of them (well, us) are happy with the result. But if what you're looking for is on the sides of the Gaussian chart, you must add context.
(Btw: on Google, if you enter "Paris Hilton" and press "I'm feeling lucky", you get the Wikipedia page on Paris Hilton, but if you enter "Hilton Paris" you get the website of Hotel Hilton in Paris.)
I think formulating a phrase, a question, can provide a lot about the context. With Google, users learned to enter one of two keywords and to look up in the returned links which ones could contain the information they need.
With Wolfram Alpha, they will re-learn how to formulate questions and they will get direct answers, not a list of links.
Both will be useful.
E.
My worry is that it will suit journalists and students in search of quick "facts" but will fail to satisfy more demanding users, since access to source data is presumably not that easy (I haven't tried too hard but it doesn't look like it). The more scientific users will be suspicious of anything where they can't check the inputs. "Garbage in..." Maybe not, and I wish them luck since I find google useful and mindless in roughly equal measure.
On an utterly trivial note, since rumour has it (I can't actually remember that far back) that we spent a term together in the same junior school class, I typed the name of the school into both Alpha and Google. Alpha produced no result; google its usual 8m or so. So I tried his secondary (high school) level, figuring that Eton ought to produce some sort of result, since it's arguably the most famous school in the world. Sort of...google produced the school's website straight up, as you would expect. And Alpha? No mention of the school - apparently Eton is a tribal dialect in Cameroon spoken by only 52000 souls. My conclusion? I haven't got much further than figuring that particle physicists are admirably unsentimental.
Not to get too caught up in pop-culture and consumerism, but if I were Wolfram I would be working hard on those two areas. For a mass amount of people to adopt it, it will need to take queries along the lines of "Star Trek Los Angeles" and give you all of the show times and theaters for that day. That would be a better solution than what Google or Yahoo offer, or if you entered "New York Hawaii" it would go beyond comparing the two places and would present all of the different travel options from New York to Hawaii. Wolfram should go beyond an academic search tool to something that can help people get through their every day lives. I think it will probably be able to do this over time, and if so, it may be a breakthrough that lives up to the hype.
Error in use of input language method
The author is writing search words in a way that he has lernt how to write such words in google. He even states that he _makes assumtions_ in how he's search words should be interpreted. These assumtions can he only have acuired from interaction with google (or similar search engine).
However, the specific paths of answers and the compuations involved in Alpha relies heavily on correct search input assumtions. These assumtions are mostly derived from the use of our natural language. In it's most simple form; the word "in" can make an enormous difference. And where google's natural language interpretation ends with the word "in", this is only where Alpha begins...
>>> it's VERY HARD to win the battle against Google >>>
.
it's VERY HARD for Wolfram Alpha to win the battle against Google!
ONLY Microsoft can (just "try" to) compete with Google with its new "Kumo" Search Engine or (better) if merged with Yahoo!
however, the BEST name for Kumo ISN'T "Kumo" as explained in this post:
http://tinyurl.com/cw3q4z
.
Re: >>> it's VERY HARD to win the battle against Google >>>
I don't think that it's even attempting to face off or win any sort of battle against Google. Their intent is completely different. If anything I believe the services will be complimentary.
Why WolframAlpha will Not be a Google Killer
Manufacturing in the United States is in trouble. That's bad news not just for the country's economy but for the future of innovation.
On Twitter
Become a Fan on Facebook
Subscribe to the Feed
smihael
1 Comment
you can do conversions in google too
http://www.google.com/search?hl=en&q=10+pounds+in+kilograms&btnG=Search
http://www.google.com/search?hl=en&q=10+EUR+in+USD&btnG=Search
more features are here:
http://www.google.com/experimental/
Reply