Computation engine: The new Wolfram Alpha engine computes answers based on search terms. In this case, entering “Sydney New York” produced a flight path and comparative data on the two cities, much of which can be easily found via Google too.
Wolfram Research
Our exclusive test shows how the two Web engines compare when given the same queries.
(See answers to readers' queries, submitted in response to this article, here.)
Last week, as physicist Stephen Wolfram was demonstrating his new Web-based "computation engine"--Wolfram Alpha--to the public, Google announced a data-centric service of its own. Alpha accesses databases that are maintained by Wolfram Research, or licensed from others, and deploys formulas and algorithms to compute answers for searchers.
Using some prelaunch log-in credentials provided by the Wolfram team, I decided to run my own Wolfram Alpha versus Google test. I used a handful of search terms that could produce data-centric answers and tried variations in a few cases to see what might happen.
This was an effort to get beyond the characterizations and produce some real data. I also wanted to explore the claims made during my visit to Wolfram Research last week: that Alpha can add unique value in computing answers based on your search queries.
Here's what I entered, and what I found.
SEARCH TERM: Microsoft Apple
WOLFRAM ALPHA: I got side-by-side tables and graphics on the stock prices and data on the two companies, plus a chart plotting the price of both stocks over time.
GOOGLE: The top hits were mostly news stories, from major and minor publications, containing both words.
VARIATION: When I changed the Google search term to just "Microsoft" or just "Apple," I got a chart with today's stock price up top; when I clicked that link, I received tons of information--comparable to what Alpha provides--but only on the single company.
SEARCH TERM: Sydney New York
WOLFRAM ALPHA: I got tables showing the distance between the two cities in miles, kilometers, meters, even nautical miles; a map of the world with the optimal flight path; and the fact that the trip spans 0.4 of the earth's circumference. I learned how long it would take to make the trip: 18.1 hours flying; 13 hours for a sound wave, 74 milliseconds for a light beam in fiber, and 53 milliseconds for a light beam traveling in a vacuum. I also got comparative populations, elevation in meters, and current local times.
GOOGLE: I got a mix of things: a form for finding flights between Sydney and New York; a Google Maps-plotted list of businesses in New York City that contain the word "Sydney"; and links to the municipal government of Sidney, a small town in upstate New York.
VARIATION: When I tried "Sydney New York distance" (adding the word "distance"), Wolfram gave me only the distance information mentioned above while Google gave me links to distance-finding websites. I opened the first one, was able to enter "New York" and "Sydney" in some forms, and wound up with much the same information provided by Wolfram (but without the light-beam and sound-wave details).
SEARCH TERM: 10 pounds kilograms
WOLFRAM ALPHA: The site informed me that it interpreted my search term as an effort to multiply "10 pounds" by "1 kilogram" and gave me this result: 4.536 kg2 (kilograms squared) or 22.05 lb2 (pounds squared).
GOOGLE: Google gave me links to various metric conversion sites.
A new site provides answers to life, the universe, and just about everything.
comparisons not quite applicable
This is fair and interesting data, but my concern is that you're not using these two sites appropriately. From what I understand, one of the chief benefits of Wolfram Alpha is that it can produce results based on natural-language questions, whereas Google relies on more machine-based queries.
Putting in multiple-word keywords naturally favors Google, because you're not really asking a question there - you're just searching for data. In this instance, Google is most likely going to return more of the "right" kind of data that you're looking for, because they've had 10 years to finetune their algorithm in this way.
I think a more practical test would be to think about a few real-life questions you want answered, and then put in the best Google search you can think of, and compare that to simply asking Wolfram Alpha the question.
These sites/engines are tools, and it's tough to compare them by telling them to do the same thing the same way. You have to use them to their strengths.
Thanks for the article, though - interesting data. Can't wait to try it out.
Nate
Re: comparisons not quite applicable
Post a pair of queries and I'll try it out. But I think it's fair to use identical search terms -- otherwise how can you compare the two?
Anyway, I'd wager that most average users don't/won't know how to customize a query (though they might learn over time). Thanks for your comment.
Re: comparisons not quite applicable
I think what the previoues poster meant was to use natural language:
"How many pounds in a 10 kilogram?" versus "10 kilogram pound". The idea being that Google does not parse your query as anything other than keywords, while Alpha apparently does look for more meaning.
So when you use a keyword-oriented query ("10 kilograms pound") you are favoring Google.
If it's true that Alpha actually parses your query, it will ultimately yield a far more natural query than keyword orientation.
It's instinctive to us, having been warped by Google queries, to use "10 kilograms pound", but that's certainly not the way we'd ask another human and it's certainly more open to misunderstanding than "How many pounds is 10 kilograms?" or "10 kilograms is how many pounds" or even the shortened "10 kilograms as pounds".
Re: comparisons not quite applicable
Thanks for the response. You're right - identical queries ARE fair, but let's try both kinds out. You just did the Google-type queries, now let's run some natural language questions (your pick of which).
Nature:
What is the cheetah's top speed?
Are any two snowflakes alike?
What is the genetic sequence of the common fruit fly?
Commerce:
Where can I see The Soloist in Milwaukee?
How many copies of Harry Potter have been sold?
What are the 100 biggest companies in the world?
Astronomy:
How much larger than earth is jupiter?
How far away is the sun?
How many solar systems are in the Milky Way?
Thanks for the opportunity to run some of these!
Nate
Re: comparisons not quite applicable
Nate I ran those searches, see this new blog:
http://www.technologyreview.com/blog/editors/23495/
Re: comparisons not quite applicable
A search engine, and even more so a question answering system such as Alpha, is supposed to respond to an information need. However, none of your queries actually express what information you are looking for. It seems that you are just throwing random keywords at it to see what comes out. Did you even have an idea what you were hoping to obtain?
"Microsoft Apple" could mean just anything, and I as a human have no idea what you meant:
- What are the top products sold by each?
- How many employees do they have?
- Who are the companies' CEOs?
- Which name is longer?
- What funny anagrams of "Microsoft Apple" are there?
Why did you always use pairs of names? That doesn't look like a typical query at all to me. Would you ever use the queries you tested here? What for?
Alpha is a system for answering factual questions, there was not a single question in all your queries, and I wouldn't even know what questions were supposedly implied by your keyword queries.
Questions would be:
- What was the population of France in 1980?
- What was the population of France in 2000?
- What is the population growth in France between 1980 and 2000? (so that it would have to deduce answers mathematically from the data)
Obviously you may want to ask more difficult things, but keep in mind that you need to actually ASK something. Random words with no indication of what you are actually looking for are not helpful.
Google falls back on "give me a list of web pages that happen to contain the given words, irrespective of whether they have anything to do with what I'm looking for." Alpha does not.
Re: comparisons not quite applicable
Wolfram did fine on the France population questions -- see new post:
http://www.technologyreview.com/blog/editors/23495/
I think the fundamentals of Google and Wolfram are different. Google is a "Search" tool where as Wolfram is a "Lookup" tool.
If my intention is to Search for information across internet, I will use Google. If I am looking for an answer to a straight forward ques, I will use tools like ask or yahoo answers or wiki or Wolfram.
Google too has the facility of "Lookup" but you need to use keywords, as you have stated in your example. If you search "10 kg pounds" in Google it will point you to sites having those words and that can do the conversion for you. If you give "10 kg in pounds" it will answer your question with precisely. So comparing Google to Wolfram is like comparing apples to oranges.
Re: Fundamentals of this test.
The "apples and oranges" criticism is valid in that Google and Wolfram are quite different. Google mainly searches the web (but also does some computation for you, such as for metric conversion questions), while Wolfram mainly tries to calculate or summon answers from its own databases (but also points you to Wikipedia entries and source URLS).
The question is: if you want data (or a calculated answer) does Wolfram clearly do a better job--as it seeks to? My limited comparison shone a tiny bit of light on this question. Users will be the ultimate judge.
It's pointless to say that Wolfram Alpha awaits natural language questions. Many users will just use it like Google, maybe because they don't get the difference, maybe because they're in a hurry or too lazy. So it's obvious that there will be disappointments. This will not be the fault of Wolfram Research but let's face it, it will reveal how dumb a crowd is out there on the net.
Many users formulate questions with Goggle too...
Also, I think many users don't really take time or know how to formulate queries that would narrow results down to what their are looking for.
Luckily, Google is here since more than 10 years, it has a lot of stats about user behavior and past queries and it has its ranking mechanism. It learns and improves.
First, I think that Wolfram Alpha will also learn and improve with time. It's still a baby.
But anyway, any query or question will return inaccurate results if it lacks contextual information. Asking "New York Hawaii?" to a travel agent or to a football fan will give different results...
Just entering two words in a web engine could give many different answers in many different domains. With Google the results are sorted according to the users majority and many of them (well, us) are happy with the result. But if what you're looking for is on the sides of the Gaussian chart, you must add context.
(Btw: on Google, if you enter "Paris Hilton" and press "I'm feeling lucky", you get the Wikipedia page on Paris Hilton, but if you enter "Hilton Paris" you get the website of Hotel Hilton in Paris.)
I think formulating a phrase, a question, can provide a lot about the context. With Google, users learned to enter one of two keywords and to look up in the returned links which ones could contain the information they need.
With Wolfram Alpha, they will re-learn how to formulate questions and they will get direct answers, not a list of links.
Both will be useful.
E.
My worry is that it will suit journalists and students in search of quick "facts" but will fail to satisfy more demanding users, since access to source data is presumably not that easy (I haven't tried too hard but it doesn't look like it). The more scientific users will be suspicious of anything where they can't check the inputs. "Garbage in..." Maybe not, and I wish them luck since I find google useful and mindless in roughly equal measure.
On an utterly trivial note, since rumour has it (I can't actually remember that far back) that we spent a term together in the same junior school class, I typed the name of the school into both Alpha and Google. Alpha produced no result; google its usual 8m or so. So I tried his secondary (high school) level, figuring that Eton ought to produce some sort of result, since it's arguably the most famous school in the world. Sort of...google produced the school's website straight up, as you would expect. And Alpha? No mention of the school - apparently Eton is a tribal dialect in Cameroon spoken by only 52000 souls. My conclusion? I haven't got much further than figuring that particle physicists are admirably unsentimental.
Not to get too caught up in pop-culture and consumerism, but if I were Wolfram I would be working hard on those two areas. For a mass amount of people to adopt it, it will need to take queries along the lines of "Star Trek Los Angeles" and give you all of the show times and theaters for that day. That would be a better solution than what Google or Yahoo offer, or if you entered "New York Hawaii" it would go beyond comparing the two places and would present all of the different travel options from New York to Hawaii. Wolfram should go beyond an academic search tool to something that can help people get through their every day lives. I think it will probably be able to do this over time, and if so, it may be a breakthrough that lives up to the hype.
Error in use of input language method
The author is writing search words in a way that he has lernt how to write such words in google. He even states that he _makes assumtions_ in how he's search words should be interpreted. These assumtions can he only have acuired from interaction with google (or similar search engine).
However, the specific paths of answers and the compuations involved in Alpha relies heavily on correct search input assumtions. These assumtions are mostly derived from the use of our natural language. In it's most simple form; the word "in" can make an enormous difference. And where google's natural language interpretation ends with the word "in", this is only where Alpha begins...
>>> it's VERY HARD to win the battle against Google >>>
.
it's VERY HARD for Wolfram Alpha to win the battle against Google!
ONLY Microsoft can (just "try" to) compete with Google with its new "Kumo" Search Engine or (better) if merged with Yahoo!
however, the BEST name for Kumo ISN'T "Kumo" as explained in this post:
http://tinyurl.com/cw3q4z
.
Re: >>> it's VERY HARD to win the battle against Google >>>
I don't think that it's even attempting to face off or win any sort of battle against Google. Their intent is completely different. If anything I believe the services will be complimentary.
Why WolframAlpha will Not be a Google Killer
Manufacturing in the United States is in trouble. That's bad news not just for the country's economy but for the future of innovation.
On Twitter
Become a Fan on Facebook
Subscribe to the Feed
smihael
1 Comment
you can do conversions in google too
http://www.google.com/search?hl=en&q=10+pounds+in+kilograms&btnG=Search
http://www.google.com/search?hl=en&q=10+EUR+in+USD&btnG=Search
more features are here:
http://www.google.com/experimental/
Reply