A View from David Talbot
Where Wolfram's 'Knowledge Engine' May Have the Edge
TR takes a look behind the scenes at Wolfram’s development lab as Google launches a similar effort of its own.
This Monday, I had a chance to visit with researchers at Wolfram Research, in Champaign, IL, who are frantically putting the final touches on Wolfram Alpha, the new “computational knowledge engine” from physicist Stephen Wolfram.
This engine is meant to go live in two or three weeks; with it, you’ll be able to enter “GDP Germany Japan” and get not a list of Web pages, but comparative charts on the economic output of those two nations. Or you can enter “GATACTTCA” and find the spots on the human genome where that sequence appears.
Everybody at Wolfram Research characterized the new engine as something complementary to, and not in competition with, Google. (In short: Google uses elaborate means to find you the right Web pages, while Wolfram amasses databases and deploys myriad equations to compute answers for you.)
But that peaceable dynamic changed yesterday when Google announced a data-crunching service of its own, even as Wolfram was giving a demo of his new tool at Harvard’s Berkman Center for Internet and Society. With the new Google service–which is starting in a limited way, with data from the Census and the U.S. Bureau of Labor Statistics–you can do things like enter “unemployment New York” and get a historical chart of that rate, and click to compare the rate of other states and the U.S.
Wolfram told me last week that Google cofounder Sergey Brin is among the technology titans for whom he has personally demoed Alpha. We may never know if it led to any internal exhortations from Brin; Google said yesterday that its technology comes from Google’s 2007 acquisition of Trendalyzer. But it seems clear that a new search war is on–to command structured data and curate data from new sources to make it more useful to searchers.
Let’s say you want to know the caloric and nutritional information in a recipe you are making for your family. If you enter “2 cups flour, 2 eggs, 2 cups milk” in Wolfram Alpha, it will give you a nutrition label that reflects the combined values of those foods (after first prompting you to specify which of the several kinds of milk you are using). And if any normalizing of units was required, it will have done it already. If you enter similar search terms in Google, you will get a list of recipes and nutrition websites. Useful, to be sure, but it’s up to you to scroll around to find the right data, and then break out the calculator. That’s as good an example as any of the potential value of Wolfram Alpha, assuming the launch goes well. But that’s not to say that Google won’t tap the same U.S. Department of Agriculture databases and come up with something similar.
So, will Google be able to match what Wolfram specializes in? In Wolfram Alpha cubicles, dozens of people are toiling away to prepare thousands of data sets and are working out issues like figuring out which of the many meanings of “bushel” a user might be seeking. They are entering information on tiling patterns and the periodic table of the elements. These are the same people who develop Wolfram’s Mathematica software, which is generally understood to be the most comprehensive math and science software tool set out there.
Here’s how Wolfram cofounder Theodore Gray put it to me: “Part of the challenge was finding the range of people who can cover these things down to the level of detail that’s needed. The average software company does not have the breadth and depth of people to be able to cover the subject matter. Software companies will have a bunch of CS [computer science] majors. They don’t know chemistry, physics, materials, economics, food science–every different little subject area.”
Whether that translates into a usable tool and a scalable business model is another question. Success will also depend on how useful people find Wolfram’s natural-language interface, which his researchers are customizing. We’ll know more after the launch.
But on Google’s side, success will also depend on how broad and deep it wants to make its data-searching features. As Google itself put it in its 2007 blog post announcing the acquisition of Trendalyzer, “Gathering data and creating useful statistics is an arduous job that often goes unrecognized.” And so far, Google is limiting its new effort to input public data–which includes “statistics for prices of cookies, CO2 emissions, asthma frequency, high school graduation rates, bakers’ salaries, number of wildfires, and the list goes on.”
Wolfram says that he’s licensing data from private sources too, and he told me last week that he’s not interested in selling Alpha: “I like building stuff. With Mathematica, I’m at Year 23 right now. This project will also be a multidecade project, and I’m going to do this project. I didn’t do this in order to flip it.”