Keeping an Eye on Google

Wal-Mart, McDonald’s, Microsoft: a handful of companies are so dominant in their markets that almost everything they do is condemned by someone as an abuse of power. Now Google has joined that exclusive club. As the proprietor of the Internet’s most popular search engine, Google has become the de facto gatekeeper of the Web-with the ability to make or break a site simply by moving it up or down a few notches in its search rankings. And while that hasn’t affected Google’s pristine brand image among hundreds of millions of Internet users, it has some programmers and Web publishers thoroughly riled. “Search engines are an essential part of the Internet now, and yet they’re all controlled by private organizations, and their mechanisms are secret,” says Doug Cutting, an independent software consultant based in Petaluma, CA. “There’s a lot of room for these companies to manipulate their services for commercial gain. It’s an unhealthy situation.”

Cutting’s remedy is an open-source search engine, called Nutch, that uses ranking algorithms similar to Google’s, but with a twist: each search result is accompanied by a link labeled “Explain” that produces a detailed accounting of the various scores and weights that gave the result its rank. Says Cutting, “We want to provide something that will work as a watchdog, so that experts can compare Nutch’s results to a commercial search engine’s and see whether, for example, somebody is biasing their results toward their advertisers.” If all is on the up and up, the results should be roughly the same. Cutting’s development of Nutch is being funded in part by Internet ad agency Overture, which was recently acquired by Yahoo!; the new search engine should be publicly available later this year.

Google also boasts a few human watchdogs, including the publishers of Web sites such as Watching Google like a Hawk and Search Engine Watch. But perhaps Google’s most persistent and tendentious critic is Daniel Brandt, founder of Google Watch. Brandt, who makes his living running a reference site called NameBase.org that collects book and newspaper citations of prominent people, believes Google is too cavalier toward small Internet businesses, which can spend months working to raise their Google rankings, only to vanish back into obscurity whenever Google modifies its ranking algorithms. Google does this mainly to thwart people who game the system to unfairly inflate their status, but the periodic adjustments can also have a devastating impact on legitimate sites who find the rules of search engine success changing in the middle of the game.

After a major reshuffling of the Google rankings in November 2003, Brandt published a tool he called Scroogle; it shows which sites are “missing” from Google’s top 100 search results for a given term, compared to the pre-November rankings. Outraged visitors have since boosted Google Watch’s traffic tenfold. “The Mom and Pop’ Web sites were collateral damage that didn’t deserve to get whacked,” says Brandt. He supports Cutting’s Nutch project and believes that a public entity, such as the U.S. Library of Congress, should develop Nutch into a comprehensive, noncommercial search tool to blunt Google’s influence.

As a private company, Brandt concedes, Google has the right to use its technology however it wants. But “as soon as their power infringes on the lives and livelihoods of other people,” he says, “it’s a matter of public concern.”

Return to “Search Beyond Google”