Monday, June 30, 2008

Search Engines' Chinese Self-Censorship

U.S.-based search engines are choosing what they censor in China and could be blocking more than they have to.

By Erica Naone

Credit: Technology Review

To operate in China, search engine companies based in the United States have built products that cooperate with China's policies of Internet censorship. That much has long been recognized. But a new analysis suggests that search companies, including Google, Microsoft, and Yahoo, are independently deciding what to censor and could be censoring more information than Chinese laws demand.

A report released last week by the Citizen Lab at the Munk Centre for International Studies at the University of Toronto found that different search engines are blocking fairly different content. "The low overlap means that companies are choosing the exact content to censor or, alternatively, to not censor," says Nart Villeneuve, a senior research fellow at the Citizen Lab and the author of the report. "That doesn't mean that they're not getting guidance from the Chinese government in other ways," he notes. But Villeneuve says that if search engines are interpreting Chinese policies to decide what to censor, that introduces the possibility that they may block more content than is strictly necessary.

The U.S. search engine companies say that they cooperate with Chinese censorship policies because some presence in the country is better than none at all. "We started Google.cn because Google.com was strangled in China," says Robert Boorstin, director of policy communications for Google. Though Boorstin says he can't comment on the specifics of Google's Chinese product, he says, "We were faced with a very clear choice: we can either start a new public library where people could see 98 percent of the stuff in the stacks, or we would have no library at all, and nobody would get a library card."

Indeed, according to Villeneuve's report, even with the self-censoring, foreign search engines provide about 20 percent more content on average than Internet users in China would otherwise be able to access through domestic search engines. But, he says, "The bigger issue is just that we don't know exactly what they're doing, and the search engine companies haven't been open publicly about what they're doing."

Villeneuve tested search engines made by Google, Microsoft, Yahoo, and Beijing-based Baidu. To make the testing fair, Villeneuve compared how the search engines handled specific keywords on specific sites--say, "Tiananmen Square" at news.bbc.co.uk. He also had to distinguish the effects of China's "Great Firewall," which blocks some information passing into or out of China, from the censorship of specific search engines. Since Yahoo and Baidu both host their Chinese products within China, their crawlers can index Chinese content without the firewall's interference. Users outside China, however, will see their results filtered. With Google and Microsoft, the situation is reversed. So Villeneuve tested Yahoo and Baidu from within China and Google and Microsoft from outside, avoiding the firewall.