Select your localized edition:

Close ×

More Ways to Connect

Discover one of our 28 local entrepreneurial communities »

Be the first to know as we launch in new countries and markets around the globe.

Interested in bringing MIT Technology Review to your local market?

MIT Technology ReviewMIT Technology Review - logo


Unsupported browser: Your browser does not meet modern web standards. See how it scores »

{ action.text }

Myth #2: The Net Is Too Interconnected To Control

Before BearShare came Gnutella, a program written by Justin Frankel and Tom Pepper. Frankel and Pepper were the two lead figures in Nullsoft, a tiny software firm that America Online purchased in June 1999 for stock then worth about $80 million. Rather than resting on their laurels after the buyout, Frankel and Pepper became intrigued by the possibilities of file swapping that arose in the wake of Napster. When college network administrators tried to block Napster use on their campuses, Frankel and Pepper spent two weeks throwing together Gnutella, file-swapping software that they thought would be impossible to block. They released an experimental, unfinished version on March 14, 2000. To their surprise, demand was so immediate and explosive that it forced the unprepared Pepper to shut down the Web site almost as soon as it was launched. Within hours of Gnutella’s release, an embarrassed AOL pulled the plug on what it characterized as an “unauthorized freelance project.”


It was too late. In an example of the seeming impossibility of stuffing the Internet cat back into the bag, thousands of people had already downloaded Gnutella. Amateur programmers promptly reverse-engineered the code and posted non-AOL versions of Gnutella on dozens of new Gnutella Web sites. Unlike Napster or Swaptor, Gnutella lets every user directly search every other user’s hard drive in real time. With member computers connecting directly to each other, rather than linking through powerful central servers, these “peer-to-peer” networks have no main hub, at least in theory. As a result, there is no focal point, no single point of failure, no Gnutella world headquarters to sue or unplug. “Gnutella can withstand a band of hungry lawyers,” crows the Gnutella News Web site. “It is absolutely unstoppable.”

Peer-to-peer networks have a number of important advantages, such as the ability to search for documents in real time, as opposed to looking for them in the slowly compiled indexes of search engines such as Google and HotBot. Excited by these possibilities, such mainstream firms as Intel and Sun Microsystems have embraced peer-to-peer network technology. But the focus of interest, among both the proponents and critics of peer-to-peer networks, has been the purported impossibility of blocking them. “The only way to stop [Gnutella],” declared Thomas Hale, former CEO of the Web-music firm WiredPlanet, “is to turn off the Internet.”

Such arguments have been repeated thousands of times in Internet mailing lists, Web logs and the press. But the claims for peer-to-peer’s uncontrollability don’t take into consideration how computers interact in the real world; a network that is absolutely decentralized is also absolutely dysfunctional. In consequence, the way today’s Gnutella networks actually work is quite different from the way they have been presented in theory.

To begin, each Gnutella user isn’t literally connected to every other user-that would place impossibly high demands on home computers. Instead, Gnutellites are directly connected to a few other machines on the network, each of which in turn is connected to several more machines, and so on. In this way, the whole network consists of hundreds or thousands of overlapping local clusters. When users look for a file, whether it is a copy of the Bible, a bootleg of A.I. or smuggled documents on the Tiananmen massacre, they pass the request to their neighbors, who search through the portion of their hard drives that they have made available for sharing. If the neighbors find what is being looked for, they send the good news back to the first machine. At the same time, they pass on the search request to the next computer clusters in the Gnutella network, which repeat the process.

Hopping through the network, the search is repeated on thousands of machines-which leads to big problems. According to a report in December by Kelly Truelove of Clip2, a Palo Alto, CA-based consulting group that specializes in network-performance analysis, a typical Gnutella search query is 70 bytes long, equivalent to a very small computer file. But there are a great many of them-as many as 10 per second from each machine to which the user is connected. In addition, there is a constant flow of “ping” messages: the digital equivalent of “are you there?” Inundated by these short messages, the 56 kilobit-per-second modems through which most people connect to the Net are quickly overwhelmed by Gnutella. Broadband connections help surprisingly little; the speed with which the network processes requests is governed by the rate at which its slowest members can pass data through.

With BearShare, Vinnie Falco developed one potential fix. BearShare, like other new Gnutella software, automatically groups users by their ability to respond to queries, ensuring that most network traffic is routed through faster, more responsive machines. These big servers are linked into “backbone” chains that speed along most Gnutella search requests. Further unclogging the network, Clip2 has developed “reflectors”-large servers, constantly plugged into the Gnutella network, that maintain indexes of the files stored on adjacent machines. When reflectors receive search queries, they don’t pass them on to their neighbors. Instead they simply answer from their own memories-“yes, computer X has this file.” Finally, to speed the process of connecting to Gnutella, several groups have created “host caches,” servers that maintain lists of the computers that are on the Gnutella network at a given time. When users want to log on, they simply connect with these host caches and select from the list of connected machines, thus avoiding the slow, frustrating process of trying to determine who else is online.

As their capacity improved, Gnutella-like networks soared in popularity. Napster, buffeted by legal problems, saw traffic decline 87 percent between January and May, according to the consulting firm Webnoize. Meanwhile, LimeWire, another Gnutella company, reported that the number of Gnutella users increased by a factor of 10 in the same period. “The networks are unclogging, and as a result they’re growing,” Truelove says. “And the content industries should be concerned about that.”

But the problem with these fixes is that they reintroduce hierarchy. Gnutella, once decentralized, now has an essential backbone of important computers, Napster-like central indexes and permanent entryway servers. “We’ve put back almost everything that people think we’ve left out,” says Gene Kan, a programmer who is leading a peer-to-peer project at Sun. “Ease of use always comes at some expense, and in this case the expense is that you do have a few points of failure that critically affect the ability to use the network.”

Rather than being composed of an uncontrollable, shapeless mass of individual rebels, Gnutella-type networks have identifiable, centralized targets that can easily be challenged, shut down or sued. Obvious targets are the large backbone machines, which, according to peer-to-peer developers, can be identified by sending out multiple searches and requests. By tracking the answers and the number of hops they take between computers, it is possible not only to identify the Internet addresses of important sites but also to pinpoint their locations within the network.

Once central machines have been identified, companies and governments have a potent legal weapon against them: their Internet service providers. “Internet service providers enjoy limitations on liability for their users’ actions if they do certain things specified by law,” says Jule Sigall, an Arnold and Porter lawyer who represents copyright owners. “If you tell them that their users are doing something illegal, they can limit their exposure to money damages if they do something about it when they are notified.” Internet service providers, he says, do not want to threaten their customers, “but they like not being sued even more, so they’ve been cooperating pretty wholeheartedly” with content owners.

As Ballon of Manatt, Phelps and Phillips notes, Gnutella traffic has a distinctive digital “signature.” (More technically, the packets of Gnutella data are identified in their headers.) Content companies are also learning how to “tag” digital files. The result, in Ballon’s view, is easy to foresee: “At a certain point, the studios and labels and publishers will send over lists of things to block to America Online, and 40 percent of the country’s Net users will no longer be able to participate in Gnutella. Do the same thing for EarthLink and MSN, and you’re drastically shrinking the pool of available users.”

Perhaps sensing that Gnutella cannot escape the eye of authority, bleeding-edge hackers have searched for still better solutions. Determined to create a free-speech haven, a Scottish activist/programmer named Ian Clarke in 1999 began work on a Gnutella-like network called Freenet that would be even more difficult to control, because it would encrypt all files and distribute them in chunks that constantly shifted location. Unsurprisingly, it has attracted enormous media attention. But the system is so incomplete-searchability is an issue-that one cannot judge whether it will ever be widely used. (A small number of people are already using Freenet. Most of them are pornography fans, but a few, according to Clarke, are Chinese dissidents who employ Freenet to escape official scrutiny.) Even if Freenet does not end up in the crowded graveyard of vaporware, Internet service providers can always pull the plug-treating Freenet, in essence, as an unsupported feature, in the way that many providers today do not support telnet, Usenet and other less popular services.

0 comments about this story. Start the discussion »

Tagged: Web

Reprints and Permissions | Send feedback to the editor

From the Archives


Introducing MIT Technology Review Insider.

Already a Magazine subscriber?

You're automatically an Insider. It's easy to activate or upgrade your account.

Activate Your Account

Become an Insider

It's the new way to subscribe. Get even more of the tech news, research, and discoveries you crave.

Sign Up

Learn More

Find out why MIT Technology Review Insider is for you and explore your options.

Show Me