Skip to Content

Peering into Video’s Future

The Internet is about to drown in digital video. Hui Zhang thinks peer-to-peer networks could come to the rescue.
March 12, 2007
JOHN HERSEY

Ted Stevens, the 83-year-old senior senator from Alaska, was widely ridiculed last year for a speech in which he described the Internet as "a series of tubes." Yet clumsy as his metaphor may have been, Stevens was struggling to make a reasonable point: the tubes can get clogged. And that may happen sooner than expected, thanks to the exploding popularity of digital video.

TV shows, YouTube clips, animations, and other video applications already account for more than 60 percent of Internet traffic, says CacheLogic, a Cambridge, England, company that sells media delivery systems to content owners and Internet service providers (ISPs). "I imagine that within two years it will be 98 percent," adds Hui Zhang, a computer scientist at Carnegie Mellon University. And that will mean slower downloads for everyone.

Zhang believes help could come from an unexpected quarter: peer-to-peer (P2P) file distribution technology. Of course, there's no better playground for piracy, and millions have used P2P networks such as Gnutella, Kazaa, and BitTorrent to help themselves to copyrighted content. But Zhang thinks this black-sheep technology can be reformed and put to work helping legitimate content owners and Internet-backbone operators deliver more video without overloading the network.

For Zhang and other P2P proponents, it's all a question of architecture. Conventionally, video and other Web content gets to consumers along paths that resemble trees, with the content owners' central servers as the trunks, multiple "content distribution servers" as the branches, and consumers' PCs as the leaves. Tree architectures work well enough, but they have three key weaknesses: If one branch is cut off, all its leaves go with it. Data flows in only one direction, so the leaves'--the PCs'--capacity to upload data goes untapped. And perhaps most important, adding new PCs to the network merely increases its congestion--and the demands placed on the servers.

In P2P networks, by contrast, there are no central servers: each user's PC exchanges data with many others in an ever-shifting mesh. This means that servers and their overtaxed network connections bear less of a burden; data is instead provided by peers, saving bandwidth in the Internet's core. If one user leaves the mesh, others can easily fill the gap. And adding users actually increases a P2P network's power.

There are just two big snags keeping content distributors and their ISPs from warming to mesh architectures. First, to balance the load on individual PCs, the most advanced P2P networks, such as BitTorrent, break big files into blocks, which are scattered across many machines. To re­assemble those blocks, a computer on the network must use precious bandwidth to broadcast "metadata" describing which blocks it needs and which it already has.

Second, ISPs are loath to carry P2P traffic, because it's a big money-loser. For conventional one-way transfers, ISPs can charge content owners such as Google or NBC.com according to the amount of bandwidth they consume. But P2P traffic is generated by subscribers themselves, who usually pay a flat monthly fee regardless of how much data they download or upload.

Zhang and others believe they're close to solving both problems. At Cornell University, computer scientist Paul Francis is testing a P2P system called Chunkyspread that combines the best features of trees and meshes. Members' PCs are arranged in a classic tree, but they can also connect to one another, reducing the burden on the branches.

Just as important, Chunkyspread reassembles files in "slices" rather than blocks. A slice consists of the nth bit of every block--for example, the fifth bit in every block of 20 bits. Alice's PC might obtain a commitment from Bob's PC to send bit five from every block it possesses, from Carol's PC to send bit six, and so forth. Once these commitments are made, no more metadata need change hands, saving bandwidth. In simulations, Francis says, Chunkyspread far outperforms simple tree-based multicast methods.

Zhang thinks new technology can also make carrying P2P traffic more palatable for ISPs. Right now, opera­tors have little idea what kind of data flows through their networks. At his Pittsburgh-based stealth startup, Rinera Networks, Zhang is developing software that will identify P2P data, let ISPs decide how much of it they're willing to carry, at what volume and price, and then deliver it as reliably as server-based content distribution systems do--all while tracking everything for accounting purposes. "We want to build an ecosystem such that service providers will actually benefit­ from P2P traffic," Zhang explains. Heavy P2P users might end up paying extra fees--but in the end, content owners and consumers won't gripe, he argues, since better accounting should make the Internet function more effectively for everyone.

If this smells like a violation of the Internet's tradition of network neu­trality­--­­­­­­­­­­­­­­­­­­­­­
the­ principle that ISPs should treat all bits equally, regardless of their origin--then it's because the tradition needs to be updated for an era of very large file transfers, Zhang believes. "It's all about volume," he says. "Of course, we don't want the service providers to dictate what they will carry on their infra­structure. On the other hand, if P2P users benefit from transmitting and receiving more bits, the guys who are actually transporting those bits should be able to share in that."

Networking and hardware companies have their eyes on technologies emerging from places like Rinera and Francis's Cornell lab, even as they build devices designed to help consumers download video and other files over P2P networks. Manufacturers Asus, Planex, and QNAP, for example, are working with BitTorrent to embed the company's P2P software in their home routers, media servers, and storage devices. With luck, ­Senator ­Stevens's tubes may stay unblocked a little longer.

Keep Reading

Most Popular

Large language models can do jaw-dropping things. But nobody knows exactly why.

And that's a problem. Figuring it out is one of the biggest scientific puzzles of our time and a crucial step towards controlling more powerful future models.

OpenAI teases an amazing new generative video model called Sora

The firm is sharing Sora with a small group of safety testers but the rest of us will have to wait to learn more.

Google’s Gemini is now in everything. Here’s how you can try it out.

Gmail, Docs, and more will now come with Gemini baked in. But Europeans will have to wait before they can download the app.

This baby with a head camera helped teach an AI how kids learn language

A neural network trained on the experiences of a single young child managed to learn one of the core components of language: how to match words to the objects they represent.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at customer-service@technologyreview.com with a list of newsletters you’d like to receive.