We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

A More Personalized Internet?

Yahoo Pipes lets people make highly customized feeds that combine information from multiple sources and weed out the junk.

Last week, Yahoo announced the release of an early version of a tool designed to help users personalize the Internet. The tool, called Pipes, lets people combine all sorts of oft-updated Internet information, known as feeds. Pipes could, for instance, enable a feed that includes New York Times articles featuring the phrase “plasma TV,” Flickr-posted pictures taken in a specific neighborhood, and traffic updates along a commute. So, instead of drowning in headlines from standard feed aggregators, the user gets information that is winnowed down and personal.

This image is a collection of screen shots taken from the Yahoo Pipes site. Pipes is a new Web-based tool that lets people customize the information they receive on the Web by combining and filtering online subscriptions known as feeds.

There are some early examples of Pipes on Yahoo’s site. One allows a person to search for an apartment near “something,” such as a park, library, or school. Another Pipe extracts keywords, such as “snow,” from the New York Times news feed and displays Flickr photos that have a matching tag.

The basic idea behind Pipes, says Yahoo, is to give software developers and motivated nondevelopers a simple programming tool to mix and match collections of data on the Web, says Pasha Sadri, principal software engineer at Yahoo and developer of Pipes. “The goal of Pipes,” he says, “is to significantly lower the barrier to writing simple applications by eliminating the need to write code and by hosting the application for you.”

In fact, no knowledge of a programming language such as C++ or Java is needed to build a Pipe. When you begin to build a new Pipe, you select a set of programming instructions that are premade and packaged as an icon, called a module. A Pipe is made by dragging and dropping these modules, stringing them together, and adding a few extra instructions. A similar approach is employed to program Lego Mindstorm robots. “This simplifies the process and means that more people will be able to write programs for very specific tasks,” says Sadri.

The tool consists of two major components: an interface, called an editor, where a Pipe is put together; and an execution engine that runs the Pipe instructions. Once a project is saved in the editor, the instructions are saved as a special kind of document on the engine. To run the Pipe, the engine reads the document and then accesses anywhere from dozens to hundreds of Web services–from feeds supplied by Craigslist to geography data on Yahoo Maps. To optimize the response time, says Sadri, the engine parallelizes as much of the execution as possible, breaking up the instructions into chunks that run simultaneously.

Almost immediately after its release, Pipes garnered a lot of attention from bloggers, software developers, and experts on Web-based applications. Perhaps the most glowing endorsement it received was from Tim O’Reilly, founder of O’Reilly Media, a computer-book, magazine, and online publisher. On his blog O’Reilly wrote that the tool is a “milestone in the history of the Internet.” He added that while it’s still a bit “rough around the edges,” Pipes has “enormous potential to turn the Web into a programmable environment for everyone.”

And it is the potential of Pipes that is exciting to many. “I think it’s great,” says Eric Lunt, chief technology officer at Feedburner, a Chicago-based company that manages feed publication information. As feeds become more integrated into Internet use, either from Internet Explorer 7 or applications such as Google Homepage, Lunt says, there will be an increased demand for tailoring feeds to individual needs. “It might take a couple of years for the impact to be felt,” he says, “but I think it will be.” Lunt adds that Pipes gives people the power of being able to take the huge amount of content that’s on the Web, repurpose it, and change it in various ways–and “you don’t have to jump into code to use this.”

However, familiarity with software code could help. While programming experience is not necessary, understanding the logic of stringing modules together and the detailed module functions might be easier with some training. (How-to discussions are available on the Pipes site, and a helpful step-by-step tutorial can be found here.) But Yahoo, says Sadri, was really aiming Pipes at a level in between expert developer and programming novice. “At this early state, our focus has been to establish a tool that is great for developers and allows them to create innovative [applications],” says Sadri. “Ultimately, however, we expect that average Internet users will be the ultimate beneficiaries of the system, though they may not know–or even care–that Pipes is the enabling technology.”

Some Internet experts question the transformative potential of Pipes. “Can you build useful systems with it?” asks Dan Weld, professor of computer science and engineering at the University of Washington, in Seattle. The programming interface is fairly easy to use, he says, but some of the functions are limited. For instance, one Pipes module functions as a filter that blocks selected information from getting through, but “the problem with the filters is [that] sometimes duplicates [such as multiple identical headlines] get through,” Weld says. He adds that ranking results, in particular, is tricky to do without modifying the instructions with low-level code. “That’s a reason to be a little skeptical about it,” he says.

Sadri says that Yahoo will tweak Pipes over time, using feedback from developers. He hopes to expand the scope of the tool–for instance, by allowing users to build a Pipe within a Pipe–to enable more-complex applications. And since the project is in its experimental phase, he says, he and his team are still working on new modules with varying functions, and they’re looking at ways to improve the scalability and usability of the Pipes site.

Tech Obsessive?
Become an Insider to get the story behind the story — and before anyone else.

Subscribe today
Want more award-winning journalism? Subscribe to Insider Basic.
  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning print magazine, unlimited online access plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Print Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.