In 2001, Jesse Robbins applied for two jobs: one as a Seattle bus driver and another as a backup systems engineer at Amazon.com. Amazon called first, and in the decade that followed, Robbins transformed the way Web companies design and manage complex networks of servers and software.

A former volunteer firefighter, Robbins brought an emergency responder’s mind-set to his work. He taught Amazon that with data centers distributed around the world, a massive shopping site, and intricate fulfillment operations, some unpredictable and spectacular failures were inevitable. Rather than try to defy that inevitability, Robbins says, he made it safe for Amazon to fail, building fault tolerance into its architecture. Then he tested the Web operations teams with live drills, knocking entire data centers offline. Customers didn’t notice a thing.

After leaving Amazon in 2006, Robbins began blogging about his techniques. In 2007, he founded Velocity, now an annual conference, where fierce competitors such as Microsoft and Google share information about handling infrastructure problems.

In 2008, Robbins cofounded Opscode. Its main product, Chef, is an open-source programming language that automates management of cloud-based infrastructure. For example, one client used Chef to help scientists bring up and configure in 45 minutes a 10,000-­processor supercomputing cluster on Amazon’s pay-as-you-go cloud, solve some difficult problems related to protein binding in eight hours, and then close out the operation, for a small fraction of what it would cost to build or buy time on a supercomputer.

“When you look at ­YouTube, Flickr, Facebook, or similar services, there’s so much collaborative work going on. But we don’t really understand why,” says Judd Antin, a research scientist in the Internet Experiences Group at Yahoo Research. Indeed, many of the marvels of the Internet age, such as Wikipedia, have come from rethinking traditional ideas about how workers should be organized and rewarded. Antin is finding out what motivates people to participate in such projects, in hopes of attracting the broadest possible spectrum of contributors and decreasing the likelihood that the projects will fizzle out after the novelty wears off.

One finding: using gamelike approaches and software–for example, to prompt people to reveal their location to advertisers—might be overrated as a way to shape behavior. The popular tactic of rewarding people with points and badges doesn’t motivate them for long on most websites, Antin says.

But he still believes some game dynamics will remain effective in the long term, and he’s working to figure out what those are. He also wants to learn how motivation varies across cultures. With those tools, he hopes, organizations will be able to consistently nurture the Web’s collaborative spirit and turn it to good use.

Search engines are pretty good at matching keywords with relevant websites. Xiao Li is helping teach Bing, Microsoft’s search engine, to go a step further: figure out the specific task a user is trying to tackle with a query, whether it’s buying a digital camera or booking a hotel room, and return the most useful results related to that task.

Li created software that can automatically crunch through terabytes of Bing’s logs. By building relationships between keywords, the links people click, and the type of information presented on Web pages, the software can predict what a user is trying to accomplish, even if unfamiliar search terms are used. Once Bing determines the intent of a query, it can pass the query to one of a number of specialized search engines that index the most relevant subsets of the Web and can offer task-specific tools. For example, if Li’s system decides that a user searching for “pulled pork downtown” is more likely to be trying to make a dinner reservation than to find a butcher shop, it can kick the query over to a specialized engine that deals in restaurant searches, providing quick links to reviews and reservation systems.

Groupon is one of the fastest-growing companies of all time. Less than three years after CEO Andrew Mason launched it, it has more than 7,000 employees, operates in 43 countries, and is on pace for more than $2 billion in revenue this year. Its model is simple—it e-mails consumers with discount offers for goods and services in their cities–but it has taken off because previously there were few other cost-effective ways for small brick-and-mortar businesses to advertise to local customers.

Mason created Groupon by repurposing technology he’d developed in 2007 to help groups of people pledge to perform some civic action as long as a critical mass of users agreed to take part. The twist in Groupon was that the collective action would be buying something. Starting in Chicago, Mason began offering to deliver a daily promotional discount to consumers, who get the deal only if enough people sign up to make it worthwhile for the business. If not, the deal is off.

Groupon isn’t close to being profitable yet, because it’s spending hundreds of millions of dollars annually on marketing, sales-force expansion, and other things required to move on from startup mode. It also has to contend with new competitors hawking local bargains. Mason plans to adapt with, for instance, a mobile service called Groupon Now that lets people see offers near where they happen to be. Groupon Now is also more flexible for businesses, which can offer bargains at targeted times, such as when sudden cancellations leave a restaurant with empty tables to fill.

4Chan, founded in 2003 by Chris Poole (better known as “moot”), is one of the few corners of the Web that still celebrate faceless commentary and action. Poole’s commitment to anonymity has helped 4Chan acquire more than 12 million active users; it boasts 600 million page views a month.

The 4Chan discussion boards have given rise to Internet memes that have helped shape popular culture, such as the continuing vogue for abruptly inserting Rick Astley’s 1987 hit “Never Gonna Give You Up” into an online video (or live event). Less frivolously, the site was the birthplace of Anonymous, a collective of activist hackers who have targeted Scientology and companies that shunned WikiLeaks for publicizing government and corporate secrets.

When Poole built 4Chan, he did so with the concept of anonymity at its center, seeking to create a place where people have their mistakes forgotten rather than being haunted by everything they’ve ever posted. There is no registration system, and users can post anonymously under whatever pseudonym they choose, even one associated with another user. There is no archive: content uploaded to the site by users disappears as new images and commentary are added.

Poole hopes to apply the lessons he has learned from 4Chan to a recent startup called Canv.as, which will allow users to share and edit images collaboratively using a built-in editor—anonymously, of course.