The Information Hub
There are two versions of the story of Amazon Web Services’ genesis. One is told by Tim O’Reilly, a revered futurist and CEO of the software-manual publisher O’Reilly Media in Sebastopol, CA. O’Reilly says he began touting the idea of Web services as early as 2000 but wasn’t seeing it blossom into the next-generation Internet programming language he envisioned. “I went up to see Jeff Bezos with a Web services pitch,” O’Reilly wrote in his weblog in July 2002. “Amazon isn’t just an e-commerce-site. It’s become the information hub of the publishing industry. How about giving us some tools for building out services based on that hub?” Once O’Reilly enumerated the ways Amazon itself would benefit, he says, Bezos became intrigued. Bezos then “told me a day or two later that he’d discovered that his skunk-works team already had a Web services API in the works. But he says that without my presentation he ‘might have done something stupid like shutting the project down.’”
Naturally, Amazon software engineers tell a different story. Robert Frederick, a 31-year-old senior technical manager at Amazon, was part of the skunk-works team that Bezos consulted after his meeting with O’Reilly. Frederick dates the origins of the project all the way back to his hiring in 1999, when he was assigned to figure out how the rich information on Amazon’s website could be simplified and translated for display on the tiny screens of devices such as cell phones, PDAs, and pagers. These devices used an alphabet soup of different display formats, such as WML, CHTML, and XHTML. “To expose this information in varying markup languages was extremely challenging, given the infrastructure that [Amazon had] put in place,” says Frederick. But his team eventually came up with special Web server software that could retrieve product data and reformat it for a few select devices such as the Palm VII, one of the first wireless organizers.
This project, dubbed Amazon Anywhere, succeeded so well that Frederick’s bosses asked him to equip Amazon’s entire battalion of Web servers with the software, the better to help partners present Amazon data in more ways. “In order to do this,” Frederick says, “I advocated the separation of display logic from our business logic.” More geek-speak–but that insight was perhaps the key to Amazon Web Services. It meant that Amazon needed to stop viewing its website as one giant application capable of storing product information, managing user accounts, sending product details to customers’ Web browsers, processing purchases, and the like. Rather, Frederick said, Amazon.com should be reconceived as a set of independent parts, including the database, shared APIs for accessing it and repackaging the data in XML format, and the final layout as displayed in browsers. In a series of meetings with senior managers in 2001 and early 2002, Frederick argued that the more these things were separated, the easier it would be for Amazon’s partners to build their own online storefronts that would draw on the Amazon infrastructure.
A few Amazon managers reacted cautiously. But “the funny thing is that it did not take a great deal of convincing,” recalls Frederick. “The main concern was that we were going to expose valuable information and concepts that we had spent years developing.” Absolutely true. But Frederick and his supporters argued that Amazon’s payback would be the innovative applications dreamed up by external developers. In theory, these applications would vastly increase the variety of contexts in which Web surfers might encounter products from Amazon.
“Once we had incorporated XML, we realized that ‘Wow, the future is here. We are the first to do this,’” says Jeff Barr, program manager for Amazon Web Services. “So we decided to formalize it and invite the world’s developers to take a look, understand what it was all about, and start creating and innovating.”