Visit Amazon Light at www.kokogiak.com/amazon4, and youll see a plain search box that allows you to locate any product in Amazon.coms database. Click on an item, and youll be taken to a page with the usual product image, price information, and customer reviews, and, of course, the familiar Buy This button. Amazon Lights pages are deliberately less cluttered than those at Amazon itself, but the family relationship is obvious.
Look closer, however, and youll spot some distinctly non-Amazonian features. If the item youre viewing is a DVD, for example, there will be a button that lets you see in a single click whether the same disc is for rent at Netflix. If its a CD, you can check whether Apples iTunes music store has a downloadable version. And if its a book, Amazon Light will even tell you whether its on the shelf at your local public library.
Whats going on here? Surely, executives at Seattle-based Amazon would never condone an online service that encourages people to buy things from sites other than Amazon?
Actually, they would. Amazon Light, created by former Amazon programmer Alan Taylor and hosted on his personal website, kokogiak.com, is one of thousands of independent sites incorporating the product data and programming tools that Amazon has been sharing freely since July 16, 2002. Thats the day Amazon celebrated its seventh anniversaryand unveiled a startling new project, called Amazon Web Services, that promises to change, once again, the way retailers of all stripes think about reaching their customers.
While companies such as Google and Microsoft are also experimenting with the idea of letting outsiders tap into their databases and use their content in unpredictable ways (see Whats Next for Google?), none is proceeding more aggressively than Amazon. The company has, in essence, outsourced much of its R&D, and a growing portion of its actual sales, to an army of thousands of software developers, who apparently enjoy nothing more than finding creative new ways to give Web surfers access to Amazon merchandiseand earning a few bucks in the process. The result: a syndicate of mini-Amazons operating at very little cost to Amazon itself and capturing customers who might otherwise have gone elsewhere. Its as if Starbucks were to recruit 50,000 of its most loyal caffeine addicts to strap urns of coffee to their backs each morning and, for a small commission, spend the day dispensing the elixir to their officemates.
Amazon is pouring so many resources into their Web services that its almost frightening, says Paul Bausch, one of the inventors of the well-known weblogging tool Blogger and, more recently, the author of OReilly Medias Amazon Hacks, a collection of tips for tapping into Amazons rich database. They are extremely aggressive, and that separates them from Google and from other people who are still just experimenting with the technology. They really believe that this is where their business is heading.
Making the Web Safe for Computers
The strategy behind Amazon Web Services is to give programmers virtually unlimited access to the very foundation of Amazons businessits product databasewhether they are inside or outside the companys walls. Developers can grab product data, reformat it, add related services, and use it to attract eyeballs to their own sites. If they feel like it, theyre even free, like Taylor, to create parallel-universe Amazons that have the added features they crave. Amazon demands only one thing in return: that visitors to these satellite sites complete any purchases through Amazon.com itself. The site owners, meanwhile, earn a decent commission on each sale.
Exposing the worlds largest product databasealong with the editorial content and personalization functions that make Amazon.com so uncannily usefulis such a counterintuitive business strategy* that analogies are hard to come by. In a way, the risk the company is taking is like the one Apple Computer has always avoided by refusing to license its Macintosh operating system to other manufacturers. Some observers still think Apple missed its golden opportunity to mount a comeback against Microsoft, while others maintain that the Mac OS is Apple, and that putting it on other machines would have diluted the brand. In Amazons case, outside programmers could find cleverer ways of using Amazons data than Amazon itself and end up sucking away so much traffic that Amazons own site cedes e-retailings center stage.
And yet Amazons move also reflects the spirit of the ageor at least, the spirit among the Webs technical avant-garde. As Amazon CEO Jeff Bezos put it in a speech at Octobers Web 2.0 conference in San Francisco, Web 1.0 was making the Internet for people; Web 2.0 is making the Internet better for computers.
Web services, whose name became a marketing meme years before the technology to make them work had actually matured, boil down to the simple but powerful idea that the Web should be more than just a way for human users to call up preformatted documents. It can also be a medium for software programs to communicate and share data with one another. Through a nonproprietary formatting scheme called the Extensible Markup Language, or XML, a string of data can be labeled according to typeas, say, a phone number, a price, or a book title. Web software can then harvest the data from a remote site and re-present it to the end user in any way a programmer wishes. That means a company with a trove of data it wants to share need only put it on the public Web and give programmers a few simple tools for accessing it.
Though these tools have a geeky and daunting nameapplication programming interfaces, or APIstheir function is easy to understand: they are short hooks of computer code that allow smaller programs to communicate with large software systems, such as Microsoft Windows. In the case of Web services, an API amounts to a set of commands programmers can use to interact with large, database-driven Web sites.
For a company that makes its APIs public, the real trick is figuring out how it will be rewarded. And in this respect, too, Amazon has done exceedingly well, signing up thousands of programmers who are sending it millions of hits a day, though the company wont disclose how much its earning through their satellite sites.
There isnt one tutelary genius responsible for Web services ascent at Amazon. Even Bezos was reportedly awakened to their possibilities only after a bit of prodding from friendly outsiders.
Rather, this is a tale of clever Amazon software engineers and marketing professionals who were attuned to emerging technologies like XML, the Simple Object Access Protocol (SOAP), and the Semantic Web, and who saw how their efforts in the 1990s to increase Amazon.coms accessibility led inexorably to the Web services model. Increasingly, its also the tale of tens of thousands of outside Web developers with a fixation on new technologies and an urgewith Amazons blessingto see just how far they can stretch that model.
The Information Hub
There are two versions of the story of Amazon Web Services genesis. One is told by Tim OReilly, a revered futurist and CEO of the software-manual publisher OReilly Media in Sebastopol, CA. OReilly says he began touting the idea of Web services as early as 2000 but wasnt seeing it blossom into the next-generation Internet programming language he envisioned. I went up to see Jeff Bezos with a Web services pitch, OReilly wrote in his weblog in July 2002. Amazon isnt just an e-commerce-site. Its become the information hub of the publishing industry. How about giving us some tools for building out services based on that hub? Once OReilly enumerated the ways Amazon itself would benefit, he says, Bezos became intrigued. Bezos then told me a day or two later that hed discovered that his skunk-works team already had a Web services API in the works. But he says that without my presentation he might have done something stupid like shutting the project down.
Naturally, Amazon software engineers tell a different story. Robert Frederick, a 31-year-old senior technical manager at Amazon, was part of the skunk-works team that Bezos consulted after his meeting with OReilly. Frederick dates the origins of the project all the way back to his hiring in 1999, when he was assigned to figure out how the rich information on Amazons website could be simplified and translated for display on the tiny screens of devices such as cell phones, PDAs, and pagers. These devices used an alphabet soup of different display formats, such as WML, CHTML, and XHTML. To expose this information in varying markup languages was extremely challenging, given the infrastructure that [Amazon had] put in place, says Frederick. But his team eventually came up with special Web server software that could retrieve product data and reformat it for a few select devices such as the Palm VII, one of the first wireless organizers.
This project, dubbed Amazon Anywhere, succeeded so well that Fredericks bosses asked him to equip Amazons entire battalion of Web servers with the software, the better to help partners present Amazon data in more ways. In order to do this, Frederick says, I advocated the separation of display logic from our business logic. More geek-speakbut that insight was perhaps the key to Amazon Web Services. It meant that Amazon needed to stop viewing its website as one giant application capable of storing product information, managing user accounts, sending product details to customers Web browsers, processing purchases, and the like. Rather, Frederick said, Amazon.com should be reconceived as a set of independent parts, including the database, shared APIs for accessing it and repackaging the data in XML format, and the final layout as displayed in browsers. In a series of meetings with senior managers in 2001 and early 2002, Frederick argued that the more these things were separated, the easier it would be for Amazons partners to build their own online storefronts that would draw on the Amazon infrastructure.
A few Amazon managers reacted cautiously. But the funny thing is that it did not take a great deal of convincing, recalls Frederick. The main concern was that we were going to expose valuable information and concepts that we had spent years developing. Absolutely true. But Frederick and his supporters argued that Amazons payback would be the innovative applications dreamed up by external developers. In theory, these applications would vastly increase the variety of contexts in which Web surfers might encounter products from Amazon.
Once we had incorporated XML, we realized that Wow, the future is here. We are the first to do this, says Jeff Barr, program manager for Amazon Web Services. So we decided to formalize it and invite the worlds developers to take a look, understand what it was all about, and start creating and innovating.
Developers accepted the invitation, and downloaded Amazons Web services APIs, help manuals, and sample code in droves (see Developing Interest, right). I was amazed that they did what they did, because companies are usually so reluctant to expose their databases, says Paul Bausch, who had become interested in Web services while building Blogger at San Franciscobased Pyra Labs. After Google acquired Pyra in 2003, Bausch returned to the life of an independent software developer and had so much fun playing with Amazons new programming interfaces that he ended up writing Amazon Hacks. Among other things, the book teaches readers how to embed Amazon product information in their Web pages and (on a less serious note) how to add pizzazz to their personal archives of MP3 songs by grabbing CD cover art from Amazon.
In one way, Amazon Web Services is simply an extension of longstanding collaborations such as Amazon Associates, in which external website owners can earn commissions of up to 10.5 percent for referrals. But previously, affiliate sites could only show simple links to Amazon products; with Web services, the possibilities became much broader (see Amazon Everywhere, below).
At one end of the spectrum are sites like Taylors Amazon Light that simply proffer Amazon in a different skin. At the other end are noncommercial sites where Amazon data isnt the main attraction but is used to enhance other offerings. One example is AllConsuming.net, which appeals to bibliophiles by scouring thousands of blogs every hour, listing the most talked-about books, and providing snippets from each bloggers comments. Designed by Erik Benson, another ex-Amazonian, the site dips into the Amazon database for cover images and other details about the listed books and offers links to each books page at Amazon.
Benson says AllConsuming isnt making him rich, but it does bring in enough commissions to cover its own hardware and bandwidth costs (about $400 a month). Other companies earn far more by acting as middlemen. Monsoon of Portland, OR, for example, builds software that helps its own customers use Amazons Web services to simplify inventory management.
By November 2004, the number of developers participating in Amazon Web Services had grown to 65,000. To keep up with their demands, the company has kept updating its APIs to open up more types of product information and more functions, such as wish lists and advanced searches. How many purchases originate each day with Amazons growing web of syndicated storefronts? The company wont say, but experts have estimated that sites using the companys Web services send 10 million requests a day to Amazons servers.
Amazon is far from the only company exploring Web services. IBM, for example, has opened its Websphere server software to outside developers and expects to invest $1 billion this year in new business-to-business Web services, according to Michael Liebow, director of Web services for IBM Global Services, the companys consulting wing. One IBM creation: a system that uses XML and other standards to tie together the disparate databases used by merchants, banks, and credit-card firms, helping to resolve disputed credit-card charges faster.
In fact, Web services biggest impact may not be the syndication of individual businesses information, as in Amazons case, but the standardization of business processes across whole industries, such as finance, electronics, or automobiles, according to Liebow. Amazon is unique, he says. Its kind of a closed system, and theres a level of control.
Thats trueand Amazon reserves the right to shut down its Web services at any time. But doing so would destroy the rare symbiosis that has emerged from years of careful community building. Developers are another kind of customer for us, says Jeff Barr. The work they do is going to bring even more diverse types of customers. In other words, by sharing not just its data but also its retailing tools and a modest slice of the profits, Amazon has turned a programming subculture typically ruled by anticorporate suspicion and paranoia into a wellspring of evangelism, not to mention a funnel for revenue. Surely, thats one for the books.
Geoffrey Hinton tells us why he’s now scared of the tech he helped build
“I have suddenly switched my views on whether these things are going to be more intelligent than us.”
ChatGPT is going to change education, not destroy it
The narrative around cheating students doesn’t tell the whole story. Meet the teachers who think generative AI could actually make learning better.
Meet the people who use Notion to plan their whole lives
The workplace tool’s appeal extends far beyond organizing work projects. Many users find it’s just as useful for managing their free time.
Learning to code isn’t enough
Historically, learn-to-code efforts have provided opportunities for the few, but new efforts are aiming to be inclusive.
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.