In 2006, as Facebook was starting to accumulate information about its users faster than the data could be analyzed and stored, Jeff Hammerbacher was brought in to deal with the problem. A former Wall Street number cruncher, he soon developed techniques for handling and mining unprecedented amounts of data.
Realizing that the company needed entirely new technology to handle the information overload, Hammerbacher threw Facebook’s muscle behind a relatively new open-source database project called Hadoop, which allows nearly real-time processing of data in quantities that had previously been impossible. Hadoop enabled Hammerbacher to create the suite of analytics that underpins Facebook’s targeted advertising system—the key to the company’s profitability.
Though much of the technology he constructed is still in use at Facebook, Hammerbacher left in 2008 to found a company called Cloudera, where he is now chief scientist. Cloudera is devoted to developing Hadoop and related open-source technologies. Hammerbacher points out that industries including oil and gas, retail, and life sciences are all dealing with increasing amounts of data. They could use such technology to extract valuable insights from the deluge. —Erica Naone