From Facebook to the Department of Motor Vehicles, the world is catalogued in databases. No one knows it better than MIT adjunct professor and entrepreneur Michael Stonebraker, who has spent the last 25 years developing the technology that made it so. He got his big break by inventing and commercializing technology that underlies most of the databases, known as relational databases, that rule today. But Stonebraker now happily calls his earlier inventions largely obsolete. He’s working on a new generation of database technology that can handle the flood of digital data that is starting to overwhelm established methods.
“Relational databases are omnipresent as the solution for enterprise data. They have been fabulously successful,” Stonebraker says. But he says that the largest database vendors, including Oracle, IBM, and Microsoft, still sell such products as being appropriate for any business. Stonebraker has a different view: that new database technologies are required to handle the exponential increases in the information that businesses must handle. Stonebraker, 67, is already finding success with several of his own new approaches.
One is a database system called C-Store. Unlike most systems in use today, it stores data on disk column by column, not row by row. That simple tweak required a complete rewrite of how databases have worked, but it dovetails neatly with both the way computer memory works and the way databases are accessed. That yields much faster performance and more compressed data.
That tweak and others made by Stonebraker and colleagues at MIT, Brown, Brandeis, Yale, and the University of Massachusetts enabled the launch of Vertica, a company that commercialized C-Store and helped customers to query large databases almost in real time. Vertica was acquired by Hewlett-Packard in February and boasts clients including Comcast, which uses it to monitor the millions of devices that make up its TV and Internet networks, and Groupon, which uses it to analyze the actions of its millions of subscribers.
A related system from Stonebraker and some of the same academic colleagues, H-Store, builds on the same ideas with extra improvements such as running entirely in a computer’s memory, not on disk; this method is particularly useful in online transaction processing. H-Store’s code is open source, but the technology is being commercialized by venture-backed VoltDB, with Stonebraker as CTO. He argues that this kind of use-specific, built-for-speed databasing system is what most enterprises will need to adopt sooner rather than later to deal with the flood of digital data.