Cloud Programming

TOBY BURDITT

Cloud computing offers the promise of virtually unlimited processing and storage power, courtesy of vast data centers run by companies like Amazon and Google. But programmers don’t know how best to exploit this power.

Today, many developers are converting existing programs to run on clouds, rather than creating new types of applications that could work nowhere else. And they are held back by difficulties in keeping track of data and getting reliable information about what’s going on across a cloud. If programmers could solve those problems, they could start to really take advantage of what’s possible with a cloud. For example, an online music retailer could monitor popular social-media feeds; if a singer suddenly became a hot topic, advertising and special offers across the retailer’s site could be instantly reconfigured to make the most of the spike in interest.

At the University of California, Berkeley, Joseph Hellerstein thinks he can make it much easier to write complex cloud applications by developing software that takes over the job of tracking data and keeping tabs on what’s happening. His big idea is to modify database programming languages so that they can be used to quickly build any sort of application in the cloud–social networks, communication tools, games, and more. Such languages have been refined over the years to hide the complexities of shuffling information in and out of large databases. If one could be made cloud-friendly, programmers could just think about the results they want, rather than micromanaging data.

The challenge is that these languages process data in static batches. They can’t process data that is constantly changing, such as readings from a network of sensors. The solution, ­Hellerstein explains, is to build into the language the notion that data can be dynamic, changing as it’s being processed. This sense of time enables a program to make provisions for data that might be arriving later–or never.

The result is called Bloom. So far, Hellerstein’s group has used the Bloom language and its predecessors to quickly rebuild and add major features to popular cloud tools such as Hadoop, a platform used to manipulate very large amounts of data. By lowering the complexity barrier, these languages should increase the number of developers willing to tackle cloud programming, resulting in a wave of ideas for new types of powerful applications.

Hellerstein’s group is getting Bloom ready for a release in late 2010. They and others are also working on demonstrating how the techniques can be used for real-time applications such as online multiplayer games, or to watch for the warning signs of an earthquake or tsunami.