Thursday, August 07, 2008

Cloud Computing's Perfect Storm?

An Intel, Yahoo, and HP initiative will use large-scale research projects to test a new Internet-based computing infrastructure.

By John Borland

Credit: Technology Review

Last week, Intel, Yahoo, HP, and an international trio of research institutions announced a joint cloud-computing research initiative. The ambitious six-site project is aimed at developing an Internet-based computer infrastructure stable enough to host companies' most critical data-processing tasks. The project also holds an unusual promise for advances in fields as diverse as climate change modeling and molecular biology.

The new array of six linked data centers, one operated by each project sponsor, will be one of the largest experiments to date focusing on cloud computing--an umbrella term for moving complex computing tasks, such as data processing and storage, into a network-connected "cloud" of external data centers, which might perform similar tasks for multiple customers.

The project's large scope will allow researchers to test and develop security, networking, and infrastructure components on a large scale simulating an open Internet environment. But to test this infrastructure, academic researchers will also run real-world, data-intensive projects that, in their own right, could yield advances in fields as varied as data mining, context-sensitive Web search, and communication in virtual-reality environments.

"Making this marriage of substantial processing power, computing resources, and data resources work efficiently, seamlessly, and transparently is the challenge," says Michael Heath, interim head of the computer-science department at the University of Illinois at Urbana-Champaign, an institute that is part of the alliance. Heath says that for the project to be successful, the team, which also includes Germany's Karlsruhe Institute of Technology and Singapore's Infocomm Development Authority, needs "to be running realistic applications."

Much of the technology industry has recently focused on cloud computing as a next critical architectural advance, but even backers say that the model remains technologically immature.

Web-based software and the ability to "rent" processing power or data storage from outside companies are already common. The most ambitious visions of cloud computing expand on this, predicting that companies will ultimately use remotely hosted cloud services to perform even their most complex computing activities. However, creating an online environment where these complicated tasks are secure, fast, reliable, and simple still presents considerable challenges.

Virtually every big technology company, including Google, IBM, Microsoft, and AT&T, already has a cloud-computing initiative. Farthest along commercially may be Amazon, whose Web Services division already hosts computing, storage, databases, and other resources for some customers.

The new cloud-computing project will consist of six computing clusters, one housed with each founding member of the partnership, with each containing between 1,000 and 4,000 processors. Each of the companies involved has a specific set of research projects planned, with many broadly focusing on operational issues such as security, load balancing, managing parallel processes on a very large scale, and how to configure and secure virtual machines across different locations.

Researchers will be given unusually broad latitude to modify the project's architecture from top to bottom, developing and experimenting with ideas applying to hardware, software, networking functions, and applications. Project managers say that one goal is to see how changes at one technical level affect others.