The researchers aren't yet willing to disclose the performance scores of specific providers, but they plan to make their tools publicly available. "We are building a website where people will be able to download the software we used and see the results of the benchmarks," says Yang. "We gathered our initial data by running the trials for a few hours, sometimes over two days. But ideally they should run every day to provide live data on the clouds' performance."

"Porting your application to four different cloud providers with different APIs and deployment methods is tedious, expensive, and error prone," says Edward Walker, a researcher at Whitworth University in Washington, who separately developed a way to measure how good one of Amazon's cloud services was at crunching scientific data. "A tool that allows you to automatically predict the performance of your application on a cloud platform is useful."

Greater transparency in the cloud market could increase competition and ultimately result in better services for consumers, Walker says. However, he notes that the work Yang and her colleagues have done so far applies only to relatively simple Web applications. "More work needs to be done into investigating how feasible the framework will be for more complicated applications," he says.

Yang agrees. "If you know you app is computationally intensive--for example, because it does a lot of sorting--you could choose a provider based on the metrics we gather on speed," she says. "But if your app is 30 percent computation and 70 percent storage, it is harder to choose."

Thus, she and her colleagues have started work on a more sophisticated way to test cloud services. They are building software that captures the performance of an application when running on a local server and then creates a dummy version of it on several clouds to compare how they hold up. The dummy doesn't actually perform the function of the software, but it exerts the same computational, storage, and network demands. "Without actually migrating the app, I could just try out that representation of what it does," says Yang.