Technology Review

Computing

A Faster Way to the Cloud

Amazon's new protocol should make accessing the cloud faster and more reliable.

  • Friday, September 11, 2009
  • By Duncan Graham-Rowe

Cloud computing offers a cheap way to carry out data-intensive computing, allowing companies to effectively lease processing power from an online supplier. But uploading large amounts of data to cloud computing systems has remained costly and time-consuming.

Today, Amazon announced a new, ultrafast file transfer protocol designed to make uploading to its cloud service easier. The move could broaden the appeal of cloud computing by allowing smaller organizations and even individuals to upload data without expensive infrastructure.

"The biggest bottleneck in cloud computing is without a doubt the data transmission--uploading and downloading data to and from the cloud," says Ian Sommerville, of the Co-laboratory for Cloud Computing at the University of St. Andrews, Scotland. Small businesses often must choose between enduring slow data transfer rates or investing in extra infrastructure, Sommerville says.

The heart of the problem is the way one of the Internet's core features--the transmission control protocol (TCP)--works. TCP regulates the flow of data by breaking it up into small packets of information, sending each packet, and then waiting for an acknowledgment that the packet has been received before sending the next one. If a packet does not arrive, TCP either resends it or assumes that the network is being overloaded and initiates an aggressive congestion-control strategy, slowing the data rate down to avoid triggering a network collapse.

Advertisement

While TCP works fine for sending relatively small amounts of data over small distances, it can cause major headaches for cloud-computing customers. The distance that data has to travel--measured geographically as well as by the number of network nodes it has to pass through--affects the number of errors that creep into the signal. For example, transferring data across the United States on a 100-megabits-per-second (Mbps) Internet link can result in a latency of 100 milliseconds and a loss of about 1 percent of packets, which translates to real transfer rates of just 10 Mbps or less.

Nick Trigg of Constellation Technologies, a cloud-computing company spun out of the Rutherford Appleton Laboratory, in Oxfordshire, U.K., and CERN, in Geneva, Switzerland, says that TCP can be a dramatic bottleneck for large amounts of data. This means that sometimes it is faster to physically deliver data on a disk than to upload it, he says.

To solve this problem, Amazon Web Services will use technology developed by Aspera, based in Emeryville, CA, called the Fast And Secure Protocol (FASP).

"Our core technology is an alternate bulk data moving protocol," says Michelle Munson, Aspera's CEO and cofounder. "The inefficiency [with TCP] is really very noticeable when transferring large amounts of data," she says.

Print

Related Articles

Apple Puts the Cloud into All Its Devices

Photos, songs, and documents created on one Apple device will soon magically appear on others you own.

Why Didn't Apple Launch a Music Streaming Service?

Data-hungry apps are colliding with limited data plans.

A Faster Wireless Web

A new protocol called fasp-AIR promises speedier mobile downloads.

Close Comments

To comment, please sign in or register

Forgot my password

andy_mydur

6 Comments

  • 886 Days Ago
  • 09/11/2009

cloud computing

With the ever increasing data uploading & transmission "cloud computing " will play an decesive role in the Internet  world.

Reply

  • 886 Days Ago
  • 09/11/2009

Image

It's unfortunate that the image of a tower of binary numbers terminating in a cloud of smoke was chosen for page 1 of this story on 9.11.2009.  On any other day I wouldn't think twice about it, but today it turns my stomach.

Reply

mtarabul

6 Comments

  • 886 Days Ago
  • 09/11/2009

Would this be applicable to torrenting? If everyone used this protocol could it increase upload and download speeds?

Reply

ms

190 Comments

  • 885 Days Ago
  • 09/12/2009

TCP misinformation

The description of TCP in this article is misleading. While it is true that you can make TCP behave as described, that is by no means the only option. TCP provides for negotiable packet size, a negotiable size for a sliding window of outstanding (unacknowledged) packets, and also provides for selective acknowledgment of packets. This means that TCP can efficiently utilize a large bandwidth with significant latency and packet loss. It pipelines packet transmissions and only retransmits lost packets. There are issues with TCP that might require a different protocol (usually based on UDP), but they don't appear to be described in the article.

Reply

theradicalmoderate

48 Comments

  • 881 Days Ago
  • 09/16/2009

Re: TCP misinformation

While it is true that TCP can be implemented to achieve very high transfer rates, such implementations are not conformant to the standard and are usually deemed bad network citizens.  The reason has to do with congestion control.

TCP has a behavior built into it called "slow start," which causes TCP senders to stop and wait for each one or two packets to be acknowledged when it starts up or, after stabilizing, it detects a missing acknowledgment.  After dropping into slow start, the sender will rapidly increase the amount of data sent before requiring an acknowledgment, then attempt to stabilize the amount of data sent at a size appropriate for avoiding congestion.  (I'm oversimplifying this a bit...)  The net result is that TCP is intentionally very sensitive to packet loss, which is usually a very good indication of network congestion.

Slow start was added to TCP to prevent it from congesting intervening network routers, and then continuing to overload them until they underwent congestive collapse.  In essence, slow start causes any individual connection to behave suboptimally, but it tends to make transfer over the internet as a whole, with tens of thousands of TCP connections, as efficient as possible. 

Congestion control is very, very tricky.  I wonder if FASP has been engineered adequately to avoid congestion problems.  If not, you'll see a lot of pushback from tier 1 and 2 ISPs and FASP will go the way of the other twenty or thirty attempts to engineer something better than TCP.

Reply

Advertisement

MAGAZINE

Can We Build Tomorrow's Breakthroughs?

Manufacturing in the United States is in trouble. That's bad news not just for the country's economy but for the future of innovation.

Sponsored Content

Technologies from National Instruments

Adding Data Logging
Log measured data to a file and open it in Microsoft Excel

> Click here for more National Instruments Videos <
Whitepaper

Temperature Measurements with Thermocouples: How-To Guide

This document is part of the “How-To Guide for Most Common Measurements” centralized resource portal. This tutorial provides a detailed guide for measurement and device considerations to take temperature measurements using thermocouples. Get an introduction to thermocouples, which are inexpensive sensing devices widely used with PC-based data acquisition systems. Also review some specific thermocouple examples and learn how thermocouples work and ways to integrate them into a data acquisition measurement system.

View full PDF > Listen to story >
Find us on Youtube

Videos

A Robot Recruit that Can Do It All

More

Advertisement

Technology Review Lists

TR50

Our list of the 50 most innovative companies, including the following:

Amazon.com

A123 Systems

Geron

ARM Holdings

More

Advertisement

Facebook

Advertisement