The latest smart phones and tablets at the Consumer Electronics Show in Las Vegas last month came with an emphasis on faster processors and compatibility with faster wireless networks. But new research shows that the biggest performance bottleneck with popular smart-phone apps such as Facebook and Google Maps is, in fact, how fast they can read and write a device’s data storage. The results suggest that without changing how mobile gadgets store data, the benefits of new networks and processors will be limited.
“We spend a lot of time of our lives using these apps, but a large part of that is spent waiting—for a Web page to load or a [user interface] to refresh,” said Nitin Agrawal, a researcher in the labs of electronics manufacturer NEC, presenting at the Usenix File and Storage Technology conference in San Jose, California, last week. “We found that storage is the key bottleneck for mobile devices,” said Agrawal. “The storage systems on mobile devices require a fresh look.”
Agrawal and his colleagues Hyojun Kim and Cristian Ungureanu came to that conclusion after testing how the performance of a series of popular apps for Google’s Android mobile operating system was affected by the memory card inside a device. The apps included Facebook, Google Maps, Angry Birds, and Android’s Web browser.
Apps were put through their paces again and again while eight different memory cards from several manufacturers were swapped in and out to see whether they made apps more or less responsive. Android devices normally rely on internal memory chips when running apps, but to ease experimentation, the researchers tweaked the operating system to rely almost exclusively on a device’s removable memory card, which is typically used to store photos and music. But Agrawal says the findings should also apply to internal storage, which has similar properties.
Some of the results were striking. From a cold start, the Gmail app launched more than three times faster running on the best performing memory card than on the worst; the Twitter app launched more than twice as fast with faster storage. The Android Web browser was tested by asking it to navigate to and load 50 top Web pages one after another; it was around three times faster in the best case than in the worst. The tests were carried out on a Nexus One phone running the “Gingerbread” version of Android, which was only recently superseded by the latest Android release, Ice Cream Sandwich.
Agrawal says the fact that data storage limits performance suggests faster network speeds and processors, the areas that get the most attention from researchers, engineers, and marketers, could make little difference to how apps perform for users.
When tests were done using a cable to simulate a connection 10 times faster than Wi-Fi, apps hardly became more responsive. “We were expecting the performance to improve quite a bit with the faster connection, but that didn’t happen,” said Agrawal.
The problem, the researchers say, is partly the way that nonvolatile memory used inside phones and memory cards works. Product specifications suggest these chips should be able to read and write data faster than processors or wireless data links can serve it up, but in practice, they can’t. This is because benchmarks used to categorize memory are for reading or writing bits of data on a disk in sequence, but many of the popular apps in reality access bits in a less ordered way. That shortcoming was shown to be magnified by the way that many of the apps studied used data management code that relies heavily on random data access.
“Random write performance is orders of magnitude worse for mobile device storage,” and slower than a typical 3G network, said Agrawal. “Storage performance has clearly not kept pace with the improvements in the network.”
The NEC researchers tested a handful of strategies that could limit the chokehold of data storage on app performance. Facebook became four times faster when they forced it to use a different system of data caching, for example. That shows that design choices that app developers make can mitigate the data storage bottleneck, although removing that bottleneck would require changes to mobile operating systems and hardware.
Agrawal told Technology Review that he didn’t know how Apple’s iPhone might perform in similar tests. Apple devices use only internal storage, and their technology is similar to that used by Android devices. However, Apple’s iOS operating system and apps built for it store data in different ways.
After Agrawal’s presentation, Eno Thereska of Microsoft Research suggested that the results need to be validated with tests involving real users, to see if they notice the performance limits uncovered. He questioned whether these limits really even mattered. “I can only read a webpage so fast,” he said, “and have to admit I’m pretty satisfied with mobile apps. What is users’ perception of these experiences?”
Agrawal says his group has not yet tested whether users notice memory bottlenecks, but notes that most of their tests replicated how people use their apps. For example, for Google Maps, they timed how long it took to get results after “find directions” was tapped. Agrawal says he is confident that people could tell the difference and do find limitations imposed by memory frustrating. “Why would anyone want to see a 20-second wait if they can get away with five seconds?”