We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Emerging Technology from the arXiv

A View from Emerging Technology from the arXiv

The Hidden Damage From Waste Data (And How To Deal With It)

Waste data needlessly burns power and degrades computer memory. That’s why we need a coherent plan to reduce, reuse and recycle data that has been left to die, argue computer scientists

  • July 4, 2011

Back in 1999, a computer scientist at Cornell University began monitoring the way that the Windows NT 4.0 operating system used files. What he found was astonishing.

About 80 per cent of all files that NT creates are either over-written or deleted within 5 seconds of being born.

Today, Ragib Hasan and Randal Burns at Johns Hopkins University in Baltimore say this ought to give programmers pause for thought. Deleting data requires energy, which means that a substantial fraction of a computer system’s energy budget is currently devoted to creating and then almost immediately scrubbing data.

And if the wasted energy weren’t bad enough, computer memory has a limited life span. Flash memory, for example, has a lifespan of 100,000 cycles. So cycling it needlessly brings the inevitable breakdown closer

Surely, there’s a better way, say Hasan and Burns.

As it turns out, waste management is a maturing discipline, at least as far as physical waste is concerned. Why not use the same ideas in the data industry that are now used to manage physical waste elsewhere, they suggest.

In many ways, the ideas are easy to translate. Physical waste falls into four categories which translate easily into the virtual world:

  • Unintentional data. Data unintentionally created, as a side effect or by-product of a process, with no purpose
  • Used data. Good data that has served its purpose and is no longer useful to the user
  • Degraded data. Data that has degraded in quality such that it is no longer useful to the user
  • Unwanted data. Data that was never useful to the user

Why not apply the three Rs of conventional waste management to this virtual rot. In other words, use well known mantra of ‘reduce, reuse, recycle’ to better manage data

Hasan and Burns suggest that operating systems could provide incentives for applications to reduce the amount of waste files they generate, perhaps by reducing the I/O bandwidth or scheduling fewer CPU cycles to the worst offenders.

“This concept is equivalent to the Pay-as-You-Throw scheme and the polluter-pays principle used in real life waste management,” they say.

They also suggest setting up “digital landfills”, made of a semi-volatile storage medium that gradually degrade in time. “Unwanted data objects will fade automatically and the storage space can be reclaimed,” say the pair from Johns Hopkins.

It’s tempting to say that these ideas sound worthy but are otherwise impractical and unrealistic. But there are substantial benefits to be reaped from this approach, in particular for portable devices which are limited by battery life.

What’s needed, of course, is a hard core grass roots movement that campaigns for these kinds of changes and pioneers their use (just as the environmental movement has had for many years).

Perhaps this paper of Hasan and Burns will turn out to be the call to arms that this incipient movement needs.

Ref: arxiv.org/abs/1106.6062: The Life and Death of Unwanted Bits: Towards Proactive Waste Data Management in Digital Ecosystems

Want to go ad free? No ad blockers needed.

Become an Insider
Already an Insider? Log in.
Want more award-winning journalism? Subscribe to Insider Basic.
  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning print magazine, unlimited online access plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Print Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.