Skip to Content

The Hidden Damage From Waste Data (And How To Deal With It)

Waste data needlessly burns power and degrades computer memory. That’s why we need a coherent plan to reduce, reuse and recycle data that has been left to die, argue computer scientists

Back in 1999, a computer scientist at Cornell University began monitoring the way that the Windows NT 4.0 operating system used files. What he found was astonishing.

About 80 per cent of all files that NT creates are either over-written or deleted within 5 seconds of being born.

Today, Ragib Hasan and Randal Burns at Johns Hopkins University in Baltimore say this ought to give programmers pause for thought. Deleting data requires energy, which means that a substantial fraction of a computer system’s energy budget is currently devoted to creating and then almost immediately scrubbing data.

And if the wasted energy weren’t bad enough, computer memory has a limited life span. Flash memory, for example, has a lifespan of 100,000 cycles. So cycling it needlessly brings the inevitable breakdown closer

Surely, there’s a better way, say Hasan and Burns.

As it turns out, waste management is a maturing discipline, at least as far as physical waste is concerned. Why not use the same ideas in the data industry that are now used to manage physical waste elsewhere, they suggest.

In many ways, the ideas are easy to translate. Physical waste falls into four categories which translate easily into the virtual world:

  • Unintentional data. Data unintentionally created, as a side effect or by-product of a process, with no purpose
  • Used data. Good data that has served its purpose and is no longer useful to the user
  • Degraded data. Data that has degraded in quality such that it is no longer useful to the user
  • Unwanted data. Data that was never useful to the user

Why not apply the three Rs of conventional waste management to this virtual rot. In other words, use well known mantra of ‘reduce, reuse, recycle’ to better manage data

Hasan and Burns suggest that operating systems could provide incentives for applications to reduce the amount of waste files they generate, perhaps by reducing the I/O bandwidth or scheduling fewer CPU cycles to the worst offenders.

“This concept is equivalent to the Pay-as-You-Throw scheme and the polluter-pays principle used in real life waste management,” they say.

They also suggest setting up “digital landfills”, made of a semi-volatile storage medium that gradually degrade in time. “Unwanted data objects will fade automatically and the storage space can be reclaimed,” say the pair from Johns Hopkins.

It’s tempting to say that these ideas sound worthy but are otherwise impractical and unrealistic. But there are substantial benefits to be reaped from this approach, in particular for portable devices which are limited by battery life.

What’s needed, of course, is a hard core grass roots movement that campaigns for these kinds of changes and pioneers their use (just as the environmental movement has had for many years).

Perhaps this paper of Hasan and Burns will turn out to be the call to arms that this incipient movement needs.

Ref: The Life and Death of Unwanted Bits: Towards Proactive Waste Data Management in Digital Ecosystems

Keep Reading

Most Popular

DeepMind’s cofounder: Generative AI is just a phase. What’s next is interactive AI.

“This is a profound moment in the history of technology,” says Mustafa Suleyman.

What to know about this autumn’s covid vaccines

New variants will pose a challenge, but early signs suggest the shots will still boost antibody responses.

Human-plus-AI solutions mitigate security threats

With the right human oversight, emerging technologies like artificial intelligence can help keep business and customer data secure

Next slide, please: A brief history of the corporate presentation

From million-dollar slide shows to Steve Jobs’s introduction of the iPhone, a bit of show business never hurt plain old business.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.