Skip to Content

The Hidden Damage From Waste Data (And How To Deal With It)

Waste data needlessly burns power and degrades computer memory. That’s why we need a coherent plan to reduce, reuse and recycle data that has been left to die, argue computer scientists

Back in 1999, a computer scientist at Cornell University began monitoring the way that the Windows NT 4.0 operating system used files. What he found was astonishing.

About 80 per cent of all files that NT creates are either over-written or deleted within 5 seconds of being born.

Today, Ragib Hasan and Randal Burns at Johns Hopkins University in Baltimore say this ought to give programmers pause for thought. Deleting data requires energy, which means that a substantial fraction of a computer system’s energy budget is currently devoted to creating and then almost immediately scrubbing data.

And if the wasted energy weren’t bad enough, computer memory has a limited life span. Flash memory, for example, has a lifespan of 100,000 cycles. So cycling it needlessly brings the inevitable breakdown closer

Surely, there’s a better way, say Hasan and Burns.

As it turns out, waste management is a maturing discipline, at least as far as physical waste is concerned. Why not use the same ideas in the data industry that are now used to manage physical waste elsewhere, they suggest.

In many ways, the ideas are easy to translate. Physical waste falls into four categories which translate easily into the virtual world:

  • Unintentional data. Data unintentionally created, as a side effect or by-product of a process, with no purpose
  • Used data. Good data that has served its purpose and is no longer useful to the user
  • Degraded data. Data that has degraded in quality such that it is no longer useful to the user
  • Unwanted data. Data that was never useful to the user

Why not apply the three Rs of conventional waste management to this virtual rot. In other words, use well known mantra of ‘reduce, reuse, recycle’ to better manage data

Hasan and Burns suggest that operating systems could provide incentives for applications to reduce the amount of waste files they generate, perhaps by reducing the I/O bandwidth or scheduling fewer CPU cycles to the worst offenders.

“This concept is equivalent to the Pay-as-You-Throw scheme and the polluter-pays principle used in real life waste management,” they say.

They also suggest setting up “digital landfills”, made of a semi-volatile storage medium that gradually degrade in time. “Unwanted data objects will fade automatically and the storage space can be reclaimed,” say the pair from Johns Hopkins.

It’s tempting to say that these ideas sound worthy but are otherwise impractical and unrealistic. But there are substantial benefits to be reaped from this approach, in particular for portable devices which are limited by battery life.

What’s needed, of course, is a hard core grass roots movement that campaigns for these kinds of changes and pioneers their use (just as the environmental movement has had for many years).

Perhaps this paper of Hasan and Burns will turn out to be the call to arms that this incipient movement needs.

Ref: The Life and Death of Unwanted Bits: Towards Proactive Waste Data Management in Digital Ecosystems

Keep Reading

Most Popular

The inside story of how ChatGPT was built from the people who made it

Exclusive conversations that take us behind the scenes of a cultural phenomenon.

How Rust went from a side project to the world’s most-loved programming language

For decades, coders wrote critical systems in C and C++. Now they turn to Rust.

Design thinking was supposed to fix the world. Where did it go wrong?

An approach that promised to democratize design may have done the opposite.

Sam Altman invested $180 million into a company trying to delay death

Can anti-aging breakthroughs add 10 healthy years to the human life span? The CEO of OpenAI is paying to find out.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.