Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Emerging Technology from the arXiv

A View from Emerging Technology from the arXiv

Human Brain Is Limiting Global Data Growth, Say Computer Scientists

Evidence has emerged that the brain’s capacity to absorb information is limiting the amount of data humanity can produce

  • December 1, 2011

In the early 19th century, the German physiologist Ernst Weber gave a blindfolded man a mass to hold and gradually increased its weight, asking the subject to indicate when he first became aware of the change. Weber discovered that the smallest increase in weight a human can perceive is proportional to the initial mass.

This is now known as the Weber-Fechner law and shows that the relationship between the stimulus and perception is logarithmic.

It’s straightforward to apply this rule to modern media. Take images for example. An increase in resolution of a low resolution pictue is more easily perceived than the same increase to a higher resolution picture.

When two parameters are involved, the relationship between the stimuli and perception is the square of the logarithm. An example would be video in which images change with time.

This way of thinking about stimulus and perception clearly indicates that the Weber-Fechner law ought to have a profound effect on the rate at which we absorb information.

Today, Claudius Gros and a couple of pals at Goethe University Frankfurt in Germany look for signs of the Weber-Fechner law in the size distribution of files on the internet. And they say they’ve found them.

These guys measured the type and size of files pointed to by every outward link from Wikipedia and the open directory project, dmoz.org. That’s a total of more than 600 million files. Some 58 per cent of these pointed to image files, 32 per cent to application files, 5 per cent to text files, 3 per cent to audio and 1 per cent to video files.

Gros and co then plotted the size of each of these files types against the number of files to get the file size distribution.

Sure enough, they discovered that the audio and video file distribution followed a log-normal curve, which is compatible with a logarithmic squared-type relationship. By contrast, image files follow a power law distribution, which is compatible with a logarithmic relationship. That’s exactly as the Weber-Fechner law predicts

“[This] strongly indicates that [the distributions] are determined by the underlying neurophysiological limitations of the producing agents,” say Gros and co. In other words ‘us’.

Further evidence comes from a closer look at the tails of these curves. If the size of files were determined by some kind of economic factor, like the cost of producing a file, then the distribution ought to have an exponential tail. But that’s not the case. The absence of this feature suggests some other origin for the file size distribution.

Gros and co put it like this: “The neuropsychological capacity of the human brain to process and record information may constitute the dominant limiting factor for the overall growth of globally stored information, with real-world economic constraints having only a negligible influence.”

Quite! In other words, global information cannot grow any faster than our ability to absorb or monitor it.

That makes sense and raises some interesting avenues for future research too. For example, it’ll be interesting to see how machine intelligence might change this equation. It may be that machines can be designed to distort our relationship with information.

If so, then a careful measure of file size distribution could reveal the first signs that intelligent machines are among us!

Ref: arxiv.org/abs/1111.6849: Neuropsychological Constraints To Human Data Production On A Global Scale

12 December 2011: updated to correct the spelling of the the Weber-Fechner law. Thanks Gavin Owens!

Couldn't make it to EmTech Next to meet experts in AI, Robotics and the Economy?

Go behind the scenes and check out our video
Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    Print + Digital Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

    Technology Review PDF magazine archive, including articles, images, and covers dating back to 1899

    10% Discount to MIT Technology Review events and MIT Press

    Ad-free website experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning print magazine, unlimited online access plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Print Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

  • Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.