How to Handle the World’s Largest Digital Images
Much has been made of the ‘unprecedented’ scale of the IT infrastructure required to store all the data coming from the world’s largest physics experiment - the Large Hadron Collider (LHC) near Geneva, Switzerland. But another gigantic science experiment, one you’ve probably never heard of, will some day pump out data on a similar scale. And, unlike the LHC, the data it produces will be comprehensible by non-scientists, and made freely available.

Once it’s complete, The Large Synoptic Survey Telescope on top of Cerro Pachón ridge in Chile will sport the world’s largest digital camera. It will peer deep into space yet have a field of view unprecedented in modern telescopes – ten square degrees of the heavens encompassed by a collecting area of forty square meters. The telescope will take 800 panoramic pictures a night, covering the entire night sky twice each week.

The result will be an unfathomably huge photo collection: 20 terabytes of data stored every 24 hours. Running all-out (and telescopes this expensive are usually booked year round) that’s 7.3 petabytes of data a year – half as much data as the 15 petabytes the LHC is producing each year. (To put that in perspective, over its lifetime LHC will produce as much data as all the words spoken by humankind since its appearance on earth.)
In order to handle that much data, once a day all the raw images from the telescope will be transferred to the Archive Center at the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign, where 100 teraflops of processing power will digest and archive it on what will initially be 15 petabytes of storage (to be expanded as the experiment continues).
The results will be freely available to the public via existing open standards, and could help with everything from tracking killer asteroids to unraveling the mysteries of dark energy and dark matter. According to the LSST’s homepage, “Anyone with a computer will be able to fly through the Universe, zooming past objects a hundred million times fainter than can be observed with the unaided eye.”
Keep Reading
Most Popular
What to know about this autumn’s covid vaccines
New variants will pose a challenge, but early signs suggest the shots will still boost antibody responses.
DeepMind’s cofounder: Generative AI is just a phase. What’s next is interactive AI.
“This is a profound moment in the history of technology,” says Mustafa Suleyman.
Human-plus-AI solutions mitigate security threats
With the right human oversight, emerging technologies like artificial intelligence can help keep business and customer data secure
Next slide, please: A brief history of the corporate presentation
From million-dollar slide shows to Steve Jobs’s introduction of the iPhone, a bit of show business never hurt plain old business.
Stay connected
Get the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.