Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not an Insider? Subscribe now for unlimited access to online articles.

Emerging Technology from the arXiv

A View from Emerging Technology from the arXiv

The Algorithm That Unscrambles Fractured Images

The ongoing revolution in image processing has produced yet another way to extract images from a complex environment.

  • January 9, 2015

Take a hammer to a mirror and you will fracture the image it produces as well as the glass. Keep smashing and the image becomes more broken. When the pieces of glass are the size of glitter, the reflections will be random and the image unrecognizable.

It’s easy to imagine that reconstructing this image would be close to impossible. Not so, say Zhengdong Zhang and pals at the Massachusetts Institute of Technology in Cambridge. Today, these guys unveil SparkleVision, an image processing algorithm that reassembles the smashed imaged.

The problem that Zhang and co attack is to work out the contents of a picture reflected off a screen covered in glitter. The approach is to photograph the glitter and then process the resulting image in a way that unscrambles the picture.

It turns out that there is a straightforward way to approach this. Zhang and co consider each piece of glitter to be a randomly oriented micromirror. So light from the picture hits a micromirror and is reflected to a sensor inside the camera.

That means there is a simple mapping from each pixel in the original picture to a sensor in the camera. The task is to determine that mapping for every pixel. “There exists a forward scrambling matrix, and in principle we can find its inverse and unscramble the image,” they say.

To find this unscrambling matrix, Zhang and co shine a set of test images at the glitter screen and record where the pixels in the original image end up in the camera.

From this, they can create an algorithm that unscrambles any other image placed in exactly the same spot as the test images. They call this algorithm SparkleVision.

That’s a handy piece of software that could have interesting applications in retrieving images reflected off glitter-like surfaces such as certain types of foliage, wet surfaces, metals and so on.

And Zhang and co hope to make the software more useful. In its current incarnation, the software can only unscramble images placed in the exact location of the test images. But in theory, the test images should provide enough data to unscramble images from any part of the light field. “Thus, our system could be naturally extended to work as a lightfield camera,” they say.

The work is part of a growing body that is currently revolutionising photography and image processing, Other researchers have worked out how to unscramble images from all kinds of distorted reflections and surfaces, sometimes even without using lenses. 

These so-called “random cameras” are dramatically widening the capability of optics specialists. And SparkleVision looks set to take its place among them.

Ref: http://arxiv.org/abs/1412.7884 : SparkleVision: Seeing the World through Random Specular Microfacets

Be the leader your company needs. Implement ethical AI.
Join us at EmTech Digital 2019.

Register now
Want more award-winning journalism? Subscribe and become an Insider.
  • Insider Plus {! insider.prices.plus !}* Best Value

    {! insider.display.menuOptionsLabel !}

    Everything included in Insider Basic, plus the digital magazine, extensive archive, ad-free web experience, and discounts to partner offerings and MIT Technology Review events.

    See details+

    Print + Digital Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

    Technology Review PDF magazine archive, including articles, images, and covers dating back to 1899

    10% Discount to MIT Technology Review events and MIT Press

    Ad-free website experience

  • Insider Basic {! insider.prices.basic !}*

    {! insider.display.menuOptionsLabel !}

    Six issues of our award winning print magazine, unlimited online access plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Print Magazine (6 bi-monthly issues)

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

  • Insider Online Only {! insider.prices.online !}*

    {! insider.display.menuOptionsLabel !}

    Unlimited online access including articles and video, plus The Download with the top tech stories delivered daily to your inbox.

    See details+

    Unlimited online access including all articles, multimedia, and more

    The Download newsletter with top tech stories delivered daily to your inbox

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.