Technology Review - Published By MIT
Advertisement
[1] 2 Next »

Monday, March 17, 2008

Identifying Manipulated Images

New tools that analyze the lighting in images help spot tampering.

By Erica Naone

smaller text tool iconmedium text tool iconlarger text tool icon
True or false? The tool used above spots whether an image has been manipulated by modeling the lighting in the image based on an analysis of visible surfaces. To analyze an image, a user indicates the surfaces to consider using contour lines (shown above in white). The system checks for inconsistencies in the way that those surfaces are lit.
Credit: Micah Kimo Johnson, Hany Farid

Photo-editing software gets more sophisticated all the time, allowing users to alter pictures in ways both fun and fraudulent. Last month, for example, a photo of Tibetan antelope roaming alongside a high-speed train was revealed to be a fake, according to the Wall Street Journal, after having been published by China's state-run news agency. Researchers are working on a variety of digital forensics tools, including those that analyze the lighting in an image, in hopes of making it easier to catch such manipulations.

Tools that analyze lighting are particularly useful because "lighting is hard to fake" without leaving a trace, says Micah Kimo Johnson, a researcher in the brain- and cognitive-sciences department at MIT, whose work includes designing tools for digital forensics. As a result, even frauds that look good to the naked eye are likely to contain inconsistencies that can be picked up by software.

Many fraudulent images are created by combining parts of two or more photographs into a single image. When the parts are combined, the combination can sometimes be spotted by variations in the lighting conditions within the image. An observant person might notice such variations, Johnson says; however, "people are pretty insensitive to lighting." Software tools are useful, he says, because they can help quantify lighting irregularities--they can give solid information during evaluations of images submitted as evidence in court, for example--and because they can analyze more complicated lighting conditions than the human eye can. Johnson notes that in many indoor environments, there are dozens of light sources, including lightbulbs and windows. Each light source contributes to the complexity of the overall lighting in the image.

Johnson's tool, which requires an expert user, works by modeling the lighting in the image based on clues garnered from various surfaces within the image. (It works best for images that contain surfaces of a fairly uniform color.) The user indicates the surface he wants to consider, and the program returns a set of coefficients to a complex equation that represents the surrounding lighting environment as a whole. That set of numbers can then be compared with results from other surfaces in the image. If the results fall outside a certain variance, the user can flag the image as possibly manipulated.

Hany Farid, a professor of computer science at Dartmouth College, who collaborated with Johnson in designing the tool and is a leader in the field of digital forensics, says that "for tampering, there's no silver button." Different manipulations will be spotted by different tools, he points out. As a result, Farid says, there's a need for a variety of tools that can help experts detect manipulated images and can give a solid rationale for why those images have been flagged.

[1] 2 Next »

Comments

  • Apollo
    mdale9 on 03/17/2008 at 4:04 AM
    Posts:
    2
    Avg Rating:
    2/5
    I would be nice to use this tool to determine if any of the Apollo moon landing photos were faked.  Theoretically there should only be a single light source, shouldn't there?
    Rate this comment: 12345
    • Re: Apollo
      nealkrawetz on 03/17/2008 at 10:36 AM
      Posts:
      3
      Avg Rating:
      4/5
      Hi mdale9,

      There is one primary light source (the Sun), but there should be lots of ambient light of varying degrees (stars and reflection of any earthlight).

      As far as analysis goes, there is usually no difference between a "real" and "staged" photo -- both are "real" in that they were not digitally manipulated. However, this does not mean that the content is real. For example, Imad Mughniyeh staged the famous photo of him holding a gun to a pilot's head (http://edition.cnn.com/2008/WORLD/meast/02/13/btsc.clancy.mugniyah/index.html). The interpretation of the photo's content is deceptive but the image itself is real.

      When you ask about the moon landing, the photos that I have looked at appear to be real. However, I will not to get into a debate on whether they were staged or actually taken on the moon -- too many people ignore facts in place of agendas and conspiracies.

      - Dr. Neal Krawetz
      Rate this comment: 12345
    • Re: Apollo
      smithsomian on 03/19/2008 at 11:42 AM
      Posts:
      18
      Avg Rating:
      3/5
      The surface of the Moon is lightly-colored and, with no atmosphere, subject to brilliant light from the sun - how can you say there's only one light source?
      Rate this comment: 12345
  • Identifying Manipulated Images
    deejay on 03/17/2008 at 5:15 AM
    Posts:
    22
    Avg Rating:
    4/5
    I'm curious - is this only for digital photographs, or will it work with a film-based photograph that has been scanned and digitised?
    Rate this comment: 12345
    • Re: Identifying Manipulated Images
      Erica Naone on 03/17/2008 at 9:03 AM
      Technology Review TR Staff
      Assistant Editor
      Posts:
      26
      Avg Rating:
      4/5
      My understanding is it would work with scanned and digitized photos as well as photos that were originally digital.

      -- Erica Naone
      Rate this comment: 12345
    • Re: Identifying Manipulated Images
      nealkrawetz on 03/17/2008 at 10:17 AM
      Posts:
      3
      Avg Rating:
      4/5
      Hi deejay,

      It depends on the type of manipulation. Light direction may still be detectable, as will any poorly pasted borders. However, my tools also look for compression artifacts and format anomalies. These will not be present if a photo is manipulated before being digitized.

      - Dr. Neal Krawetz
      Rate this comment: 12345
  • New moon & mars pictures
    dsouza on 03/17/2008 at 12:01 PM
    Posts:
    1
    Avg Rating:
    2/5
    Is this tool available somewhere, or is it source closed ? I think it would be nice to test the images on the new moon images (specially those from the chinese probes) and the mars pictures (which are said to be [color-]tampered, although maybe it wouldn't work for color tampering detection).
    Rate this comment: 12345
    • Re: New moon & mars pictures
      Erica Naone on 03/18/2008 at 10:04 AM
      Technology Review TR Staff
      Assistant Editor
      Posts:
      26
      Avg Rating:
      4/5
      At this point, neither of the tools mentioned in the article are publicly available, as far as I'm aware. I should also note that, while other digital forensics tools might detect color changes, tools that pay attention to lighting (which I've focused on here) seem best suited to flagging cases where multiple photos have been combined into one.

      -- Erica Naone
      Rate this comment: 12345
  • A picture can be a thousand lies.
    gabrielg01 on 03/18/2008 at 12:17 PM
    Posts:
    305
    Avg Rating:
    3/5
    Even if such software tools are available, one could still produce fake images.

    As a forger, you would take the fake image and run tests on it with the forensics software, and see which part of the image is flagged as fake. Then you modify the image accordingly, and test again. A few iterative steps like these, and you would be able to produce an image that passes the test.

    Sure this means more work on the part of the forgers, but then they can claim to have a "genuine" picture.

    We should never rely solely on pictures as proof anymore. A picture can tell a thousand lies.
    Rate this comment: 12345
    • Re: A picture can be a thousand lies.
      Erica Naone on 03/19/2008 at 9:39 AM
      Technology Review TR Staff
      Assistant Editor
      Posts:
      26
      Avg Rating:
      4/5
      You're absolutely right. However, when I asked Johnson about this, he pointed out that, for now, forgers tend not to have access to these tools. It'll be a different game once they're packaged with Photoshop.

      One additional point is that, like all questions of security, if someone is intent on pulling off a beautiful and sophisticated forgery, and has the skills to back that up, they probably can. However, many forgeries simply aren't done with that level of dedication and skill, and tools can help filter out those forgeries.
      Rate this comment: 12345
      • Re: A picture can be a thousand lies.
        nealkrawetz on 03/19/2008 at 12:30 PM
        Posts:
        3
        Avg Rating:
        4/5
        One of the things I mentioned at my Black Hat Briefings talk was specifically about the necessary skills. The tools are widely available -- Photoshop, Gimp, 3DS Max, Maya, Bryce, etc. either come with your system (or camera or scanner) or can be purchased as off the shelf software.

        Having the tools is not difficult. However, having the skills to use the tool is a different story. (Just because I own a hammer does not mean I can build a house.) Most people who modify images lack the skills to overcome most image analysis techniques. In fact, many forgeries by professional artists still test positive for being modified.

        My own tools are around 86% accurate. The few misses were by professional artists who have won awards from the CG Society. And two of the misses were by the same artist who spent over two years on the same model (wireframe, skin texture, hair, etc.). Fortunately for detecting forgeries, most amateurs don't have the skill or time to devote to their edits. (Even I, knowing exactly how the tools work, cannot make a good forgery because I lack the skills to use the drawing tools at a professional level.)

        Given enough time, an amateur artist should be able to defeat a few of the detection methods. However, I use a suite of analysis tools and methods -- light analysis is just one approach. Very few doctored images can make it past all of the tests, and those that do were likely by professional artists who have plenty of time to devote to the forgery.

        You are correct that a professional forger could use these forensic techniques to better the forgery. Fortunately, most doctored images are not by professionals, and even professionals may not have the time to overcome all of the signs of modification.

        There is a saying in the computer security field: the defender has an impossible task; they must defend against all possible attacks, while the attacker only has to find one vulnerability. The same can be said about photo forensics: the forger must hide all possible signs of modification or forgery, while the forensic examiner only needs to find one clue.
        Rate this comment: 12345
  • Identifying Fakes
    jmaximus9 on 04/04/2008 at 9:22 PM
    Posts:
    34
    Avg Rating:
    3/5
    Now if we could figure out a way to tell when Hillary is lying; oh yeah thats right we did, when her lips are moving.
    Rate this comment: 12345
Advertisement

Current Issue

Technology Review September/October 2008
How Obama Really Did It
Social technology helped bring him to the brink of the presidency.
•  Subscribe
Save 41%
•  Table of Contents
•  MIT News

Magazine Services

Career Resources

MIT Technology Insider

Stories and breaking news from inside MIT about the latest research, innovations, and startups--in a convenient monthly e-newsletter. Subscribe today

Follow us on Twitter

Twitter

Get Technology Review updates via the web, cellphone, or Instant Messager – Follow techreview on Twitter!

Advertisement

More Technology News from Forbes

Advertisement
TECHNOLOGY RESOURCES
Advertisement
MIT Massachusetts Institute of Technology