July 2000

Lying With Pixels

Seeing is no longer believing. The image you see on the evening news could well be a fake - a fabrication of fast new video-manipulation technology.

By Ivan Amato

Last year, Steven Livingston, professor of political communication at George Washington University, astonished attendees at a conference on the geopolitical pros and cons of satellite imagery. He didn't produce evidence of new military mobilizations or global pandemics. Instead, he showed a video of figure skater Katarina Witt during a 1998 skating competition.

In the clip, Witt gracefully plies the ice for about 20 seconds. Then came what is perhaps one of the most unusual sports replays ever seen. The background was the same, the camera movements were the same. In fact, the image was identical to the original in all ways except for a rather important one: Witt had disappeared, along with all signs of her, such as shadows or plumes of ice flying from her skates. In their place was exactly what you would expect if Witt had never been there to begin with-the ice, the walls of the rink and the crowd.

So what's the big deal, you ask. After all, Stalin's staff routinely airbrushed persona non grata out of photos more than a half-century ago. And Woody Allen ushered a variation on reality morphing into the movies 17 years ago with Zelig, in which he inserted himself next to Adolf Hitler and Babe Ruth. In films such as Forrest Gump and Wag the Dog, reality twisting has become commonplace.

What sets the Witt demo apart-way apart-is that the technology used to "virtually delete" the skater can now be applied in real time, live, even as a camera records a scene and instantly broadcasts it to viewers. In the fraction of a second between video frames, any person or object moving in the foreground can be edited out, and objects that aren't there can be edited in and made to look real. "Pixel plasticity," Livingston calls it. The implication for those at the satellite imagery conference was sobering: Pictures from orbit may not necessarily be what the satellite's electronic camera actually recorded.

But the ramifications of this new technology reach beyond satellite imagery. As live electronic manipulation becomes practical, the credibility of all video will become just as suspect as Soviet Cold War photos. The problem stems from the nature of modern video. Live or not, it is made of pixels, and as Livingston says, pixels can be changed.

The best-known examples of real-time video manipulation so far are "virtual insertions" in professional sports broadcasts. Last January 30, for instance, nearly one-sixth of humankind in more than 180 countries repeatedly saw an orange first-down line stretched across the gridiron as they watched the Super Bowl. New York-based Sportvision created that line and inserted it into the live feed of the broadcast. To help determine where to insert the orange pixels, several game cameras were fitted with sensors that tracked the cameras spatial positions and zoom levels. Adding to the illusion of reality was the ability of the Sportvision system to make sure that players and referees occlude the virtual line when their bodies traverse it.