July 2000

Lying With Pixels

Continued from page 1

By Ivan Amato

Last spring and summer, as Sportvision and rivals such as Princeton Video Imaging (PVI) in Lawrenceville, N.J,, were airing virtual insertion products, including simulated billboards on walls behind major league batters, a team of engineers from Sarnoff Corp. in Princeton, N.J., flew to the Coalition Allied Operations Center of NATO's Operation Allied Force in Vicenza, Italy. Their mission: transform their experimental video processing technology into an operational tool for rapidly locating and targeting Serbian military vehicles in Kosovo. The project was dubbed TIGER, for "targeting by image georegistration." "Just dial in the coordinates and the thing goes," explains Michael Hansen, a young, caffeinated Sarnoff gadgeteer who can hardly believe he was helping fight a war last year.

Compared to PVI's job, the military's technical task was more difficult-and the stakes were much higher. Instead of altering a football broadcast, the TIGER team manipulated a live video feed from a Predator, an unmanned reconnaissance craft flying some 450 meters above Kosovo battlefields. Rather than superimposing virtual lines or ads into sports settings, the task was to overlay, in real time, "georegistered" images of Kosovo onto the corresponding scenes streaming in live from the Predator's video camera. The terrain images had been previously captured with aerial photography and digitally stored. The TIGER system, which automatically detected moving objects against the background, could almost instantly feed to the targeting officers the coordinates for any piece of Serbian hardware in the Predator's view. This was quite a technical feat, since the Predator was moving and its angle of view was constantly changing, yet those views had to be electronically aligned and registered with the stored imagery in less than one-thirtieth of a second (to match the frame rate of video recording).

In principle, the targeting step could have been hotwired to precision guided weapons. "We weren't actually doing that in Allied Force," Hansen notes. "We were just telling targeting officers exactly where Serbian targets were and then they would vector in planes to go strike the targets." That way the human decision makers could pre-empt flawed machine-made decisions. According to the Defense Advanced Research Projects Agency, TIGER technology was used extensively in the final three weeks of the Kosovo operation, during which "80 to 90 percent of the mobile targets were hit."

So far, real-time video manipulation has been within the grasp only of technologically sophisticated organizations such as TV networks and the military. But developers of the technology say it's becoming simple and cheap enough to spread everywhere. And that has some observers wondering whether real-time video manipulation will erode public confidence in live television images, even when aired by news outlets. "Seeing may no longer be believing," says Norman Winarsky, corporate vice president for information technology at Sarnoff. "You may not know what to trust."