Hello,

We noticed you're browsing in private or incognito mode.

To continue reading this article, please exit incognito mode or log in.

Not a subscriber? Subscribe now for unlimited access to online articles.

Intelligent Machines

Real-World Video Game

Surveillance was never so much fun.

The producers of the 1999 sci-fi martial-arts extravaganza The Matrix used elaborate and costly movie camera technology to circle around characters frozen mid-kung-fu kick. Such vertiginous visual effects can now be generated on the cheap from live images provided by stationary video cameras. But even more pertinent in the post-September 11 era, the package of video-processing and 3-D modeling technologies delivering these tricks promises to set a new standard for surveillance and security systems-and bring new meaning to the term “reality programming.”

The system, developed at Sarnoff in Princeton, NJ, allows users to joystick their way through a live, 3-D scene as if it were the latest video game. Indeed, by stitching together scenes captured by dozens or hundreds of networked cameras, the technology makes it possible to conduct a virtual patrol of an entire urban center, or every hallway in a building, in real time. “It’s difficult for eyeballs to make sense of what a collection of cameras sees. This is an integrated way to use hundreds of cameras to make one display,” says Rakesh Kumar, computer scientist and lead developer of the technology, dubbed “Video Flashlight.”

What makes the new system so unusual is that it melds 3-D models of a background scene-say, a cluster of  buildings-with real-time camera views of the same area. Image-processing chips developed at Sarnoff detect new or moving objects, construct 3-D images of these objects, and integrate them into the model. New software for “tweening”-filling in the gaps between video frames-lets security personnel “fly” around a subject such as a pedestrian, getting a detailed look without jumping between widely separated views. “This is beyond anything else out there” in vision processing, says Mari Maeda, a physicist who shepherded Sarnoff’s project as a program manager at the U.S. Defense Advanced Research Projects Agency.

This story is part of our July/August 2002 Issue
See the rest of the issue
Subscribe

Uses of the technology could include marking and tracking individuals as they move, or setting off an alarm when an unusual pattern is detected, such as a large group of people entering a building. Sarnoff engineers say the system needs refinement-for example, they aim to make it work with cameras that zoom, pan and tilt, not just fixed ones. But early versions have already been installed at U.S. Army Intelligence headquarters and are under consideration for New York City’s three airports-perhaps bringing us all a step closer to living inside the Matrix.

Keep up with the latest in intelligent machines at EmTech Digital.

The Countdown has begun.
March 25-26, 2019
San Francisco, CA

Register now
More from Intelligent Machines

Artificial intelligence and robots are transforming how we work and live.

Want more award-winning journalism? Subscribe to Print + All Access Digital.
  • Print + All Access Digital {! insider.prices.print_digital !}*

    {! insider.display.menuOptionsLabel !}

    The best of MIT Technology Review in print and online, plus unlimited access to our online archive, an ad-free web experience, discounts to MIT Technology Review events, and The Download delivered to your email in-box each weekday.

    See details+

    12-month subscription

    Unlimited access to all our daily online news and feature stories

    6 bi-monthly issues of print + digital magazine

    10% discount to MIT Technology Review events

    Access to entire PDF magazine archive dating back to 1899

    Ad-free website experience

    The Download: newsletter delivery each weekday to your inbox

    The MIT Technology Review App

/3
You've read of three free articles this month. for unlimited online access. You've read of three free articles this month. for unlimited online access. This is your last free article this month. for unlimited online access. You've read all your free articles this month. for unlimited online access. You've read of three free articles this month. for more, or for unlimited online access. for two more free articles, or for unlimited online access.