Select your localized edition:

Close ×

More Ways to Connect

Discover one of our 28 local entrepreneurial communities »

Be the first to know as we launch in new countries and markets around the globe.

Interested in bringing MIT Technology Review to your local market?

MIT Technology ReviewMIT Technology Review - logo


Unsupported browser: Your browser does not meet modern web standards. See how it scores »

{ action.text }

The Fifth Utility

In many ways, the drama of pervasive surveillance is being played out first in Orwell’s native land, the United Kingdom, which operates more closed-circuit cameras per capita than any other country in the world. This very public surveillance began in 1986 on an industrial estate near the town of King’s Lynn, approximately 100 kilometers north of London. Prior to the installation of three video cameras, a total of 58 crimes had been reported on the estate. None was reported over the next two years. In 1995, buoyed by that success, the government made matching grants available to other cities and towns that wanted to install public surveillance cameras-and things took off from there.

Most of these closed-circuit TV systems are installed in business districts or shopping centers by British Telecommunications, the national phone network, and jointly operated and managed by law enforcement and private industry. In addition, some townships are using BT to hook up video telephony, a technology that allows transmission of video images via telephone lines-but in a monitor-friendly network that provides officials quick and easy remote access to the images. On another front, the U.K. Home Office, the government department responsible for internal affairs in England and Wales, is starting construction of what promises to be the world’s biggest road and vehicle surveillance network, a comprehensive system of cameras, vehicle and driver databases, and microwave and phone-based communications links that will be able to identify and track the movements of vehicles nearly nationwide. All told, the country’s electronic eyes are becoming so prevalent that Stephen Graham of the Centre for Urban Technology at the University of Newcastle upon Tyne has dubbed them a “fifth utility,” joining water, gas, electric and telephones.

The United States and many other parts of the developed world are not far behind in video surveillance. Just look at the cameras looking at you. They’re in ATMs, banks, stores, casinos, lobbies, hallways, desktops, and along highways, main streets and even side streets. And those are the cameras you can see. Companies like All Security Systems of Miami, FL, advertise Clock Cameras, Exit Sign Cameras, Smoke Detector Cameras, and Covert Tie and Button Cams, as well as Nanny Cams and other easily hidden eyes, some of which send video signals wirelessly to a recorder located elsewhere.

But cameras seem relatively benign when compared to new technology being developed and deployed. Until recently, closed-circuit systems have fed video signals to monitors, which human beings had to watch in real time, or sent the images to recording media for storage. Now, however, the job of spotting suspicious people and behavior in this stream of electronic imagery is becoming automatic, with computers programmed with special algorithms for matching video pixel patterns to stored patterns associated with criminals or criminal actions-and the machines themselves passing initial judgment on whether a behavior is normal.

For example, last January at the Super Bowl in Tampa, FL, law enforcement agencies, without announcement, deployed a face recognition system from Viisage Technology of Littleton, MA. Cameras snapped face shots of fans entering the stadium. Computers instantly extracted a minimal set of features from each captured face, a so-called eigenface, and then compared the eigenfaces to those of criminals, stored in a database. The system purportedly found 19 possible matches, although no one was arrested as a result of the test. Less than six months later, in mid-July, Tampa police sparked public protests after deploying a face recognition system from Visionics, of Jersey City, NJ, to scan city sidewalks for suspected criminals and runaways.

And this is just the beginning of the technology being piloted and prototyped to watch you-and judge your behavior. Beginning in 1997, the U.S. Defense Advanced Research Projects Agency (DARPA) funded some 20 projects under a three-year program called Video Surveillance and Monitoring. That effort has just gathered new momentum under a $50 million follow-up program known as Human ID at a Distance. The aim is to determine if it’s feasible to identify specific individuals at distances up to 150 meters.

Under the program, researchers at Carnegie Mellon University in Pittsburgh are investigating whether a remote sensing technique known as “hyperspectral imaging”-a technology typically used by satellites to find minerals or peer through military camouflage-can be adapted for identifying specific human beings by measuring the color spectrum emitted by their skin. Skin absorbs, reflects and emits distinct patterns of color, and those patterns are specific enough to individual people to serve as spectral signatures. Such systems already work. But according to Robert Collins, a computer scientist at Carnegie Mellon’s Robotics Institute, the process currently requires a person to sit stiffly in a chair as a sensor sweeps through hundreds of emitted wavelengths over a period of about five seconds. “Ideally, what will happen is we’ll find some small group of wavelengths that we can use to distinguish people,” explains Collins. That could reduce the scan time to a fraction of a second.

Another approach being developed involves a video-based network of sensors that would automatically measure such characteristics as leg length and waist width to provide, as Collins says, “the measurements you give to a tailor.” The idea here, he says, is that those numbers should be able to serve as a kind of body fingerprint for identifying specific individuals.

There is no shortage of cleverness when it comes to building the surveillance state. At the Georgia Institute of Technology, scientists are developing sensor-riddled “smart floors” that can identify people by the “force profiles” of their walking feet. Meanwhile, Princeton, NJ-based Sarnoff is working toward an antiterrorist technique that uses a special camera to identify individuals from a hundred meters off by the patterns of color, striation and speckles in their irises. This isn’t easy, since the iris and its elements move so quickly relative to a distant camera that the technical task bears some resemblance to “tracking a ballistic missile,” says Norman Winarsky, president of nVention, Sarnoff’s venture technology company. Still, the technology is coming.

Beyond identity is intention-and there are technologies in the works for divining that as well. IBM has introduced a software product called BlueEyes (see “Behind BlueEyes,” TR May 2001) that’s currently in use at retail stores to record customers’ facial expressions and eye movements, tracking the effectiveness of in-store promotions. And psychologist Jeffrey Cohn of Carnegie Mellon’s Robotics Institute and colleagues have been trying to teach machines an even more precise way to detect facial expressions.

From video signals, the Carnegie Mellon system detects and tracks both invariant aspects of a face, such as the distance between the eyes, and transient ones, like skin furrows and smile wrinkles. This raw data is then reclassified as representing elemental actions of the face. Finally, a neural network correlates combinations of these measurable units to actual expressions. While this falls short of robotic detection of human intentions, many facial expressions reflect human emotions, such as fear, happiness or rage, which, in turn, often serve as visible signs of intentions.

Cohn points out that this particular work is just part of the team’s more encompassing “goal of developing computer systems that can detect human activity, recognize the people involved, understand their behavior, and respond appropriately.” In short, the effort could help lead to the kind of ubiquitous surveillance system that can automatically scan collective human activity for signs of anything from heart-attack-inducing Type-A behavior to sexual harassment to daydreaming at the wheel to homicidal rage.

0 comments about this story. Start the discussion »

Tagged: Communications

Reprints and Permissions | Send feedback to the editor

From the Archives


Introducing MIT Technology Review Insider.

Already a Magazine subscriber?

You're automatically an Insider. It's easy to activate or upgrade your account.

Activate Your Account

Become an Insider

It's the new way to subscribe. Get even more of the tech news, research, and discoveries you crave.

Sign Up

Learn More

Find out why MIT Technology Review Insider is for you and explore your options.

Show Me