Skip to Content

Will Spyware Work?

Monitoring voice and e-mail traffic sounds like a good way to thwart terrorism. The problem? Sorting through the results takes too long for early warning.
December 1, 2001

As the United States tries to grapple with the new realities of war and terrorism, questions for its intelligence community keep coming: How could something like September 11 occur without plans being detected? Who was tracking the activities of suspected terrorists inside the country? How were they even here in the first place? What happened to those high-tech, Big Brother-type surveillance tools like the notorious global-communications eavesdropping network Echelon, or Carnivore, the FBI’s Internet snoopware, that were supposed to sniff out criminal activity?

For several decades, electronic systems have been quietly put in place to intercept satellite communications, tap phone calls, monitor e-mail and Web traffic and then turn this massive flow of information into intelligence reports for U.S. leaders and investigative aids for law enforcement. Yet despite the $30 billion invested in them, and all the secrecy afforded them, government information technologies still could not connect the proverbial dots of the World Trade Center plot. “Obviously, there were intelligence failures on a number of levels,” says Barry Posen, a defense policy analyst with MIT’s Center for International Studies.

Now that it is apparent that these supposedly all-seeing government systems are not all-knowing, how can we ascertain that they work at all? While the technologies to intercept and capture any and every communication conjure images of an Orwellian omniscience (see “Big Brother Logs On,” TR September 2001), many experts say the ability to derive useful knowledge from all that data is still far from plausible. Even as the processing times get faster and the software  gets smarter, the process of turning raw data into assured intelligence is far from perfect. If the goal is capturing, listening to and then actually sussing every single electronic communication in the United States, “In practical terms, we’re not even close,” says Gary McGraw, CTO at Cigital, a Dulles, VA-based network security software vendor.

It doesn’t seem to be for lack of trying, however. Today, the U.S. intelligence community comprises more than a dozen major agencies, including the CIA, FBI and the National Security Agency. Within these bodies, there are dozens more departments, such as the CIA’s directorate of science and technology, that specifically develop information technologies to aid in the practice of knowing what other people don’t want them to know.

While the agencies theoretically cooperate, especially since September 11, there is no centralized information system to compare and contrast data collected  among them. Critics claim that this bureaucratic and technical fragmentation is one reason terrorists were able to hatch their plan under the government’s radar.

It is far from the only one. Even if intelligence agencies seamlessly integrate their knowledge, the tools available to them now and for the foreseeable future do not appear up to the task of providing the early warning needed to thwart terrorist plots. “My first reaction is not necessarily a question of why didn’t these tools work, but how hard it would have been to discover this in the first place,” says Sayan Chakraborty, vice president of engineering at Sigaba, a San Mateo, CA-based company specializing in e-mail encryption.

Hearing Without Listening

Despite its most recent, catastrophic lapses, the United States has a long and distinguished history of successfully using advanced information-gathering and analysis tools against its enemies. The Signals Intelligence Section, the forerunner of today’s National Security Agency, came into being in World War II, when the United States broke the Japanese military code known as Purple and discovered plans to invade Midway Island. The NSA’s early forays in cryptography contributed to the development of the first supercomputers and other information technologies. In his book The Wizards of Langley: Inside the CIA’s Directorate of Science and Technology, National Security Archive senior fellow Jeffrey T. Richelson published more than 40 declassified documents that trace the CIA’s exploitation of science and technology for the purposes of intelligence gathering. “From the early 1950s to the present, technology has played an essential part in analysis,” he says.

The granddaddy of today’s governmental electronic surveillance is Echelon, the National Security Agency’s infamous, yet officially unacknowledged, global surveillance network. Said to be the most comprehensive and sophisticated signals intelligence setup in existence, Echelon reportedly has the capability to monitor every communication transmitted by satellite outside of U.S. borders-by some counts, three billion telephone calls, e-mail messages, faxes and broadcasts daily. Technically, Echelon technology could monitor domestic communications too, though that is prohibited under U.S. law.

According to a European Parliament report released in September, Echelon collects information through a complex web of radio antennae at listening stations across the planet. Other sources claim that one listening station in particular, at Menwith Hill in England, operated by U.S. and British intelligence services, is placed in the most convenient spot to tap transatlantic communications cables as well. Investigations cited by the American Civil Liberties Union and others report that Echelon rakes these immense volumes of data through “dictionary” software that operates on a vast computer network hosted by intelligence agencies from five countries-the United States, Britain, Australia, Canada and New Zealand. The dictionary program flags messages containing any of a set of predetermined keywords, such as “bomb” or “President Bush.” The words are rumored to be changed on a regular basis.

How the actual process of data sifting works remains a mystery. National security restrictions prohibit anyone from speaking publicly about the program. Quips one source who has followed the technology, “Anyone who knows about it won’t talk about it, and anyone who talks about it doesn’t really know about it.” Some experts suspect, however, that Echelon’s data processing is based on a variety of technologies in use in the commercial world today, including speech recognition and word pattern finding. “Word pattern recognition is nothing new,” says Winn Schwartau, a security consultant in Seminole, FL, and the author of Information Warfare and Cybershock. “We’ve been using that sort of stuff for years. But if you look at how advanced the searching abilities for the average person have become, I can only imagine the type of stuff that government security agencies have in operation.”

According to Schwartau and others, the ability to sort through billions of messages and divine anything useful encompasses a number of techniques. Speech recognition systems and optical character readers convert spoken words (from phone conversations) and printed text (as from intercepted faxes) into catalogued and searchable digital data. Language translation software turns many of the world’s spoken tongues into the English that the U.S. intelligence community prefers. Data-mining software searches volumes of data and establishes relationships among them by finding similarities and patterns.

Echelon has supposedly been using techniques like these to churn data into knowledge about foreign governments, corporations and even specific individuals since the 1970s. Subjects of surveillance are reported to have even included the likes of Princess Diana, whose work eliminating land mines ran counter to U.S. policy. And in the months leading up to September 11, 2001, according to reports from the German newspaper Frankfurter Allgemeine Zeitung, snippets produced by Echelon intimated that “a big operation” was in place by terrorists seeking to destroy “American targets.” Other information collected may in hindsight be pieced together to divine a much clearer picture of the operation. Unfortunately, things did not come together in time to warn of the attacks.

Watch What You Type

Another government snooping technology that has been the subject of controversy since long before September 11 is Carnivore. Comprising a set of programs in development by the FBI since 1996, Carnivore is devised to intercept data traffic sent over the Internet to assist federal authorities in criminal investigation. According to the FBI, Carnivore is installed only with the cooperation of an Internet service provider and after obtaining appropriate judicial approval to track e-mail, instant messages and Web search trails. And the system inspects only those communications that are legally authorized for interception.

That, at least, is the theory. Civil liberties organizations such as the ACLU, the Electronic Frontier Foundation and the Electronic Privacy Information Center worry Carnivore could be used to monitor much more than that.

To counter that suspicion, the U.S. Department of Justice hired Chicago-based IIT Research Institute to perform the only testing of Carnivore permitted outside government agencies. According to IIT’s report, published last December, Carnivore works much like the commercial network diagnostic programs-called “sniffers”-that are used to monitor corporate networks, and runs on nothing more than an average personal computer.

After securing the proper warrants, the FBI will approach an Internet service provider to attach a Carnivore-loaded PC to its internal cabling. When plugged into a hub, the collection computer sees all data packets going by. It then copies only those packets that match settings prescribed by the FBI and approved by court order. Agents can view the captured packets in two different modes. In so-called pen mode, the system displays only information that identifies the sender and the intended recipient-numerical Internet addresses and e-mail names-and subject lines. In “full mode,” the agent can access not just this address information but also the entire contents of the message.

Once Carnivore has been installed at the Internet service provider, it is controlled remotely, according to the IIT report. The collection computer is connected to an analog voice line installed specifically for the particular tap. The intercepted data are stored on a two-gigabyte disk, which is then taken back to FBI laboratories for analysis. The data packets-broken bits of e-mail messages, Web pages and any other form of data sent across the Internet-can then be rebuilt and reviewed.

While Echelon and Carnivore are the most infamous intelligence collection tools, they are not the only ones, however. Government skunk works are constantly cooking up new tools to assist in covert surveillance operations. These include other quasi-legendary projects like Tempest, the code word for a number of surveillance technologies that can capture data displayed on computer screens by picking up electromagnetic emissions from the internal electron beams that create the images.

Every once in a while, the intelligence community opens its cloak to show off some of its tricks. Last March, for example, Larry Fairchild, director of the CIA’s office of advanced information technology, brought a group of reporters into the basement of the agency’s headquarters in Langley, VA. There, he demonstrated two programs deemed safe for public consumption: Fluent and Oasis.

Fluent performs computer searches of documents written in different languages. An analyst types in a query in English, just as if he or she were using a garden-variety search engine like Google. The software fishes out relevant documents in a number of foreign languages-including Russian, Chinese, Portuguese, Serbo-Croatian, Korean and Ukrainian-and then translates them into English.

Oasis converts audio signals from television and radio broadcasts, such as those from Qatar-based al-Jazeera, into text. It distinguishes accents, whether the speaker is male or female, and whether one voice is different from another of the same gender. The software then generates a transcript of those transmissions, identifying which voice uttered which statements. While Oasis can today comprehend only English-language programs, the CIA is developing versions that work in Chinese and Arabic, among other languages. Oasis can reportedly process and analyze a half-hour broadcast in as little as 10 minutes, as opposed to the 90 minutes that the task typically takes for an analyst working without the software.

Future Futility

Assuming all this impressive high-tech wizardry is fully operational, how could a band of terrorists, including many already suspected as such, operate within U.S. borders for years and still escape detection-undoubtedly making phone calls and exchanging e-mail with coconspirators all the while? The answers, unfortunately, don’t provide a basis for  optimism about the ability of these systems to offer much protection in the new war against terrorism.

First, security and intelligence experts agree that the mass of information generated every day around the world far outstrips the capacity of present-day technologies to process it. “You’re talking about incredible mountains of information, and trying to find that needle,” says McGraw.

Intelligence agency leaders themselves have admitted their vulnerabilities. “We’re behind the curve in keeping up with the global telecommunications revolution,” National Security Agency director Michael Hayden told CBS’s 60 Minutes in a rare public admission last February. In testimony to Congress days after the attacks on the World Trade Center and Pentagon, Attorney General John Ashcroft warned that terrorists still have the “competitive advantage” when it comes to domestic espionage, and that “we are sending our troops into the modern field of battle with antique weapons.”

Then there is the matter of encryption technologies that can turn even intercepted communications into gobbledygook. “The odds are nigh on impossible that the NSA or anybody else is going to be able to break” an encrypted message, says security expert and author Schwartau. Another technology that Osama bin Laden’s minions reportedly used falls under the rubric of steganography: cloaking one type of data file within another. It is possible, for example, to hide a text file with attack plans within a bit-mapped photo of Britney Spears. Just try to filter down the number of those images flying around the Internet.

And even the most advanced spying technology can be stymied by embarrassingly primitive countermeasures. Conspirators can go the old-fashioned route of disguising their activities by using simple ciphers that substitute letters for numbers or other letters; Thomas Jefferson used such codes in his international communiqus as George Washington’s secretary of state. Cigital’s McGraw says this would be the easiest way to avoid detection: “To use a crude example: maybe the terrorists substituted the word banana’ for bomb’ and orange’ for World Trade Center.’ Do you flag every unusual pattern with random associations?”

Beyond the pure technology issues lies the question of how these tools can be used in a way that is compatible with an open and democratic society. Even in the rally-round-the-flag mood following the attacks, many U.S. citizens expressed concern about the government’s expanding authority to snoop on their movements and communications. Organizations like the Electronic Frontier Foundation are highly vigilant about governmental attempts to expand the use of surveillance technologies such as Carnivore. “We really have no sense beyond a few basics they decided to reveal about how they use these tools,” says Lee Tien, senior staff attorney for the organization. “They just want us to accept that they need them, without explaining why or how.”

And while technologies like Carnivore have proved useful in investigations of specific individuals, they could be abused when directed at wider groups. People can quickly become “suspects” on no more evidence than an e-mail received or a Web site visited.

In the end, computer-based surveillance technologies may be best employed after the fact, says John Pike, director of, a Web-based military and intelligence policy group headquartered in Alexandria, VA. He notes that Carnivore, in particular, “was very effective in tracking down” and arresting former FBI agent and Soviet spy Robert Hanssen. “It also helped dramatically after the bombing to track down these terrorists’ activities. It helped them detain at least 400 to 500 other people as suspects.” According to Pike, U.S. citizens are going to have to become comfortable with such mass arrests if this type of technology is going to be used.

Even if the obstacles of bureaucracy, societal resistance and technical limitations were all to be surmounted, there’s no assurance that high-tech spyware would ever provide the kind of security that people now crave. Will these technologies help recognize the danger next time? Even the most sophisticated intelligence paraphernalia still can’t guarantee success when pitted against the malevolent combination of human ingenuity and capacity for evil.

Deep Dive


Capitalizing on machine learning with collaborative, structured enterprise tooling teams

Machine learning advances require an evolution of processes, tooling, and operations.

The race to destroy PFAS, the forever chemicals 

Scientists are showing these damaging compounds can be beat.

How scientists are being squeezed to take sides in the conflict between Israel and Palestine

Tensions over the war are flaring on social media—with real-life ramifications.

Stay connected

Illustration by Rose Wong

Get the latest updates from
MIT Technology Review

Discover special offers, top stories, upcoming events, and more.

Thank you for submitting your email!

Explore more newsletters

It looks like something went wrong.

We’re having trouble saving your preferences. Try refreshing this page and updating them one more time. If you continue to get this message, reach out to us at with a list of newsletters you’d like to receive.