For years, government officials, news articles, and security companies have warned about the dangers and impact of cybercrime. Patrick Peterson, chief security researcher at Cisco, has estimated that losses totaled $560 million in 2009. Killian Strauss of the Organization for Security and Cooperation in Europe has estimated them at $100 billion annually. And in March 2009, Edward Amoroso, AT&T’s chief security officer, submitted written testimony to the U.S. Senate Committee on Commerce, Science, and Transportation estimating that cybercrime was bringing in illicit revenues of approximately $1 trillion a year.
To some researchers, those wildly different numbers suggest that current methods for calculating cybercrime losses are so poor we actually have no idea how bad the problem is. And without good data, they say, there’s no way to fight it intelligently.
“How can this be?” says Cormac Herley, a principal researcher at Microsoft Research, his voice rising in incredulity. “How can you have estimates of the same problem ranging across three orders of magnitude?”
In fact, Herley says, when he saw these numbers he felt they “just didn’t make sense.” Not only are they all over the map, but some of them also seem impossibly high. For example, he says, cybercrime revenues of $1 trillion mean $5,000 for every U.S. adult who spends time online.
Bad data has consequences. “Without numbers, we can’t make good policy or sound investment decisions,” Herley says. Not only that, but we can’t figure out where key threats are coming from. Are the criminals making most of their money from key logging? Highly targeted phishing attacks (“spear phishing”)? Brute-force attacks on people’s passwords? “It’s distressing,” he says.
Herley embarked on a study of the methods used to calculate these numbers and found them severely wanting. Most of the statistics come from surveys in which respondents are asked to report whether they’ve been victims of a crime and how much they lost. “Surveys are hard,” Herley says. His research revealed a number of reasons why surveys about cybercrime are particularly hard.