Back in 2003, the US clothing manufacturer Levi Strauss made the fateful decision to upgrade its global IT system. The project was budgeted at $5million and the advantages appeared huge. 

As Levi Strauss had grown into a global company, its IT system had become fragmented and antiquated, with different systems in operation in different countries. So switching to a single integrated system seemed eminently sensible. “But very quickly all hell broke loose,” say Bent Flyvbjerg and Alexander Budzier at the University of Oxford in the UK.

For example, the supermarket giant Walmart, one of its biggest customers, demanded that the new IT system connect seamlessly with its own supply chain management system. That created an entirely unanticipated set of problems.

Then, when the new system was switched on, the company discovered that it could not fulfill orders and had to close its three US distribution centres for a week. In 2008, five years after it began the project, the company took a $193 million charge against earnings to compensate for the problems and company’s chief information officer was forced to resign.

That’s one example of a nightmare scenario–a  botched IT project that forced a major company to its knees. But this kind of problem is much more common than you might imagine, say Flyvbjerg and Budzier, who have carried out the largest global study of IT change initiatives ever conducted. “We examined 1,471 projects, comparing their budgets and estimated performance benefits with the actual costs and results,” they say.

The results are eye-opening and should come as a warning shot for anyone in charge of major IT initiatives. Flyvbjerg and Budzier say that the average cost overrrun for an IT project is 27 per cent. That seems more than manageable for most companies.

But this figure holds a surprise because more than one in six of the projects these guys examined had a cost overrun of 200 per cent.  

“This highlights the true pitfall of IT change initiatives: It’s not that they’re particularly prone to high cost overruns on average, as management consultants and academic studies have previously suggested. It’s that an unusually large proportion of them incur massive overages,” they say.

In other words, the average cost overrun is an entirely inappropriate measure of the state of affairs and gives little indication of the true risks associated with IT projects.  

The limitations of taking averages are well known in many areas of science. An average is only a useful figure when it is associated with certain distributions. For example, the height of fully grown men follows a so-called normal distribution for which an average is a useful description. However, it’s possible to calculate the average size of an earthquake but the figure is entirely nonsensical since it gives no indication of the size range of earthquakes which vary over many orders of magnitude. If building standards were based on the average size of earthquakes, they would fail to account for the huge devastation that big earthquakes can cause.

IT projects fall in to a similar category. Calculating the risk associated with an IT project using the average cost overrun is like creating building standards using the average size of earthquakes. Both are bound to be inadequate.  

These dangers have yet to be fully appreciated, warn Flyvbjerg and Budzier. “IT projects are now so big, and they touch so many aspects of an organization, that they pose a singular new risk….They have sunk whole corporations. Even cities and nations are in peril.”

They point to the IT problems with Hong Kong’s new airport in the late 1990s, which reportedly cost the local economy some $600 million.

They conclude that it’s only a matter of time before something much more dramatic occurs. “It will be no surprise if a large, established company fails in the coming years because of an out-of-control IT project. In fact, the data suggest that one or more will,” predict Flyvbjerg and Budzier.

CIO’s–you have been warned.

Ref: arxiv.org/abs/1304.0265: Why Your IT Project May Be Riskier Than You Think