The year was 1961. Computers were still in their infancy, and the race to the moon was just beginning. Edward Lorenz, an MIT meteorologist, was developing a weather- prediction model. Lorenz theorized that a miniscule occurrence, such as a tiny butterfly flapping its wings in the Amazon, could hypothetically set in motion a chain of events that could cause tornadoes to touch down in Texas a few days later.

That model (illustrations of which visually resembled a butterfly) eventually came to be known as the butterfly effect. As a metaphor, the butterfly effect has come to signify a series of seemingly trivial and unrelated events that collectively have a massive impact later, whether in causing storms or influencing the stock market.

While there’s some disagreement on the Lorenz model’s ability to accurately predict anything as intrinsically complex as the weather, the concept has clearly struck a chord with the general public. Think Ashton Kutcher, as a college-age time traveler seeking to alter his traumatic childhood in The Butterfly Effect; or the Robert Redford character in Havana, who glibly proclaims: “A butterfly can flutter its wings over a flower in China and cause a hurricane in the Caribbean. They can even calculate the odds.”

Maybe Redford (or at least the screenwriter) was on to something. The world is now awash in data, and more of it has been created in just the last two years than in the rest of human history, according to the Scandinavian research group SINTEF. The situation isn’t likely to improve anytime soon. By 2020, about 1.7 megabytes of new information will be created every second for every human being on the planet, according to a recent IDC Digital Universe study. At that point, the world will be looking at digital knowledge in the neighborhood of 44 zettabytes, or 44 trillion gigabytes, up from just 4.4 zettabytes today.

The challenge remains how to troll through this massive (and ever expanding) sea of data fast enough and meaningfully enough, so that we can extract useful insight from it and figure out what will influence what. That, in turn, would allow for pragmatic actions to be taken before the metaphoric tsunami hits, perhaps even before the wave has formed.

A SEA CHANGE IN DATA ANALYSIS

Until now, understanding and using this data has remained an imperfect science, albeit one that is changing fast. First there was business reporting, which allowed for a reactive analysis of data to spot trends and patterns. Then came predictive analytics, which deployed sophisticated mathematical tools to forecast based on historical and current data patterns.  

Now, with advances in database technology and the ability of innovative systems such as SAP HANA to handle massive data loads in memory, we are seeing a whole slew of software applications that are trying to do what the butterfly effect promises—that is, to predict the course of a tsunami from tiny ripples, as things are happening, in real time.

The business implications of being able to make early predictions are indeed huge. Imagine a trader of precious metals on a commodities exchange who can be alerted to a potential supply-chain disruption, arising out of a labor dispute that is threatening to get out of hand at a major mine in Indonesia. Or conjure up a car maker in Germany who learns that certain parts being sent from overseas will arrive later than expected—and is therefore able to change manufacturing schedules in time, saving millions of dollars that would otherwise be lost to idle time.

ANALYTICS IN ACTION

This is the kind of prediction that Semantic Visions, Verint, Factiva, Palantir, and others are all trying to do, although each has its unique area of focus. Semantic Visions, an early member of the SAP Startup Focus program, which is based in the Czech Republic, offers technology that can predict supply-chain disruptions in real time and is especially designed for large manufacturers that have thousands of suppliers worldwide. Disruption on a global scale often begins as tiny, insidious events flying well below the radar of major news outlets, often covered only in local non-English media, if at all.

To capture this information, Semantic Visions has developed a unique cross-language semantic analysis technology that enables it to extract knowledge from Web content, in whatever language it is written. The company has condensed the world’s major languages into one universal semantic language by creating machine-readable identifiers that have the same meaning and context regardless of language. This makes it possible to interconnect and leverage information independent of the language in which it is available, which means that language-defined silos (for instance, the Internet in China) are no longer bottlenecks.

Semantic Vision conducts ongoing assessment of prevailing media sentiment not just about companies or industries, but even about entire countries. Its research shows that negative sentiments in Russian-language media (which tend to comprise an echo chamber for the Kremlin) have steadily risen, increasing tenfold in the last two-plus years, well before the actual hostilities started. What that appears to indicate is that, even before guns were fired or tanks placed into position, the Russian government was effectively managing public opinion.

And then, of course, there are the applications in the national security arena. In a world where the headlines are dominated by conflict (ISIS, Iraq, Syria) and disease (Ebola in Liberia and re-emerging polio in Pakistan), it’s not surprising that the potential applications of such technology are immense, far beyond the business world where the software vendors publicly look for customers.

As might be expected, information on such nonbusiness projects is hard to come by. However, Palantir tends to dominate the headlines, with software that has helped identify the core group of 27 men behind the killing of American journalist Daniel Pearl in 2002, uncover the GhostNet computer network that was infecting the systems of many countries’ embassies, and enable several U.S. police departments to do predictive policing using data analysis to take a proactive approach to patrol deployment. The futuristic movie Minority Report—in which a special police force proactively arrests murderers before they commit crimes—is set in 2054, but it seems that in some ways, we’re already there.

MOVING CLOSER TO THE BUTTERFLY EFFECT

While scientifically it may be impossible to actually predict whether a butterfly in Brazil (or Bulgaria or Benin) will ultimately cause a twister to touch down in Texas, the social impact of the butterfly-effect theory is undeniable in today’s hyperconnected and data-driven world. The Arab Spring, after all was, essentially, started by an unknown Tunisian street vendor, Mohamed Bouazizi, who set himself on fire to protest police harassment. His widely reported death was able to accomplish what massive armies and decades of western influence had been unable to do: generate enough outrage to cause a rebellion.

As data troves become richer and data-management tools grow more sophisticated, we’re getting closer to the point where someday in the not-too-distant future, the butterfly effect may indeed describe a mathematically predictable reality.

Follow me @BansalManju and join the conversation at @SAPStartups.