The University of Iowa’s Iowa Electronic Markets, one of the earliest prediction markets, has significantly outperformed the polls in every presidential election when forecasting more than 100 days in advance: Compared to 964 polls over the five presidential elections since 1988, the Iowa market was closer to the eventual outcome 74 percent of the time. The University of Iowa also uses prediction markets to forecast seasonal flu outbreaks.
Prediction markets have a major built-in bias—those answering the questions are not polled randomly—but respondents also have an incentive to respond only to those questions they feel confident in answering with accuracy.
“Prediction markets aren’t just surveys that ask everyone to speak up,” Robin Hanson, chief scientist at Consensus Point. ” People tend to speak up only when they’re reasonably sure they know the answer.”
Consensus Point CEO Linda Rebrovick says the goal of the project is to attract a network of about 250 experts, although the organizers are still deciding how to compensate for correct answers.
“There will be some combination of rewards and financial incentives for participating,” Rebrovick says.
Even if questions generate only tepid responses, such responses can be informative, says Dan Geer, chief information security officer at In-Q-Tel, the venture capital arm of the Central Intelligence Agency (CIA). Geer is also involved in the project. “It may be that this tells us there is ambiguity, or that we are, in effect, measuring disagreement on a question that doesn’t have a quantitative aspect to it,” Geer says. “Straight-out surveys are vulnerable to idiot answers, and prediction markets are vulnerable to stupid questions.”
While the pilot project will be limited to invited information security experts, the consensus decisions reached by the group will be made public. “Even if we can’t find something useful in all of this, we feel that’s a valuable result. It’s the way you make progress,” Geer says.