 |
by Nick Charney |        |
I'd like to spend some time exploring whether or not the idea of a Policy Analysis Market (or if you prefer a prediction market for policy options) has legs in the broader public policy context. In so doing, I will be drawing on a handful of papers I've read on the subject, citing them heavily, reserving most of my own commentary for the end. Each of the quotations are linked to the source and I've provided the references as a laundry list of recommended reading at the end of this piece.
What is a prediction market?
"Prediction markets are forums for trading contracts that yield payments based on the outcome of uncertain events. There is mounting evidence that such markets can help to produce forecasts of event outcomes with a lower prediction error than conventional methods... Several researchers emphasize the potential of prediction markets to improve decisions. The range of applications is virtually limitless – from helping businesses make better investment decisions to helping governments make better fiscal and monetary policy decisions ... These markets could assist private firms and public institutions in managing economic risks, such as declines in consumer demand. and social risks, such as flu outbreaks and environmental disasters, more efficiently." (The Promise of Prediction Markets)
"The theories underlying [Policy Analysis Market] and other prediction markets are the Efficient Capital Markets Hypothesis (ECMH) and the Hayek hypotheses. These hypotheses explain how information is aggregated such that market prices provide accurate estimates on the likelihood of future outcomes." (Using Prediction Markets to Enhance US Intelligence Capabilities)
"The power of prediction markets derives from the fact that they provide incentives for truthful revelation, they provide incentives for research and information discovery, and the market provides an algorithm for aggregating opinions." (Prediction Markets)
How do prediction markets work?
"Trading in prediction markets is similar to any haggling kind of transaction: buyers and sellers exchange offers and counter-offers until they agree on a price. In a double auction, the most common mechanism used to clear prediction markets, buyers submit bids and sellers submit asking prices, which are ranked from highest to lowest to generate supply and demand curves. Trades are executed when two prices match (i.e., bid-ask spread is zero or supply intersects demand) ... [and] payoffs are determined by the occurrence (or lack thereof) of outcomes." (Using Prediction Markets to Enhance US Intelligence Capabilities)Generally speaking:
"Market prices for contracts can be interpreted as probabilities of an expected outcome ... [for] example, a contract closing at 67 cents would mean there is a 67 percent probability." (Using Prediction Markets to Enhance US Intelligence Capabilities)
What purpose do prediction markets serve?
"Numerous studies have suggested, however, that markets do lead to predictions that are more accurate than traditional forecasting techniques, including those that rely on expert opinions. (Using Prediction Markets to Enhance US Intelligence Capabilities)"
"Exploring the possibilities of prediction markets further, others have proposed that these markets should serve as mechanisms to help decide which of several policies options should be implemented." (Using Prediction Markets to Enhance US Intelligence Capabilities)
"The 9/11 Commission, in its discussion of how to reorganize the US Intelligence Community, cited the lack of unity of effort in information sharing as the “biggest impediment to all-source analysis—to a greater likelihood of connecting the dots.” The lack of information sharing is further compounded by a culture that emphasizes information compartmentalization, suffers from stovepipe mentalities, and bureaucratic distrust. One way to solve these problems is to work on [Intelligence Community]-wide software and databases and develop improved protocols for accessing classified information and for providing better coordination of interagency analyses. Another way is to use prediction markets to aggregate information and analyses ... information and judgments from different corporate divisions into probabilistic estimates of future outcomes, a prediction market could perform the same function for the Intelligence Community." (Using Prediction Markets to Enhance US Intelligence Capabilities)
Have prediction markets worked before in a policy context?
The Defense Advanced Research Project Agency (DARPA) experimented with this approach back in 2001 when it created a Future Markets Applied to Prediction (FutureMAP) program that tested whether prediction markets, could be used to improve upon existing approaches to preparing strategic intelligence. Long story short, the program was cut short when congressional (faux) outrage took over and shut down the project; apparently the optics of intelligence officers placing wagers on terrorist activities didn't sit well with them and they thought such activities might actually incite actors to take measures they otherwise wouldn't have. That said, much of the research I've done have indicated considerable upside to the approach:
"Prediction markets can function as powerful complements to the traditional process by which long-term estimates are performed. Their power is further multiplied when one imagines that the time and resources saved in running such markets means that several long-term estimates can be run concurrently and updated periodically. ... by allowing analysts to hedge their estimates in the form of conditional contracts, policymakers gain valuable probabilistic estimates, as opposed to wishy-washy judgments which policymakers may easily ignore." (Using Prediction Markets to Enhance US Intelligence Capabilities)
"Prediction markets could also be used to make ex-ante evaluation of policies. Take the question of whether the United States should continue to fund the Andean Regional Initiative (ARI). Analysts could bet on two futures contracts: (1) the tons of cocaine that will be exported from the countries affected by the ARI to the United States in 2009, conditional on the United States continuing ARI; and (2) the tons of cocaine that will be exported if ARI is terminated.The difference in the two estimates would tell policymakers how much of a reduction (or increase) in cocaine analysts expect from the implementation of ARI. A more realistic assessment would most likely involve analysts speculating on several futures contracts with different expiration dates." (Using Prediction Markets to Enhance US Intelligence Capabilities)
Who ought to participate in such a prediction market?
"Since the objective here is to effectively aggregate information and analyses of the entire Intelligence Community, implementation of prediction markets on a community-wide basis is preferable to intra-agency markets. Ideally, anyone with the relevant information should trade. If the traded contract relates to aerial suicide bombs, then even airport luggage screeners, in addition to homeland security analysts, are potential market participants. This necessarily means that expert knowledge on a particular subject is not required before making a bet.Moreover,
A more difficult question is whether there are circumstances under which the general public should be allowed to trade. Certain issues might require the aggregation of information and opinions on subjects intelligence officers may know little or not enough about. On the other hand, making public certain markets might be inadvisable because doing so might signal adversaries about intelligence interests.
A compelling case can be made for making diversity a key criterion. Diversity means that market participants have different pieces of information about their surrounding environment and consequently different judgments on the event for which they are betting. The HP experiment aggregated information across several corporate divisions. Economic theory and empirical evidence suggests that “thick” markets are preferable to “thin” ones."(Using Prediction Markets to Enhance US Intelligence Capabilities)
"Ambiguous public information may be better in motivating trade than private information, especially if the private information is concentrated, since a cadre of highly informed traders can easily drive out the partly informed, repressing trade to the point that the market exists" (Prediction markets for business and public policy)
What would a policy options prediction market look like in the Canadian policy context?
At the systems level, a cross government policy options prediction market holds out the promise of providing a better interdepartmental view on success/failure probabilities of different policy interventions because the aggregating forces of the market helps overcome the inherent information asymmetries built into Westminster accountability structures. It could also provide tremendous insight into how different departments view a given issue because you could run analysis on whom (point of origin) is investing where.
If such a market was opened up to broader public participation it would provide policy makers with a better overall societal view of the options on the table. Further, there's an interesting argument to be made here about whether or not such a market could help redefine participative democracy in a digital era and improve overall public engagement by offering continuous options for engagement across a spectrum of issues rather than limiting it public participation to issue specific and time-limited opportunities for feedback. In so doing this approach could also give policy makers valuable insight into public preference on a particular subset of issues, say whether or not the public prefers prevention or remediation strategies, by how they invest (prioritize) between competing options.
An interesting place to test this type of broader approach may be in the field of social finance where it could be used to see how different players (funders, service providers, stakeholders, clients, and government officials) see the probability of success of a given initiative and/or how they value the other actors in the ecosystem.
Overall, I think the idea has merit.
Three closing but related caveats about design
First, the research clearly shows that contracts in such a market would need to be clear and specific, written in plain language so as to be easily understood by participants.
Second, the research also shows that the market's success is contingent on participants' motivation to trade and that the profit motive is usually enough to spur activity; this tells me that the design of the overall prediction market needs to be gamified in some way.
Third, an interesting place for people interested in this type of design might be Empire Avenue, which has a whole bunch of those types of design decisions built into it under the hood. It's a social media service that a bunch of us got into years ago but subsequently walked away from. That said, it may be worth a second look in the context of this conversation around the applicability of prediction markets to policy options.
Recommended Reading / Resources
- Using Prediction Markets to Enhance US Intelligence Capabilities: A "Standard and; Poors 500 Index" for Intelligence
- Prediction Markets by Justin Wolfers and Eric Zitzewitz, National Bureau of Economic Research
- Prediction markets for business and public policy, Andrew Leigh and Justin Wolfers, The Melbourne Review
- The Promise of Prediction Markets, Science Magazine
