Transcript Combinatoric Information Markets

Information Markets
John Ledyard
April 13, 2005
Nancy Schwartz Memorial Lecture
KGSM, Northwestern University
What would you like to know?
• What will Disney’s profits be in 2006?
– Ask an accountant, a stock analyst, a consultant, the
CEO, Mickey Mouse, …
• Who will the next Pope be?
– Ask a pundit, a cardinal, take a poll, ……
• How stable will the middle-east be on 12/31/05?
– Ask the CIA, the Mossad, the Defense Department,
the President, a committee of experts, …….
• Should you set up an Information Market?
What is an Information Market?
• This is not about markets for information.
• Kihlstrom (1974), Radner and Stiglitz (1984), Kamien and
Tauman (1990), Keppo, Moscarini, Smith (2005)
• It is about using market forces to bring together
disparate bits and pieces of information and add
them up, or aggregate them, for use in predictions
or decisions.
• Google hits
• Information Markets = 25,700,000
• Prediction Markets = 1,550,000
How would it work?
• Familiar Question: Who will win an election?
– Standard approach - Polls.
• In 1988, University of Iowa Business School
securitized the Presidential election prediction on
the internet.
Quick Time™ an d a
TIFF ( Un compr ess ed ) de compr ess or
ar e n eed ed to s ee this pic ture .
The Iowa Election Market
QuickTime™ and a
TIFF(Uncompressed) dec ompress or
are needed to see this pic ture.
 In 1992, for $1.00, Iowa sold or bought a set of securities
that covered all possible outcomes of the election; Bush,
Clinton, Other (included Perot).
QuickTime™and a
TIFF(Uncompressed) decompres sor
are needed to see this pi cture.
Quick Time™ an d a
TIFF ( Un compr ess ed ) de compr ess or
ar e n eed ed to s ee this pic ture .
The Iowa Election Market
QuickTime™ and a
TIFF(Uncompressed) dec ompress or
are needed to see this pic ture.
 In 1992, for $1.00, Iowa sold or bought a set of securities
that covered all possible outcomes of the election; Bush,
Clinton, Other (included Perot).
 Each security paid $1 times the percentage of the vote for
that person. Securities were traded.
QuickTime™and a
TIFF(Uncompressed) decompres sor
are needed to see this pi cture.
Quick Time™ an d a
TIFF ( Un compr ess ed ) de compr ess or
ar e n eed ed to s ee this pic ture .
The Iowa Election Market
QuickTime™ and a
TIFF(Uncompressed) dec ompress or
are needed to see this pic ture.
 In 1992, for $1.00, Iowa sold or bought a set of securities
that covered all possible outcomes of the election; Bush,
Clinton, Other (included Perot).
 Each security paid $1 times the percentage of the vote for
that person. Securities were traded.
 After the election tally, if you owned 100 shares of Bush
and Bush received 38% of the vote then you would be paid
$38.
QuickTime™and a
TIFF(Uncompressed) decompres sor
are needed to see this pi cture.
Quick Time™ an d a
TIFF ( Un compr ess ed ) de compr ess or
ar e n eed ed to s ee this pic ture .
The Iowa Election Market
QuickTime™ and a
TIFF(Uncompressed) dec ompress or
are needed to see this pic ture.
 In 1992, for $1.00, Iowa sold or bought a set of securities
that covered all possible outcomes of the election; Bush,
Clinton, Other (included Perot).
 Each security paid $1 times the percentage of the vote for
that person. Securities were traded.
 After the election tally, if you owned 100 shares of Bush
and Bush received 38% of the vote then you would be paid
$38.
 The actual result was Clinton 43%, Bush 38%, Perot 19%
QuickTime™and a
TIFF(Uncompressed) decompres sor
are needed to see this pi cture.
How the IEM might work.
• You go to the IEM website and see
Prices
Bush
$.50
U think
.3
Clinton
$.40
.7
Other
$.10
0
• You see a way to make some money.
How the IEM might work.
Bush
Prices
$.50
U Think .3
U buy
1
Clinton
$.40
.7
1
Other
$.10
0
1
Cash
E(V)
-$1
$1-$1 = 0
How the IEM might work.
Bush
Clinton
Other
Cash
Prices
$.50
$.40
$.10
U Think .3
.7
0
U buy
1
1
1
-$1
U trade -1 @ $.35 +1 @ $.50 -1 @ $.05 -$.10
E(V)
$1-$1 = 0
How the IEM might work.
Bush
Clinton
Other
Prices
$.50
$.40
$.10
U Think .3
.7
0
U buy
1
1
1
U trade -1 @ $.35 +1 @ $.50 -1 @ $.05
U have
0
2
0
Cash
E(V)
-$1
$1-$1 = 0
-$.10
-$1.10 1.40-$1.10
You actually will make (0.38x2) - 1.10 = -$0.34.
But you don’t know that when you make this
transaction. You can only act on your beliefs.
How the IEM might work.
Bush
Clinton
Other
Prices
$.50
$.40
$.10
U Think .3
.7
0
U buy
1
1
1
U trade -1 @ $.35 +1 @ $.50 -1 @ $.05
U have
0
2
0
Cash
E(V)
-$1
$1-$1 = 0
-$.10
-$1.10 1.40-$1.10
Other traders then adjust their beliefs in response to the
price changes. And so on.
If all goes well, in equilibrium, prices will equate to the fullinformation beliefs of the traders.
And if all goes well, these will be the true vote-shares.
Source: IEM (2005)
1992 U.S. Election
Source: IEM (2005)
How Accurate Has IEM Been?
100
90
Predicted Outcome (%)
80
70
60
Tsongas
'92 MI Primary
50
Tsongas
'92 IL Primary
US Presidential Elections
Avg. Abs. Err. = 1.37%
(5 Markets, 12 Contracts)
Other US Elections
Avg. Abs. Err. = 3.43%
(14 Markets, 50 Contracts)
Non-US Elections
Avg. Abs. Err. = 2.12%
(30 Markets, 175 Contracts)
40
30
20
Brown
'92 MI Primary
10
0
0
10
20
30
40
50
60
Actual Outcome (%)
70
80
90
100
Also…..
• National election market in NY (1868-1940)
– [Rhode and Strumpf (2004)]
– “Over $165 million (in 2002 dollars) was wagered in
one election, and betting activity at times dominated
transactions in the stock exchanges on Wall Street.”
– “In only one case did the candidate clearly favored in
the betting a month before Election Day lose.”
Is this a Free Lunch?
•Iowa pays nothing.
•On average, the traders
earn nothing
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
•But, in the end, everyone is
better, maybe even
maximally, informed.
“The Next Killer App”?
or “Too Good To Be True”?
• Evidence, mostly empirical, suggests
Information Markets Work.
• Evidence, mostly theoretical, suggests
Information Markets Can’t Work.
• Today we will explore
– how “Information Markets” work
– how to design and engineer viable and accurate
“Information Mechanisms”
Why might an IM work?
• There are two of us in this scenario.
– (Neither of us is a Game Theorist.)
• There are 2 coins:
– Coin A comes up heads 80% of the time.
– Coin B comes up heads 20% of the time.
• One is chosen with probability .5.
– This is our common prior.
• The coin is flipped twice for each of us.
– You see (H,T) and I don’t see that.
– I see
and you don’t see that.
Why might an IM work?
• Remember: Coin A = .8 heads, Coin B = .2 heads,
you see (H,T), I see (?, ?).
• What is the probability that the coin is A?
• Based on only your information, the answer is 0.5.
– This is your initial posterior.
• Suppose there is a Market Maker who posts prices
and asks us whether we want to buy or sell an
asset that pays $1 if the coin is A and $0 if it is B.
• He posts a price of 0.60.
• You offer to sell and I offer to buy.
Why might an IM work?
• Remember: Coin A = .8 heads, Coin B = .2 heads,
you see (H,T), I see (?, ?), I offered to buy at .6.
• What is the probability that the coin is A?
• Since you know that I must have seen
(H,H), you know (H,T,H,H). This is
everything.
• Your
answer
should
Of course,
I only
knowbe
that0.94.
you are either
(H,T) or (T,T), so I don’t know everything - yet.
Why might an IM work?
• Suppose the Market Maker still posts a price of .6.
• We both offer to buy.
• I now know that your current posterior is .94
which means you must have seen (H,T)
• So we both now know that the total information is
(3H, 1T) and our posteriors are the same: 0.94.
• The “market” has “aggregated” the “information”!
• The underlying theories are Rational Expectations
Equilibrium and Common Knowledge
Information.
• Green (1973), Lucas (1972), Grossman (1977)
• Aumann (1976), Geanakopolos and Polemarchakis (1982)
But Wait!!!!
There is something fishy here!
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Market Maker
?
=
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Maxwell’s Demon
Why might an IM not work?
• Let’s go back to the Market and get rid of the
Market Maker.
• Remember: Coin A = .8 heads, Coin B = .2 heads,
you see (H,T), I see (?, ?), the asset pays $1 if A.
• I offer to sell you 2 units of the asset for .30.
• What should you do now?
• Infer that I saw (T,T) and believe (1 H, 3T).
• So you now should believe that P(A) = .06
Why might an IM not work?
• I offer you 2 units of the asset for .30, you saw
(H,T) and know I saw (T,T), so your P(A) = .06.
• You believe the expected value of the asset is .06.
• Obviously you should reject my offer.
• The full information is either (1H,3T) or (0H,4T).
– If you had seen (H,H) you would accept my offer.
• If you bid to buy above .004, I also know it is 0.6.
• We will not trade!
• The underlying theory is No-Trade Theorems
• [Grossman and Stiglitz (1976), Milgrom-Stokey (1988)]
What About Empirical Evidence?
• There are many naturally occurring IM’s
• And, in direct comparisons,
they beat other institutions.
Pari-mutuel betting
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Pari-mutuel betting
• Racetrack odds beat track experts
– Figlewski (1979)
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Futures markets
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Futures markets
• OJ futures improve freeze forecasts
– Roll (1984)
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Stock markets
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Stock markets
• Stock prices beat the experts panel in the
post-Challenger probe
– Maloney & Mulherin (2003)
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Less Positive Field Evidence
• The consensus forecast (median of about 30
economists) has as much predictive power as the
Goldman-Sachs pari-mutual market.
•
[Wolfers and Zieztwitz (2003)]
• Wide bid-ask spreads and thin trading on most
Tradesports.com markets.
– Politics (4/11/05) (contract, price, spread, vol.)
• 2008DemnomClinton
• 2008 RepnomJBush
40
10
1.5
.2
9135
2685
– Papacy (4/11/05) (contract, price, spread, vol.)
– Italy
– Nigeria
– USA
42
13
0.2
.9
1.7
.5
2045
1454
274
The Experimental Evidence
is Mixed
• A number of experimentalists have demonstrated
convergence to the full information rational
expectations equilibrium.
– In laboratory asset markets with one asset
• Forsythe, Palfrey, Plott (1982)
– In laboratory elections
• McKelvey & Ordeshook (1985)
– In laboratory asset markets with 3 assets
• Plott and Sunder (1988)
The Experimental Evidence
is Mixed
• A number of experimentalists have demonstrated
that information mechanisms do not always work.
– In laboratory asset markets, if preferences differ and
there are incomplete markets, there is little aggregation.
• Plott and Sunder (1988) Risk or ambiguity aversion?
– In iterative polls in the laboratory, there is incomplete
information aggregation.
• McKelvey & Page (1990) Incomplete Bayesian updating?
– In laboratory pari-mutuel betting, we observe mirages.
• Plott (2002) (Pari-mutuel) Information cascades?
One More Set of Data
[Grus and Ledyard (2005)]
•
•
•
•
•
•
•
The 2 coin example, 2 flips each
N = 3, 7, 8, 12: Caltech subjects
Market, Pari-mutuel
3 minutes of transactions per period
8 periods per mechanism
3 mechanisms per session (2.5 hours)
Earnings approximate $33/subject
Numbers Matter!
Markets and Pari-mutuels
1
0 .9
0 .8
Probability
0 .7
These are big mirages.
0 .6
PM3
0 .5
MP informational size = .58 for N = 3
= .20 for N = 7
= .02 for N = 12
0 .4
0 .3
0 .2
kl(.8,.65)=.06
kl(.8,.50)=.22
kl(.8,.20)=.83
These get it!
0 .1
0
0 .0 0
0 .1 0
0 .2 0
KL distance
0 .3 0
0 .4 0
M3
PM12
M12
Summary Statistics
32 observations
Got it
Pari-Mutuel
Market
41%
63%
“Got it” means KL < .01 or | p-FI | < .05
When N > 6, the market “gets it” 80% and the
pari-mutuel gets it 75%.
Summary Statistics
32 observations for PM, M, P
Pari-Mutuel
N= 3, 12
Got it
% Early
Market
30% (75%) 55% (80%)
0%
38%
Early means in 1st 3 transactions. Not in time.
Summary Statistics
32 observations for PM, M, P
Pari-Mutuel
Got it
Market
30% (75%) 55% (80%)
% Early
0%
38%
Mirages
25%
22%
Mirage means p is not the FI and is one of the other possibilities.
Summary Statistics
32 observations for PM, M, P
Pari-Mutuel
Got it
Market
30% (75%) 55% (80%)
% Early
0%
38%
Mirages
25%
22%
tickets or
N = 3, rest
transactions
2.5, 7.8
per subject
N = 3, rest
1.6, 6.2
Based on the 2-coin experiments
(and more complicated ones)
• Some aggregation occurs but it is not perfect.
• Markets are faster than pari-mutuels and get it
more often.
• Both are subject to mirages.
• Neither perform well at low-scale (N = 3).
• There is evidence to support the no-trade
hypothesis, particularly when N = 3.
– But there is still some trading.
• Incomplete updating? Not here.
• Boredom - yes, lots of little trades
• Greater fool? - Possibly.
Summary to here
• There is theoretical, experimental, and field
evidence that traditional IM’s may work.
• But there is also evidence that there are
impediments to complete information
aggregation especially in environments with
informationally large agents.
• Should we worry about small numbers?
• Can we find better information
mechanisms?
Numbers and Informationally
Large Traders
• Probability of success in next year for drug A?
• Expected sales of SUV model X in 2006
contingent on gas prices above $5 on July 2006?
• The expected software shipping date contingent on
retaining feature R?
• Expected benefits of a government program
conditional on one of several possible actions?
(better cost-benefit analysis?)
Remember PAM?
8 nations, 5 indices,
4 quarters
-Political stability
-Military activity
-Economic growth
-US $ aid
-US military activity
Up, Down, or Constant
Implies 320 active markets.
•Example contract: Jordan is more politically stable in
4Q2005 conditional on US military activity down in
Iraq in 3Q2005 and US$aid in 2Q2005 up in Iran.
Remember PAM?
8 nations, 5 indices,
4 quarters
-Political stability
-Military activity
-Economic growth
-US $ aid
-US military activity
Up, Down, or Constant
Implies 320 active markets.
•320 questions and completeness
implies 2^320 = 2 * 10^96 contracts.
Need Better
Information Mechanisms
• To be useful in many potential applications,
Information Markets need to perform well with
small numbers and informationally large traders.
• Traditional markets and pari-mutuels are not up to
this.
• Two possible approaches
– Subsidize the action in the traditional designs.
– Design new mechanisms.
Better Information Mechanisms?
• Let us first try to subsidize and modify the
traditional IM’s.
– Pari-mutuel: Add some tickets into the pot so
that the expected payoff of spending $1 is
larger than $1.
– Market: Randomly accept bids and offers at
“market.”
• Noise trading [Grossman and Stiglitz (1976)]
Pari-mutuel with and without subsidy
1
0 .9
0 .8
Probability
0 .7
0 .6
PM3
P M /S3
0 .5
PM12
0 .4
Helps when N = 3
P M /S1 2
0 .3
0 .2
kl(.8,.65)=.06
kl(.8,.50)=.22
kl(.8,.20)=.83
0 .1
0
0 .0 0
0 .1 0
0 .2 0
KL distances
0 .3 0
0 .4 0
Markets with and without subsidy
1
0 .9
0 .8
Probability
0 .7
0 .6
M3
0 .5
M /S 3
0 .4
M12
Helps when N = 3
M /S 1 2
0 .3
0 .2
kl(.8,.65)=.06
kl(.8,.50)=.22
kl(.8,.20)=.83
0 .1
0
0 .0 0
0 .1 0
0 .2 0
0 .3 0
KL distance
Does this mean “Noise Creates Information”?
0 .4 0
Better but Still not Great
PM
PM/S
M
M/S
Got it
41%
31%
63%
66%
% Early
0%
3%
47%
56%
Mirages
25%
13%
22%
13%
Tickets
or
actions
4.8
13.8
3.8
5.4
Design New Mechanisms
• With one agent, a Scoring Rule is a good
mechanism to elicit beliefs.
– Report r. Receive S*ln(r/2) in asset that pays $1 if A
and S*ln((1-r)/2) in asset that pays $1 if B.
• Incentive compatible to report true beliefs
• Encourages participation
– Provides an expected value greater than 0.
– Brier (1950) Monthly Weather Review, Goode (1952),
Savage (1971) JASA, Page (1988)(PSR  VCG)
• Let’s adapt this to multi-agent situations.
If it Works for One Person……
• Market Scoring Rules may be good mechanisms.
– Hanson (2003)
– An automated market maker (MM)
• Any trader can access the MM, at any time and trade assets
according to a scoring rule by announcing his belief.
• The scoring rule is seeded with the last trader’s announcement
of their beliefs.
• Each new announced belief is publicly reported.
– Incentive compatible to report true beliefs (somewhat)
• At one iteration, but not over all iterations.
• Possible to mislead early and gain later (particularly if N =3)
– Encourages participation (somewhat)
• Positive expected value to the group.
An Old Standby Dressed Up
• A Poll with Incentives may be a good mechanism.
– Basic design is standard.
• All are polled and asked their beliefs.
• Beliefs are averaged and publicly announced.
• Repeated m times (We did 5 and 3.)
– But at the end, each agent is paid according to the same
scoring rule.
– Incentive compatible to report true beliefs (somewhat)
• True at “equilibrium” if there is full-information aggregation
• Open question whether it is true during iterations
– Encourages participation
• Positive expected value to everyone in the group.
MS, MSR, P, PMS (N = 3)
1
0 .9
0 .8
Probability
0 .7
Pretty much the same.
0 .6
0 .5
ave KL
PMS .042
MSR .045
P
.047
MS .051
0 .4
0 .3
0 .2
0 .1
P M /S3
M SR3
M /S 3
P oll3
kl(.8,.65)=.06
kl(.8,.50)=.22
kl(.8,.20)=.83
0
0 .0 0
0 .1 0
0 .2 0
KL distance
0 .3 0
0 .4 0
MS, MSR, P, PMS (N = 7,8,12)
1
0 .9
0 .8
P and MRS get it (always)
>> MS and PMS
Probability
0 .7
0 .6
ave KL Mirage
MSR .000
0
P
.004
0
PM .013
0
PMS .061
0
MS
.085
2
M
.100
2
0 .5
0 .4
0 .3
0 .2
0 .1
P M /S
M SR
M /S
poll
kl(.8,.65)=.06
kl(.8,.50)=.22
kl(.8,.20)=.83
0
0
0 .1
0 .2
KL distance
0 .3
0 .4
Summary Statistics
32 observations (all N)
P
MSR
MS
PMS
Got it
53%
75%
66%
31%
% Early
22%
56%
47%
3%
Mirages
6%
6%
13%
13%
KL dist.
.025
.023
.068
.051
• In MSR, first mover wins. In Pari-mutuel, last mover wins.
• In time lapsed, MSR hits it really fast.
Reprise:
Do traditional IM’s work?
• YES: Large numbers of informationally small
agents and reasonably simple environments seem
to overcome some of the no-trade incentives in
traditional markets and pari-mutuels.
– I don’t think we yet know exactly why. (Open Puzzle!)
• NO: With informationally large agents, much less
incomplete markets or complex decision
problems, it does not look so good.
– N = 3 creates serious problems for both markets and
pari-mutuels, even in simple environments.
– More research is needed here.
Reprise:
Can we design IM’s that work?
• YES: For informationally small agents in simple
environments,
– The Market Scoring Rule gets it early and often.
– Polling with incentive payments also works pretty well
but is slower to get the job done.
– Markets are mirage prone, even with noise traders.
– Pari-mutuels are slow, even with subsidization
• MAYBE: For informationally large agents,
– The four best (ms, msr, p, pms) are all off by.10 to .15.
– All mechanisms we looked at can be improved upon.
– Or is there an impossibility theorem? (Open Puzzle!)
The End
• I leave you with two questions that I believe
are worth pursuing.
Is
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Market Maker
Is
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
MSR
=
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
?
Maxwell’s Demon
the best mechanism?