Transcript 9 – Judgment

9 – Judgment
Most people do not understand chance and risk, often with dire consequences.
These errors reflect misconceptions about probability.
These misconceptions are revealed by people’s response to the following questions:
The Banker Question
Linda is 31, single, outspoken, and highly educated.
Which one of the following statements is more likely to be true?
 She is a banker.
 She is a banker and a democrat.
Banker.
chance of just one  chance of both
conjunction fallacy
chance of both  chance of just one
Correct Answer
(Tversky & Kahneman, 1983)
Hospital Question
At Big City Hospital, about 50 babies are born daily.
At Small Town Hospital, about 10 babies are born daily.
At each hospital, someone counts the number of girls and boys born each day.
If at least 60% of babies born that day are girls, a pink sticker is placed on the calendar.
In a typical year, which hospital will have more pink stickers?
● Big City Hospital
● Small Town Hospital
● About the same
Modal Answer
About the same
Correct Answer
Small Town Hospital
Explanation
smaller sample  less accurate survey
Example
# of births
Probability of at least 60% girls
10
38%
50
10%
(Kahneman & Tversky, 1972)
Coin Question
A fair coin turns up Heads 5 times in a row.
Which is more likely to happen on the next toss?
● Heads
● Tails
● Neither – chance is 50-50
Correct Answer
Neither. A coin has no memory!
Common Error
Tails
gambler’s fallacy event is less likely if it just occurred
More gambler’s fallacy
When I was a kid, my best friend was 1 of 3 kids in the family.
All 3 kids were boys.
Then their mom got pregnant.
The mom said, “Since I have 3 boys, I’ll probably have a girl. ” (gambler’s fallacy)
Truth = 50%
(Though she had a girl.)
blank
Demo: Imagine rolling a fair die 25 times. Write the outcomes below (1, 2, 3, 4, 5, or 6).
___
___
___
___
___
___
4
___
___
4 
___
___
6
___
___
3
___
___
3 
___
___
3 
___
___
2
___
___
___
Step 2 Write check by each outcome that matches previous outcome
On average, there should be
4 repetitions
Typical subject
< 4 repeitions
___
___
___
___
Why?
Gambler’s Fallacy
More Gambler’s Fallacy
When teachers write multiple-choice tests, repeats are too rare.
More gambler’s fallacy
May 2012
My insurance company phone rep tells me that I need more hurricane coverage.
I point out that hurricane hasn’t hit Florida since 2005.
She says that’s why I need more coverage – the chances are getting higher!
By her logic, Iowa is really in danger.
Note: Cost of insurance coverage has climbed every year since 2005.
Weather Facts
No hurricane has hit Tampa since they started keeping records in 1851
en.wikipedia.org/wiki/List_of_Florida_hurricanes
Data show Florida hurricanes come in clusters.
Example. 4 in 2004 and 3 in 2005
That is, chance of hurricane is LESS if it’s been a long time since last one.
Hot Hand
Fact: In basketball, players often have streaks.
Example. A 50% shooter might hit 4 in a row or she might miss 4 in a row.
Widespread Belief: Chance of hit after a hit > Chance of hit after a miss
False! Basketball statistics 
Chance of hit after a hit = Chance of hit after a miss
hot hand fallacy = success is more likely after a success than after a failure
This fallacy occurs commonly in games of skill and sometimes in games of luck.
(Gilovich, Vallone, & Tversky, 1985)
Data from 9 players from Philadelphia 76ers
Data show that chance of hit is NOT greater after a hit
(Gilovich, Vallone & Tversky, 1985)
blank
Poetry Lover Question
Tom is a randomly selected U.S. citizen.
He is asked if he likes poetry. He answers “yes.”
Which is more likely to be his profession?
● Trucker
● Latin Professor
Correct Answer
Trucker
Explanation
Even if only 0.3% of truckers like poetry, they exceed 1000.
Yet fewer than 1000 people are Latin Professors.
Thus, answer depends partly on base rate (percent of population with trait X)
Wrong answer is example of base rate neglect
Disease Question
On average, 1 in 191 people has Disease X.
There is a blood test for X.
Of people who have X, 100% test positive. (True Positive rate = 100%)
Of people who don’t have X, 10% test positive. (False Positive rate = 10%)
During a routine physical, Homer tests positive for X.
Intuitively, what is the chance that Homer has X?
● less than 10%
● about 50%
● higher than 80%
Correct Answer
5%
Explanation
base rate
TP
FP
Disease X
1 in 191
100%
10%
5%
Disease Y
1 in 11
100%
10%
50%
Disease X
191
Chance that Homer has X
1 has X
1 TP
190 do not
19 FPs
1 / 20 = 5%
Disease Y
11
1 has X
1 TP
10 do not
1 FP
1 / 2 = 50%
Examples
blood test
base rate
TP
FP
Disease Q
1 in 91
100%
10%
Disease R
21 in 91
100%
10%
Disease S
1 in 11
100%
0%
1. Quinn tests positive for Q, and Rick tests positive for R.
Which event is less likely:
Quinn has Q
or
Rich has R
(smaller base rate)
2. Reece has R. Find the chance that he will test positive for R.
100%. True positive rate = chance that person with R will test positive.
3. Stan tests positive for S. What is chance that Stan has S?
100%. The test is perfect.
Study
Harvard Med students and faculty given Disease Question (with slightly different data).
Results
Answer
Percent of Subjects
2% (correct)
18%
95%
45%
Troubling Implication
Patient tests positive for X
Doctor tells patient his chance of having X is 95% but truth is 2%
Patient chooses risky surgery and dies.
(Casscells, Schoenberger, & Graboys, 1978, NEJM)
Base rate neglect in academia …
What is the chance that a manuscript will be accepted?
Academics rely heavily on their judgment of manuscript’s quality.
They tend to ignore the journal’s acceptance rate (i.e., base rate).
Journal acceptance rates vary dramatically.
For APA journals, acceptance rate is between 9% and 69% (2009 data)
For all psych journals, range is even greater.
blank
Conditional Probabilities
Disease question requires understanding of conditional probabilities.
conditional probability = Chance that A is true if B is true
In Disease X question,
Chance that Homer has disease X if he tested positive)
=
Chance that Homer will test positive if he has disease X)
= 100%
Conditional probabilities are hard to understand.
(e.g., Cosmides & Tooby, 1996; Gigerenzer, 2002)
5%
Example
In 1995, celebrity O. J. Simpson was put on trial for murdering his ex-wife.
Simpson had previously battered her.
His attorney: “Only 0.04% of male batterers later murder the battered woman.”
Translation: If Mr. X once battered Ms. Y, the chance that X will murder Y = 0.04%
Relevant data:
If Mr. X once battered Ms. Y, the chance that X is murderer if Y was murdered = 93%
Neither statistic was presented to jury.
(I. J. Good, Nature, 1995, 1996; Strogatz, New York Times, 2010)
People better understand risk if they are presented with frequency data.
Study
Doctors given two versions of the same question
Conditional Probability Version
The probability that a woman has breast cancer is 1%.
If she has breast cancer, the probability of a positive mammogram is 80%.
If a woman does not have breast cancer, the probability of a positive result is 10%.
Take, for example, a woman who has a positive result.
What is the probability that she actually has breast cancer?
Natural Frequency Version
10 out of every 1000 women have breast cancer.
Of the 10 women with breast cancer, 8 will have a positive result on mammography.
Of the 990 women who do not have breast cancer, 99 will still have a positive result.
Take, for example, a sample of women who have positive mammograms.
How many of these women actually have breast cancer?
Results
(Hoffrage & Gigerenzer, 1998)
Hungry Sailors
10 starving men in a lifeboat agree to sacrifice one man so that the others will live.
They put 9 green marbles and 1 red marble in a bag.
One at a time, each man chooses a marble.
After each marble is chosen, the marble is NOT placed back in the bag.
The first player to choose red is the loser.
Is it better to choose first or tenth, or does it not matter?
Answer: It does not matter
Hungry Sailors Version 2
Same as above, except each man replaces his marble before next man chooses.
Is it better to choose first or tenth, or does it not matter?
Answer: Tenth
The whale ship Essex was sunk by a whale on November 20, 1820. Twenty men
escaped in three small craft and wandered the Pacific. After being unable to catch
fish, they ultimately resorted to cannibalism by common consent. Three months
later there were two boats and five men left. Rescuers found bug-eyed stick figures
hunkered over a pile of human ribs, with finger bones stashed in their pockets.
(Mark Shone, 2000)
A similar scenario took place in Edgar Allen Poe novel
The Narrative of Arthur Gordon Pym of Nantucket
blank
Two Girls Question
A woman moves into the neighborhood.
When you meet her, you find out that her name is Beth and that she has two kids.
Later, you see one of Beth’s kids playing in her yard, and the kid is a girl.
What is the chance that Beth’s other child is also a girl?
Answer 1/3
Explanation
Random sample of 400 women with exactly 2 kids.
first born
second born
~ N_
B
B
100
B
G
100
G
B
100
G
G
100
1/3 of these have 2 girls
The Birthday Question
A group of 41 people is randomly selected from the general population.
What is the chance that at least two of them will have the same birthday?
10%
Answer
30%
50%
70%
90%
Explanation
The question is: does any person share a birthday with any other person.
41 people  820 different pairs.
(e.g., Voracek et al., 2008)
90%
# of people
23
32
41
50
366
# of possible matches
253
496
820
1225
66795
Probability of a match
50%
75%
90%
97%
100%
n
n(n-1)/2
-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - # of people
P(match) = 1 – P(no match)
2
1 – (364/365)
person 2
= .003
3
1 – (364/365)(363/365)
person 2 person 3
= .008
Calculation assumes
1) all birthdates are equally common (not quite true)
2) no Feb 29
Correcting for these assumptions has no discernible effect on answer.
Monty Hall Dilemma
A game show contestant is shown 3 curtains.
One curtain has a prize behind it. (Stagehands randomly choose prize location.)
The emcee knows which curtain has the prize behind it.
Player chooses a curtain.
Emcee then opens a non-chosen losing curtain.
Player is then allowed to stay with original choice or switch.
Should player switch or stay, or does it not matter?
Modal Response
It doesn’t matter.
Correct Answer
Switch  2/3 chance of winning
10-Curtain Version of Monty Hall Dilemma
1 of 10 curtains has a prize behind it. (Stagehands randomly choose prize location.)
The emcee knows which curtain has the prize behind it.
Player chooses curtain.
Emcee then opens 8 non-chosen losing curtains.
Player is then allowed to stay or switch.
Should player switch or stay, or does it not matter?
Chance of winning if player switches = 9/10
The End
Bankers
Banker-Democrats
Banker.
chance of just one  chance of both
conjunction fallacy
chance of both  chance of just one
Correct Answer
(Tversky & Kahneman, 1983)
Disease Question
On average, 1 in 91 people has Disease X.
There is a blood test for X.
Of people who have X, 100% test positive. (TP = True Positive rate = 100%)
Of people who don’t have X, 10% test positive. (FP = False Positive rate = 10%)
During a routine physical, Homer tests positive for X.
What is the chance that Homer has X?
10%
Correct Answer
10%
Modal Answer
90%
30%
50%
70%
base rate neglect
(continued)
90%
Explanation
base rate
TP
FP
Chance that Homer has X
Disease X
1 in 91
100%
10%
10%
Disease Y
21 in 91
100%
10%
75%
91
1 has X
1 TP
90 do not
9 FPs
Disease X
1 / 10 = 10%
Disease Y
91
21 have X
21 TPs
70 do not
7 FPs
21 / 28 = 75%
Real Medical Study
Success rate for two kidney stone treatments
Treatment A
All Patients
Treatment B
78% (273/350)
83% (289/350)
Small Stones
93% (81/87)
87% (234/270)
Large Stones
73% (192/263)
69% (55/80)
How can this be?
Patients were NOT randomly assigned.
Instead, doctors usually chose A for large stones, which are harder to treat.
Simpson’s Paradox trend reverses when samples combined
(Simpson, 1951)
More Simpson’s Paradox - UC Berkeley gender bias
UC Berkeley sued by because of these 1973 admission data
Men
Women
Applicants
% admitted
8442
44%
4321
35%
Then data were partitioned by department.
Department
Men
Women
Applicants
% admitted Applicants
% admitted
A
825
62%
108
82%
B
560
63%
25
68%
C
325
37%
593
34%
D
417
33%
375
35%
E
191
28%
393
24%
F
272
6%
341
7%
Question
Scientists compute p in order to test the null hypothesis (H0).
What does p equal?
P(data given H0 is true)
correct
OR
P(H0 is true given data)
common belief
(if true, p would be useful)
The Disease X question is formally answered by using:
Bayes’ Theorem
where P(A | B) = Probability of A given that B has happened
and P(B | A) = Probability of B given that A has happened
(Bayes, 1763)
Solution to Disease X question:
Let A = disease X is present
P(A)
P(B | A)
P(B | not A)
and
let B = positive test
= 1/91, which means that P(not A) = 90/91
= P(positive test given presence of X) = TP = 100% = 1
= P (positive test given absence of X ) = FP = 10% = 0.10
P(X | positive test) = P(B | A) = (1)(1/91) / [(1)(1/91) + (0.1)(90/91)] = 0.1
Based on studies described thus far today, few people understand chance.
However…
others argue that these studies are misleading because:
The questions require algorithms, but people use “fast-and-frugal” heuristics.
The questions were contrived to exploit fallibility of heuristics.
The heuristics work well in everyday life.
(e.g., Gigerenzer, 2002)
(e.g., Gigerenzer, 2002)
Two versions of mammogram data for women over 50
Relative Risk version
If woman has mammogram, her chance of cancer death in next 10 years drops 25%
Frequency version
If 1000 women skip mammogram, 4 will die of breast cancer in next decade.
If all 1000 have mammogram, the number who die will decrease from 4 to 3.
Frequency Data are easier to understand and more informative.
(e.g., Gigerenzer & Edwards, 2003)
Hungry Sailors
Six men in a lifeboat are starving, so they agree that one must be sacrificed.
They put 5 green marbles and 1 red marble in a bag.
One at a time, each man chooses a marble.
The first player to choose red is the loser.
After each marble is chosen, the marble is NOT placed back in the bag.
Is it better to choose first or sixth, or does it not matter?
Going first has obvious disadvantage: You’re the only player who must choose!
But going first has an advantage: Only 1 in 6 is red. For others, 1 in 5, 1 in 4, …
Answer: It does not matter
Optional
P(Man A gets red) = 1/6
P(Man B gets red) = P (A doesn’t choose red AND B does) = (5/6)(1/5) = 1/6
Hungry Sailors Version 2
Six men in a lifeboat are starving, so they agree that one must be sacrificed.
They put 5 green marbles and 1 red marble in a bag.
One at a time, each man chooses a marble.
The first player to choose red is the loser.
After each marble is chosen, the marble is placed back in the bag.
Is it better to choose first or sixth, or does it not matter?
Answer: Sixth
Optional
P(Man A gets red) = 1/6 = 6/36
P(Man B gets red) = P(A gets green AND B gets red)= (5/6)(1/6) = 5/36