Transcript ELSE Seminar

The influence of hierarchy on
probability judgment
David A. Lagnado
David R. Shanks
University College London
Level of hierarchy can modulate
judgment

Consider two statements about the next World Cup
 It is most likely that Brazil will win
 It is most likely that a European team will win

These appear to support opposing predictions, but
both may be true

Shows the importance of the level at which
probabilistic information is represented
Hierarchical structure

Pervasive feature of how we represent the world
Reflects pre-existing physical and social hierarchies
 Readily generated through conceptual combination


Category hierarchies serve both to organize our
knowledge, and to structure our inferences
Inference using a hierarchy
Tabloid
Sun


Mirror
Broadsheet
Guardian
Times
One powerful feature of a category hierarchy is that
given information about categories at one level, you can
make inferences about categories at another level.
This allows you to exclude alternatives, or reduce the
number you need to consider
Probabilistic Inference using a hierarchy
Tabloid
Sun


Mirror
Broadsheet
Guardian
Times
In many real-world situations we must base our initial
category judgments on imperfect cues, degraded stimuli,
or statistical data.
What effect do such probabilistic contexts have on the
hierarchical inferences that we are licensed to make?
Commitment heuristic
Tabloid
Sun


Mirror
Broadsheet
Guardian
Times
Commitment heuristic - When people select the most
probable category at the superordinate level, they assume
that it contains the most probable subordinate category.
This leads to the neglect of subordinates from the less
probable superordinate.
How adaptive is this heuristic?



The efficacy of such a heuristic depends on the
precise structure of the environment.
In certain environments it confers considerable
advantages
 increases inferential power by focus on
appropriate subcategories
 reduces computational demands by avoiding
complex Bayesian calculations.
But in some environments it can lead to anomalous
judgments and inferences.
Non-aligned hierarchy
Tabloid 60
Sun 30


Mirror 30
Broadsheet 40
Guardian 35
Times 5
In the above sample the most frequently read type of
paper is a Tabloid, but the most frequently read paper is
a Broadsheet (the Guardian).
Non-aligned hierarchy: the most probable superordinate
category does not contain the most probable
subordinate category.
Real world examples

Word frequencies: the superordinate BE- is more
frequent than BU-, but the subordinate BUT is
more frequent than any of the other subordinates
(BET, BED…etc.)

NHS statistics on survival rate for operations for
different areas & sub-areas

You are more likely to survive a hip operation in Surrey
rather than Essex, but the best sub-area for survival is
Colchester (in Essex).
Experiments 1 and 2

Learning phase - participants exposed to a non-aligned
hierarchical environment in which they learn to predict
voting behavior from newspaper readership.

100 trials ‘reading/voting profiles’
Screen during learning phase
Tabloid
Chronicle
Broadsheet
Herald
○
Reporter
Liberal
○ Progressive
Globe
Screen during learning phase
Reading profile for J. K.
Tabloid
Chronicle
Broadsheet
Herald
○
Reporter
Liberal
○ Progressive
Globe
Screen during learning phase
Reading profile for J. K.
Tabloid
Chronicle
Broadsheet
Herald
○
Reporter
Liberal
Outcome feedback
○ Progressive
Globe
Structure of environment
Tabloid 60
Sun 30
Mirror 30
Party A
50
Broadsheet 40
Guardian 35
Times 5
Party B
50
Judgment phase
Baseline
X is selected at
random
Type
Paper
Which type of
paper is X most
likely to read?
Which paper is
X most likely to
read?
What is the probability that
X votes for one party rather
than the other?
Results of Experiment 1
100
baseline
type
paper
Mean probability rating for Party B
90
80
70
60
50
40
30
20
10
0
aligned

non-aligned
Probability ratings for Party B rather than Party A with
judgments divided into those based on aligned and nonaligned choices
Experiment 2

Replication of Experiment 1, with frequency as
well as probability response formats

Frequentist hypothesis that probability biases
reduced with frequency format
Results of Experiment 2
100
baseline
type
paper
90
Mean rating for Party B
80
70
60
50
40
30
20
10
0
aligned

non-aligned
Mean ratings for Party B rather than Party A collapsed across
probability and frequency ratings
Summary of Results

Participants allow their initial probability judgment
about category membership (newspaper readership)
to shift their rating of the probability of a related
outcome (voting preference), even though all
judgments are made on the basis of the same
statistical data.

When their prior choices were non-aligned this led
to a switch in predictions about the outcome
category
Conclusions

These biases are explicable by the Commitment
heuristic:


The priming question commits people to just one
inferential path, leading them to compute an
erroneous estimate for the final probability.
This is understandable given the complexity
of the normative Bayesian computation.
Comparison of Bayesian and commitment
heuristic computations (just type level inference)
Type of paper?
0.6
Tabloid
0.77
Party A
Type of paper?
0.4
0.6
Broadsheet
0.23 0.1
Tabloid
0.9
Party B
P(A) = (0.6 . 0.77) + (0.4 . 0.1)
= 0.46 + 0.04
= 0.5
Bayesian computation
0.77
Party A
P(A) = 0.77
Simplified heuristic computation
Conclusions

Simplifying heuristic that assumes that
environment is aligned

Empowers inference when hierarchical structure
is aligned, otherwise can lead to error

Suggests tendency to reason as if a probable
conclusion is true
Process level accounts

Associative model
 People learn predictive relations between category
options (at both levels of hierarchy) and outcome. At
test responses to category questions prime the
appropriate associations and lead to a biased rating of
the outcome.

Frequency-based model
 People encode event frequencies in the learning phase.
At test responses to the category question serves as the
reference class for subsequent conditional probability
judgments about voting preferences.
Implications

Importance of the level at which probabilistic data is
represented to (or by) a decision maker
 E.g., using NHS statistics to decide on hospital

How do people search through hierarchical statistical
data?

People’s judgments can be manipulated by the level at
which statistical information is represented

More generally, in multi-step inferences people are
susceptible to biased probability judgments