Transcript Chapter10slides

Chapter 10
Methods for
Eliciting Probabilities
1
Preparing for probability assessment

Motivating

Structuring

Conditioning
2
Motivating


This phase is designed to introduce the
decision maker to the task of assessing
probabilities and to explain the importance
and purpose of the task.
Sensitivity analysis should be used by the
analyst to identify those probabilities which
need to be assessed with precision.
3
Structuring



In the structuring phase the quantity to be
assessed should be clearly defined and structured.
It is also important at this stage to agree on a
scale of measurement which the decision maker
feels comfortable with.
When the decision maker thinks that the quantity
to be assessed depends on other factors it may
be simpler to restructure the assessment task,
possibly by making use of a probability tree.
4
Conditioning



The objective of this phase is to identify and
thereby avoid the biases which might otherwise
distort the decision maker's probability assessments.
It involves an exploration of how the decision maker
approaches the task of judging probabilities.
For example, are last year's sales figures being used
as a basis for this year's estimates? If they are,
there may be an anchoring effect.
5
Assessment methods


Assessment methods for individual
probabilities
Assessment methods for probability
distributions
6
Direct assessment methods

Posing a direct question
e.g. ‘What is the probability that the product
will achieve break-even sales next
month?’
7


Unfortunately, many people would feel
uncomfortable with this sort of approach,
and they might be tempted to give a
response without sufficient thought.
Asking the individual to mark a point on a
scale which runs from 0 to 1 might be
preferred because at least the scale enables
the probability to be envisaged.
8
The probability wheel

A probability wheel is a device like that
shown in Figure 10.1, and it consists of a
disk with two different colored sectors,
whose size can be adjusted, and a fixed
pointer.
9
The probability wheel
P o in t e r
10


Bet One: If the rival launches the product
within the next week you will win $100000.
If the rival does not launch the product you
will win nothing.
Bet Two: If, after spinning the wheel once,
the pointer is in the white sector you will
win $100 000. If it is pointing toward the
black sector you will win nothing.
11



If the manager says that she would choose Bet Two
then this implies that she thinks that the probability
of the rival launching the product in the next week
is less than 80%.
The size of the white sector could then be reduced
and the question posed again.
Eventually, the manager should reach a point where
she is indifferent between the two bets. If this is
achieved when the white sector takes up 30% of
the wheel's area, this clearly implies that she
estimates that the required probability is 0.3.
12


The wheel has the advantage that it enables the
decision maker to visualize the chance of an
event occurring.
However, because it is difficult to differentiate
between the sizes of small sectors, the probability
wheel is not recommended for the assessment of
events which have either a very low or very high
probability of occurrence.
13


The analyst should also ensure that the rewards of
the two bets are regarded as being equivalent by
the decision maker.
A number of devices similar to the probability
wheel have also been used in probability
assessment. For example, the decision maker may
be asked to imagine an urn filled with 1000 colored
balls (400 red and 600 blue).
14
Assessment methods for
probability distributions



The probability method
There is evidence that, when assessing
probability distributions, individuals tend to
be overconfident, so that they quote too
narrow a range within which they think the
uncertain quantity will lie.
The following procedure is recommended.
15




Step 1: Establish the range of values within which
the decision maker thinks that the uncertain
quantity will lie.
Step 2: Ask the decision maker to imagine
scenarios that could lead to the true value lying
outside the range.
Step 3: Revise the range in the light of the
responses in Step 2.
Step 4: Divide the range into six or seven roughly
equal intervals.
16

Step 5: Ask the decision maker for the cumulative
probability at each interval. This can either be a
cumulative 'less than' distribution (e.g. what is the
probability that the uncertain quantity will fall
below each of these values?) or a cumulative
'greater than' (e.g. what is the probability that the
uncertain quantity will exceed each of these
values?), depending on which approach is easiest
for the decision maker.
17
Step 6: Fit a curve, by hand, through the assessed points.
 Step 7: Carry out checks as follows.
(i) Split the possible range into three equally likely intervals
and find out if the decision maker would be equally happy
to place a bet on the uncertain quantity falling in each
interval. If he is not, then make appropriate revisions to the
distribution.
(ii) Check the modality of the elicited distribution (a mode is a
value where the probability distribution has a peak). For
example, if the elicited probability distribution has a single
mode (this can usually be recognized by examining the
cumulative curve and seeing if it has a single inflection), ask
the decision maker if he does have a single best guess as to
the value the uncertain quantity will assume. Again revise
18
the distribution, if necessary.

Assessment methods for probability
distributions

The probability method

Graph drawing…
19
Graph drawing


Graphs can be used in a number of ways to
elicit probability distributions.
The method of relative heights is one wellknown graphical technique that is designed
to elicit a probability density function.
20


First, the decision maker is asked to identify
the most likely value of the variable under
consideration and a vertical line is drawn on
a graph to represent this likelihood.
Shorter lines are then drawn for other
possible values to show how their
likelihoods compare with that of the most
likely value.
21
The method of relative heights
22


To convert the line lengths to probabilities
they need to be normalized so that they
sum to one.
This can be achieved by dividing the length
of each line by the sum of the line lengths,
which is 36, as shown below
23
24
Assessing the validity of subjective
probabilities
25


Consistency checks are, of course, a
crucial element of probability assessment.
The use of different assessment methods
will often reveal inconsistencies that can
then be fed back to the decision maker.
26


If probability estimates derived by
different methods for the same event are
inconsistent, which method should be
taken as the true index of degree of belief?
One way to answer this question is to use
a single method of assessing subjective
probability that is most consistent with
itself. (+)
27


In other words, there should be high
agreement between the subjective
probabilities, assessed at different times by a
single assessor for the same event, given
that the assessor's knowledge of the event is
unchanged.
It was concluded that most of the subjects in
all experiments were very consistent when
using a single assessment method.
28


One useful coherence check is to elicit from the
decision maker not only the probability that an
event will occur but also the probability that it will
not occur. The two probabilities should, of course,
sum to one.
If the events are seen by the probability assessor
as mutually exclusive then the addition rule can
be applied to evaluate the coherence of the
assessments.
29

A major measure of the validity of
subjective probability forecasts is known as
calibration.

By calibration we mean the extent to which
the assessed probability is equivalent to
proportion correct over a number of
assessments of equal probability.
30
Assessing probabilities for very
rare events


Assessment techniques that differ from
those we have so far discussed are
generally required when probabilities for
very rare events have to be assessed.
Because of the rarity of such events, there
is usually little or no reliable past data
which can support a relative frequency
approach to the probability assessment.
31


Decision makers are also likely to have
problems in conceiving the magnitudes
involved in the probability assessment.
It is difficult to distinguish between
probabilities such as 0.0001 and 0.000001,
yet the first probability is a hundred times
greater than the second.
32



Event trees and fault trees allow the problem to
be decomposed so that the combinations of
factors which may cause the rare event to occur
can be identified.
Each of the individual factors may have a
relatively high (and therefore more easily
assessed) probability of occurrence.
A log-odds scale allows the individual to
discriminate more clearly between very low
probabilities.
33


Figure 10.4 shows a simplified tree for a
catastrophic failure at an industrial plant.
Each stage of the tree represents a factor
which might, in combination with others,
lead to the catastrophe.
By using the multiplication and addition
rules of probability, the overall probability
of failure can be calculated.
34
An event tree
35
Fault trees

In contrast to event trees, fault trees start
with the failure or accident and then depict
the possible causes of that failure.
36
A fault tree
37
Using a log-odds scale
38