#### Transcript Economic Evaluation using Decision Analytic Modelling (DAMn!)

```Economic Evaluation
using Decision
Analytic Modelling I
Mira Johri
Université de Montréal
2008-10-27
Outline (Part 1)
Overview
1.
Basic features of a decision model
Rationale for modelling


Decision trees
2.




2008-10-27
Conventions
Steps to perform a decision analysis
Blindness prevention example
Strengths & weaknesses
2
I. Overview
What is a model?

A health care evaluation model is a logical
mathematical framework that permits the
integration of facts and values and that links
these data to outcomes that are of interest to
health care decision makers.

Its purpose is to structure evidence on clinical
and economic outcomes to help inform
decisions about clinical practice and health-care
resource allocation under conditions of
uncertainty.
2008-10-27
4
What is a model? Cont’d

In the context of economic evaluation, a decision
analytic model uses mathematical relationships
to define a series of possible consequences
(health and economic outcomes of patients or
populations) that would flow from the alternative
options being evaluated.

It enables us to structure and examine a
complex problem.
2008-10-27
5
Example

Should we vaccinate children against
meningitis?


To vaccinate or not to vaccinate?
It depends:




2008-10-27
Risk of exposure to bacillus
Risk of contracting meningitis
Efficacy of the vaccine
Risks associated with the vaccine
6
Meningitis
Vaccinate
Choice
Do not vaccinate
2008-10-27
7
Meningitis
Exposed
Vaccinate
Not exposed
Choice
Exposed
Do not vaccina te
Not exposed
2008-10-27
8
Meningitis
Meningitis
Exposed
No m eningitis
Vaccinate
Not exposed
Choice
Meningitis
Exposed
No m eningitis
Do not vaccinate
Not exposed
2008-10-27
9
Quantifying probabilities &
outcomes

Based on the inputs of
the model, the
likelihood of each
consequence (health
and economic
outcomes of patients or
populations) is
expressed in terms of
probabilities, and each
consequence has a
cost and an outcome.
2008-10-27




Risk of exposure to
bacillus (0,2)
Risk of contracting
meningitis if exposed
(0,1)
Efficacy of the vaccine
(risk of meningitis if
vaccinated 0,01)
Risks associated with
the vaccine (0,003)
10
0
,5
Meningitis
Vaccinate
,01
Exposed
No com plications
,5
Choice
,2
No m eningitis
,99
Not exposed
Meningitis
,2
Do not vaccinate
,9
Not exposed
,8
2008-10-27
,4
Alive
,6
No m eningitis
1
0
0
0
,1
Alive
,6
,8
Exposed
,4
1
0
0
0
11
Method of analysis



Calculate the expected cost and expected outcome
of each option under evaluation.
For a given option, the expected cost (outcome) is
the sum of the costs (outcomes) of each
consequence weighted by the probability of that
consequence.
Objective: to maximise (minimise) the expected
value of positive (negative) outcomes
o
2008-10-27
In CEA, the analyses are done separately for costs and
health outcomes
12
Meningitis Vaccination
1
0
,5
Meningitis
Vaccinate
,01
Exposed
No com plications
,5
Choice
,2
,4
Alive
,6
No m eningitis
,99
Not exposed
Meningitis
,1
,2
Do not vaccinate
0
0
0
,8
Exposed
1
,4
Alive
,6
No m eningitis
,9
Not exposed
,8
1
0
0
0
#deaths with vaccination = 0,003 + (0,997 x 0,2 x 0,01 x 0,4)
= 3,8 per 1 000 children
#deaths without vaccination = 0,2 x 0,1 x 0,4
= 8,0 per 1 000 children
2008-10-27
13
Meningitis Vaccination
1
0
,5
Meningitis
Vaccinate
,01
Exposed
No com plications
,5
Choice
,2
,4
Alive
,6
No m eningitis
,99
Not exposed
Meningitis
,1
,2
Do not vaccinate
0
0
0
,8
Exposed
1
,4
Alive
,6
No m eningitis
,9
Not exposed
,8
1
0
0
0
#deaths with vaccination = 0,003 + (0,997 x 0,2 x 0,01 x 0,4)
= 3,8 per 1 000 children
#deaths without vaccination = 0,2 x 0,1 x 0,4
= 8,0 per 1 000 children
2008-10-27
14
Rationale: Synthesis

Economic evaluation studies must use all relevant evidence
(principle of EBM).

All evidence rarely comes from a single source.

A decision-model provides a framework in which a range of
evidence can by synthesised and brought to bear on the decision
problem.
 Characterise natural history of a given condition
 Impact of alternative interventions
 Costs and health effects contingent on clinical events
 Relationship between intermediate clinical measure of
effect and ultimate measure of health gain required for
CEA

2008-10-27
(Adapted from Briggs et al., 2006)
15
Rationale:
Consideration of all relevant
comparators

Cost-effectiveness must be established in
comparison with all alternative options that could
feasibly be used in practice.

A single study or RCT would rarely consider the full
range of options.

Data must be brought together from clinical studies
using appropriate statistical synthesis methods.

The decision model provides the framework to bring
the synthesis to bear on the decision problem.

2008-10-27
(Adapted from Briggs et al., 2006)
16
Rationale:
Appropriate time horizon

CEA requires that studies adopt a time horizon that
is sufficiently long to reflect all key differences
between options in terms of costs and effects.

For many interventions, this will effectively require a

Decision models become the framework within
which to structure the extrapolation of costs and
effects over time.

2008-10-27
(Adapted from Briggs et al., 2006)
17
Rationale: Uncertainty

CEA must indicate how uncertainty in the
available evidence relating to a given policy
problem translates into decision uncertainty;
that is, the probability that a given decision is
the correct one.

Through various forms of sensitivity analysis,
decision models can help to characterise this
uncertainty and present it to decision makers.

2008-10-27
(Adapted from Briggs et al., 2006)
18
II. Decision Trees
Decision Trees - Conventions

Events are ordered from left to right.


Temporal order is often followed for sake of clarity.
Different kinds of events are distinguished using
shapes called “nodes”



2008-10-27
Square – a decision node indicating a choice facing the
decision maker, typically at start of tree
Circle – a chance node represents an event which has
multiple possible outcomes and is not under the decision
maker’s control. For an individual patient, which event they
experience is uncertain.
Triangle – a terminal node denotes the endpoint of a
scenario.
20
Decision Trees - Conventions

Branches “sprouting” from a decision node represent the set of
actions being considered (strategies)
 They need not be mutually exclusive (e.g. A, B, A+B)

Branches from a chance node represent the set of possible
outcomes of the event
 Must be mutually exclusive and exhaustive
 Probabilities must sum to 1.0

Terminal nodes are assigned a value – referred to generically as
a payoff




Cost
Utility
QALY
Expected values are based on the summation of the pathway
values weighted by the pathway probabilities.
2008-10-27
21
Steps in Decision Analysis
1.
2.
3.
4.
5.
6.
Define the problem
Structure the decision and make a tree
Fill in the data (probabilities and outcomes)
Choose the consequence with the
maximum expected value (“roll back the
tree”)
Perform sensitivity analysis
Interpret results
2008-10-27
22
1. Blindness prevention

Hypothetical population presenting clinical signs of a
possible, early-stage autoimmune disorder.

If the condition is present, and if it progresses,
blindness will result.

An imperfect test (biopsy, with the possibility of false
negatives) can help determine whether an individual
has the disorder

There exists an effective and inexpensive therapy
which lowers probability of progression to blindness.

Side effects of treatment are factored into the costs
and life expectancy at the end of each path.
2008-10-27
23
2. The Data





Disease prevalence =
0,5
pBlind (without
Tx)=0,12
pBlindTreat = 0,013
TestSens = 0,8
TestSpec = 1
2008-10-27




Biopsy cost = \$150
Drug therapy cost =
\$680
Costs associated with
Blindness = \$40 000
expectancies as given
in tree
24
2 & 3. Blindness prevention effectiveness
Blindness
Disorder present
Treat None
0,5
0,12
No blindness
#
No disorder
#
6.936
11.56
13.6
True positive test,
treat w/ drug
0,8
Blindness
0,013
No blindness
Disorder present
0,5
#
False negative test,
don't treat
Treatm ent Options
#
Biopsy
No disorder
#
#
True negative test,
don't treat
1
Disorder
present
0,5
Blindness
0,013
No blindness
#
No disorder
#
2008-10-27
0,12
No blindness
#
False positive te st,
treat w/ drug
Treat All
Blindness
7.536
12.555
6.933
11.556
12.555
13.59
7.539
12.565
12.565
25
3. Blindness prevention effectiveness

Blindness
Disorder present
Treat None
0,5
0,12
No blindness
#
No disorder
#
6.936
11.56
13.6
True positive test,
treat w/ drug
0,8
Blindness
0,013
No blindness
Disorder present
0,5
#
False negative test,
don't treat
Treatm ent Options
#
Biopsy
No disorder
#
#
True negative test,
don't treat
1
Disorder
present
0,5
Blindness
0,013
No blindness
#
No disorder
#
2008-10-27
0,12
No blindness
#
False positive te st,
treat w/ drug
Treat All
Blindness
12.555
7.536

12.555
6.933

11.556

What is the value of the
decrement in QALYs
associated with
Having the disorder?
Biopsy?
Treatment?
13.59
7.539
12.565
12.565
26
3. Blindness prevention effectiveness

Blindness
Disorder present
Treat None
0,5
0,12
No blindness
#
No disorder
#
6.936
11.56
13.6
True positive test,
treat w/ drug
0,8
Blindness
0,013
No blindness
Disorder present
0,5
#
False negative test,
don't treat
Treatm ent Options
#
Biopsy
No disorder
#
#
True negative test,
don't treat
1
Disorder
present
0,5
Blindness
0,013
No blindness
#
No disorder
#
2008-10-27
0,12
No blindness
#
False positive te st,
treat w/ drug
Treat All
Blindness
12.555
7.536

12.555

6.933
11.556

7.539

13,6-11,56 = 2,04
Biopsy?

13.59
12.565
What is the value of the
decrement in QALYs
associated with
Having the disorder?
13,6-13,59 = 0,1
Treatment?
12.565

13,6-12,565 = 0,035
27
3. Blindness prevention –
costs & effectiveness
Blindness
Disorder pre sent
Treat None
0,5
40,000 / (6.9 36)
0,12
No blindness
0 / (11.56)
#
No disorder
0 / (13.6)
#
True positive test,
treat w/ drug
0,8
Disorder pre sent
0,5
False negative test,
don't treat
#
Biopsy
No blindness
Blindness
0,12
No blindness
#
False positive test,
treat w/ drug
No disorder
#
#
True negative test,
don't treat
1
Disorder
present
0,5
Blindness
0,013
No blindness
#
No disorder
#
2008-10-27
0,013
#
Treatm ent Options
Treat All
Blindness
40,830 / (7.5 36)
830 / (12.555 )
40,150 / (6.9 33)
150 / (11.556 )
830 / (12.555 )
150 / (13.59)
40,680 / (7.5 39)
680 / (12.565 )
680 / (12.565 )
28
4. Blindness prevention –
expected QALYs
Blindness
Disorder present
0,500
Treat None
11,700 QALYs
0,120
10,400 QALYs
6,000 QALYs
No blindness
0,880
11,000 QALYs
No disorder
0,500
13,000 QALYs
True positive test,
treat w/ drug
0,800
Disorder present
0,500
Treatm ent Options
Biopsy : 12,314 QALYs
11,628 QALYs
False negative test,
don't treat
0,200
Biopsy
False positive test,
treat w/ drug
0,500
0,000
Treat All
0,500
11,967 QALYs
7,000 QALYs; P = 0,005
No blindness
0,987
12,000 QALYs; P = 0,395
Blindness
0,120
10,400 QALYs
6,000 QALYs; P = 0,012
No blindness
11,000 QALYs; P = 0,088
12,000 QALYs
TrueQALYs
negative test,
13,000
don't treat
1,000
Disorder
present
0,013
11,935 QALYs
0,880
12,314 QALYs
No disorder
Blindness
13,000 QALYs; P = 0,500
Blindness
0,013
11,935
QALYs
No blindness
0,987
7,000 QALYs
12,000 QALYs
No disorder
0,500
2008-10-27
12,000 QALYs
29
4. Blindness prevention –
expected costs
Blindness
Disorder pre sent
0,500
Treat None
\$4 800
\$40 000
0,120
No blindness
\$2 400
\$0
0,880
No disorder
\$0
0,500
True positive test,
treat w/ drug
0,800
Disorder pre sent
0,500
Treatm ent Options
diseaseP rev=0,5
pB lind=0,12
pB lindTreat=0,013
testS ens=0,8
testS pec=1
False negative test,
don't treat
0,200
Biopsy
False positive test,
treat w/ drug
No disorder
0,500
0,000
negative test,
don't treat
Disorder
present
0,500
\$940
\$830
Blindness
0,120
\$4 950
\$40 150
No blindness
\$150
\$830
True
\$150
1,000
Treat All
No blindness
0,880
\$1 110
\$40 830
0,013
\$1 350
0,987
\$2 070
Treat All : \$94 0
Blindness
\$150
Blindness
0,013
\$1 No
200blindness
0,987
\$40 680; P = 0,006
\$680; P = 0,4 93
No disorder
0,500
2008-10-27
\$680; P = 0,5 00
30
4. Blindness prevention –
expected costs & QALYs
Blindness
Disorder present
Treat None
0,500
\$2 400 / 12, 003 QALYs
\$40 000 / 6, 936 QALYs
0,120
\$4 800 / 11, 005 QALYs
No blindness
\$0 / 11,560 QALYs
0,880
No disorder
\$0 / 13,000 QALYs
0,500
True positive test,
treat w/ drug
0,800
Disorder present
0,500
Treatm ent Options
diseaseP rev=0,5
pB lind=0,12
pB lindTreat=0,013
testS ens=0,8
testS pec=1
Treat All : \$9 40 / 12,532 QALYs
Biopsy
\$2 070 / 12, 192 QALYs
False negative test,
don't treat
0,200
False positive test,
treat w/ drug
0,500
0,000
Treat All
0,500
\$940 / 12,53 2 QALYs
\$40 830 / 7, 536 QALYs
No blindness
0,987
\$830 / 12,55 5 QALYs
Blindness
\$40 150 / 6, 933 QALYs
0,120
\$4 950 / 11, 001 QALYs
No blindness
\$150 / 11,55 6 QALYs
\$830 / 12,55 5 QALYs
True
negative
test,
\$150
/ 13,59
0 QALYs
don't treat
1,000
Disorder
present
0,013
\$1 350 / 12, 490 QALYs
0,880
\$1 110 / 12, 891 QALYs
No disorder
Blindness
\$150 / 13,59 0 QALYs
Blindness
0,013
\$1 No
200blindness
/ 12, 500 QALYs
0,987
\$40 680 / 7, 539 QALYs
\$680 / 12,56 5 QALYs
No disorder
0,500
2008-10-27
\$680 / 12,56 5 QALYs
31
4. Cost-effectiveness of Strategies
for Blindness Prevention
Strategy
Cost
Treat All
\$940
Biopsy
\$1 110
Treat None
\$2 400
2008-10-27
ΔC
Effectivene
ss
ΔE
AVG C/E
12,53
QALYs
75 \$/QALY
\$170
12,89
QALYs
0,36 QALYs 86 \$/QALY
\$1 290
12,00
QALYs
-0,89
QALYs
ΔC/ ΔE
(ICER)
474/ QALY
200 \$/QALY (Dominated
)
32
5. Sensitivity analyses
Blindness
Disorder present
Treat None
diseasePrev
40000 / 6,936
pBlind
No blindness
0 / 11,56
#
No disorder
0 / (13.6)
#
True positive test,
treat w/ drug
testSens
Disorder present
diseasePrev
False negative test,
don't treat
#
Biopsy
No disorder
#
#
True negative test,
don't treat
testSpec
Disorder
present
Treat All
diseasePrev
Blindness
pBlindTreat
No blindness
#
No disorder
#
2008-10-27
No blindness
Blindness
pBlind
No blindness
#
False positive test,
treat w/ drug
40830 / 7,536
pBlindTreat
#
Treatm ent Options
diseaseP rev=0,5
pB lind=0,12
pB lindTreat=0,013
testS ens=0,8
testS pec=1
Blindness
830 / 12,555
40150 / 6,933
150 / 11,556
830 / 12,555
150 / 13,59
40680 / 7,539
680 / 12,565
680 / 12,565
33
Sensitivity Analys is on
prior dis e ase pr evalence
14, 0 QALY s
Treat N one
Biops y
Expected Value
13, 5 QALY s
Treat All
13, 0 QALY s
Threshold Values:
12, 5 QALY s
prior dis eas e prev alenc e = 0
EV = 12,5 QALYs
12, 0 QALY s
11, 5 QALY s
11, 0 QALY s
0,0
0,2
0,4
0,6
0,8
1,0
prior disease prevalence
2008-10-27
34
Sensitivity Analys is on
prob. blind, w / tr e atm e nt
13, 0 QALY s
Treat N one
Biops y
Expected Value
12, 5 QALY s
Treat All
12, 0 QALY s
Threshold Values:
11, 5 QALY s
prob. blind, w/ t reat ment = 0
EV = 12,0 QALYs
11, 0 QALY s
10, 5 QALY s
10, 0 QALY s
0,0
0,2
0,4
0,6
0,8
1,0
prob. blind, w / treatment
2008-10-27
35
prob. blind, w/ treatment
Sensitivity Analys is on
prior dise ase pr evale nce and prob. blind, w / tre atm ent
1,0
0,9
0,8
0,7
0,6
Treat N one
Biops y
Treat All
0,5
0,4
0,3
0,2
0,1
0,0
0,0
2008-10-27
0,2
0,4
0,6
0,8
1,0
prior disease prevalenc e
36
prob. blind, w/ treatment
Sensitivity Analys is on
prior dise ase pr evale nce and prob. blind, w / tre atm ent
1,0
0,9
0,8
0,7
0,6
Treat N one
Biops y
Treat All
Disease probability
is 0,9
0,5
0,4
0,3
0,2
0,1
Which strategy is
best?
0,0
0,0
2008-10-27
0,2
0,4
0,6
0,8
1,0
prior disease prevalenc e
37
Cost
Cost-Effe ctivenes s Analys is
At Tr eatm e nt Options
\$2 500, 0
\$2 300, 0
\$2 100, 0
\$1 900, 0
\$1 700, 0
\$1 500, 0
\$1 300, 0
\$1 100, 0
\$900, 0
\$700, 0
\$500, 0
\$300, 0
\$100, 0
0,00 QALYs
2008-10-27
Treat N one
Biops y
Treat All
7,00 QALYs 14, 00 QALY s
Effectiveness
38
Cost
Cost-Effe ctivenes s Analys is
At Tr eatm e nt Options
\$2 500, 0
\$2 300, 0
\$2 100, 0
\$1 900, 0
\$1 700, 0
\$1 500, 0
\$1 300, 0
\$1 100, 0
\$900, 0
\$700, 0
\$500, 0
\$300, 0
\$100, 0
0,00 QALYs
2008-10-27
Treat N one
Biops y
Treat All
7,00 QALYs 14, 00 QALY s
Effectiveness
39
Cost
Sensitivity Analysis on
prior dis ease pre valence
\$5
\$4
\$4
\$3
\$3
000, 0
500, 0
000, 0
500, 0
000, 0
Treat N one
Biops y
Treat All
\$2 500, 0
\$2 000, 0
\$1 500, 0
\$1 000, 0
\$500, 0
\$0, 0
0,0 QALY s
2008-10-27
di seasePrev = 0,0
7,0 QALY s
14, 0 QALY s
Effectiveness
40
Cost
Sensitivity Analysis on
prior dis ease pre valence
\$5
\$4
\$4
\$3
\$3
000, 0
500, 0
000, 0
500, 0
000, 0
Treat N one
Biops y
Treat All
\$2 500, 0
\$2 000, 0
\$1 500, 0
\$1 000, 0
\$500, 0
\$0, 0
0,0 QALY s
2008-10-27
di seasePrev = 0,6
7,0 QALY s
14, 0 QALY s
Effectiveness
41
Cost
Sensitivity Analysis on
prior dis ease pre valence
\$5
\$4
\$4
\$3
\$3
000, 0
500, 0
000, 0
500, 0
000, 0
Treat N one
Biops y
Treat All
\$2 500, 0
\$2 000, 0
\$1 500, 0
\$1 000, 0
\$500, 0
\$0, 0
0,0 QALY s
2008-10-27
di seasePrev = 1,0
7,0 QALY s
14, 0 QALY s
Effectiveness
42
Decision Trees: Strengths &
Weaknesses
Strengths
 Intuitive, visual form of
the model
 Can generate rapid
response using
available data
 Permits long-term
projections
2008-10-27


Elapse of time not
explicit in decision trees
Tree format can
become unwieldy when
events repeat
43
Acknowledgements
Slides & examples greatly indebted to:
 Jeffrey S. Hoch
 Jean Lachaine
 Drummond et al. (2005)
 Briggs, Sculpher, Claxton (2006)
 Weinstein et al., (2003)
 Kuntz & Weinstein (200x)
 Tree Age
2008-10-27
44
```