Transcript Economic Evaluation using Decision Analytic Modelling (DAMn!)

Economic Evaluation
using Decision
Analytic Modelling I
Mira Johri
Université de Montréal
2008-10-27
Outline (Part 1)
Overview
1.
Basic features of a decision model
Rationale for modelling


Decision trees
2.




2008-10-27
Conventions
Steps to perform a decision analysis
Blindness prevention example
Strengths & weaknesses
2
I. Overview
What is a model?

A health care evaluation model is a logical
mathematical framework that permits the
integration of facts and values and that links
these data to outcomes that are of interest to
health care decision makers.

Its purpose is to structure evidence on clinical
and economic outcomes to help inform
decisions about clinical practice and health-care
resource allocation under conditions of
uncertainty.
2008-10-27
4
What is a model? Cont’d

In the context of economic evaluation, a decision
analytic model uses mathematical relationships
to define a series of possible consequences
(health and economic outcomes of patients or
populations) that would flow from the alternative
options being evaluated.

It enables us to structure and examine a
complex problem.
2008-10-27
5
Example

Should we vaccinate children against
meningitis?


To vaccinate or not to vaccinate?
It depends:




2008-10-27
Risk of exposure to bacillus
Risk of contracting meningitis
Efficacy of the vaccine
Risks associated with the vaccine
6
Meningitis
Vaccinate
Choice
Do not vaccinate
2008-10-27
7
Meningitis
Exposed
Vaccinate
Not exposed
Choice
Exposed
Do not vaccina te
Not exposed
2008-10-27
8
Meningitis
Meningitis
Exposed
No m eningitis
Vaccinate
Not exposed
Choice
Meningitis
Exposed
No m eningitis
Do not vaccinate
Not exposed
2008-10-27
9
Quantifying probabilities &
outcomes

Based on the inputs of
the model, the
likelihood of each
consequence (health
and economic
outcomes of patients or
populations) is
expressed in terms of
probabilities, and each
consequence has a
cost and an outcome.
2008-10-27




Risk of exposure to
bacillus (0,2)
Risk of contracting
meningitis if exposed
(0,1)
Efficacy of the vaccine
(risk of meningitis if
vaccinated 0,01)
Risks associated with
the vaccine (0,003)
10
Dead
0
,5
Dead
Meningitis
Vaccinate
,01
Exposed
No com plications
,5
Choice
,2
No m eningitis
,99
Not exposed
Meningitis
,2
Do not vaccinate
,9
Not exposed
,8
2008-10-27
,4
Alive
,6
No m eningitis
1
0
0
0
Dead
,1
Alive
,6
,8
Exposed
,4
1
0
0
0
11
Method of analysis



Calculate the expected cost and expected outcome
of each option under evaluation.
For a given option, the expected cost (outcome) is
the sum of the costs (outcomes) of each
consequence weighted by the probability of that
consequence.
Objective: to maximise (minimise) the expected
value of positive (negative) outcomes
o
2008-10-27
In CEA, the analyses are done separately for costs and
health outcomes
12
Meningitis Vaccination
1
Dead
0
,5
Dead
Meningitis
Vaccinate
,01
Exposed
No com plications
,5
Choice
,2
,4
Alive
,6
No m eningitis
,99
Not exposed
Dead
Meningitis
,1
,2
Do not vaccinate
0
0
0
,8
Exposed
1
,4
Alive
,6
No m eningitis
,9
Not exposed
,8
1
0
0
0
#deaths with vaccination = 0,003 + (0,997 x 0,2 x 0,01 x 0,4)
= 3,8 per 1 000 children
#deaths without vaccination = 0,2 x 0,1 x 0,4
= 8,0 per 1 000 children
2008-10-27
13
Meningitis Vaccination
1
Dead
0
,5
Dead
Meningitis
Vaccinate
,01
Exposed
No com plications
,5
Choice
,2
,4
Alive
,6
No m eningitis
,99
Not exposed
Dead
Meningitis
,1
,2
Do not vaccinate
0
0
0
,8
Exposed
1
,4
Alive
,6
No m eningitis
,9
Not exposed
,8
1
0
0
0
#deaths with vaccination = 0,003 + (0,997 x 0,2 x 0,01 x 0,4)
= 3,8 per 1 000 children
#deaths without vaccination = 0,2 x 0,1 x 0,4
= 8,0 per 1 000 children
2008-10-27
14
Rationale: Synthesis

Economic evaluation studies must use all relevant evidence
(principle of EBM).

All evidence rarely comes from a single source.

A decision-model provides a framework in which a range of
evidence can by synthesised and brought to bear on the decision
problem.
 Characterise natural history of a given condition
 Impact of alternative interventions
 Costs and health effects contingent on clinical events
 Relationship between intermediate clinical measure of
effect and ultimate measure of health gain required for
CEA

2008-10-27
(Adapted from Briggs et al., 2006)
15
Rationale:
Consideration of all relevant
comparators

Cost-effectiveness must be established in
comparison with all alternative options that could
feasibly be used in practice.

A single study or RCT would rarely consider the full
range of options.

Data must be brought together from clinical studies
using appropriate statistical synthesis methods.

The decision model provides the framework to bring
the synthesis to bear on the decision problem.

2008-10-27
(Adapted from Briggs et al., 2006)
16
Rationale:
Appropriate time horizon

CEA requires that studies adopt a time horizon that
is sufficiently long to reflect all key differences
between options in terms of costs and effects.

For many interventions, this will effectively require a
lifetime horizon.

Decision models become the framework within
which to structure the extrapolation of costs and
effects over time.

2008-10-27
(Adapted from Briggs et al., 2006)
17
Rationale: Uncertainty

CEA must indicate how uncertainty in the
available evidence relating to a given policy
problem translates into decision uncertainty;
that is, the probability that a given decision is
the correct one.

Through various forms of sensitivity analysis,
decision models can help to characterise this
uncertainty and present it to decision makers.

2008-10-27
(Adapted from Briggs et al., 2006)
18
II. Decision Trees
Decision Trees - Conventions

Events are ordered from left to right.


Temporal order is often followed for sake of clarity.
Different kinds of events are distinguished using
shapes called “nodes”



2008-10-27
Square – a decision node indicating a choice facing the
decision maker, typically at start of tree
Circle – a chance node represents an event which has
multiple possible outcomes and is not under the decision
maker’s control. For an individual patient, which event they
experience is uncertain.
Triangle – a terminal node denotes the endpoint of a
scenario.
20
Decision Trees - Conventions

Branches “sprouting” from a decision node represent the set of
actions being considered (strategies)
 They need not be mutually exclusive (e.g. A, B, A+B)

Branches from a chance node represent the set of possible
outcomes of the event
 Must be mutually exclusive and exhaustive
 Probabilities must sum to 1.0

Terminal nodes are assigned a value – referred to generically as
a payoff




Cost
Utility
QALY
Expected values are based on the summation of the pathway
values weighted by the pathway probabilities.
2008-10-27
21
Steps in Decision Analysis
1.
2.
3.
4.
5.
6.
Define the problem
Structure the decision and make a tree
Fill in the data (probabilities and outcomes)
Choose the consequence with the
maximum expected value (“roll back the
tree”)
Perform sensitivity analysis
Interpret results
2008-10-27
22
1. Blindness prevention

Hypothetical population presenting clinical signs of a
possible, early-stage autoimmune disorder.

If the condition is present, and if it progresses,
blindness will result.

An imperfect test (biopsy, with the possibility of false
negatives) can help determine whether an individual
has the disorder

There exists an effective and inexpensive therapy
which lowers probability of progression to blindness.

Side effects of treatment are factored into the costs
and life expectancy at the end of each path.
2008-10-27
23
2. The Data





Disease prevalence =
0,5
pBlind (without
Tx)=0,12
pBlindTreat = 0,013
TestSens = 0,8
TestSpec = 1
2008-10-27




Biopsy cost = $150
Drug therapy cost =
$680
Costs associated with
Blindness = $40 000
Quality-adjusted life
expectancies as given
in tree
24
2 & 3. Blindness prevention effectiveness
Blindness
Disorder present
Treat None
0,5
0,12
No blindness
#
No disorder
#
6.936
11.56
13.6
True positive test,
treat w/ drug
0,8
Blindness
0,013
No blindness
Disorder present
0,5
#
False negative test,
don't treat
Treatm ent Options
#
Biopsy
No disorder
#
#
True negative test,
don't treat
1
Disorder
present
0,5
Blindness
0,013
No blindness
#
No disorder
#
2008-10-27
0,12
No blindness
#
False positive te st,
treat w/ drug
Treat All
Blindness
7.536
12.555
6.933
11.556
12.555
13.59
7.539
12.565
12.565
25
3. Blindness prevention effectiveness

Blindness
Disorder present
Treat None
0,5
0,12
No blindness
#
No disorder
#
6.936
11.56
13.6
True positive test,
treat w/ drug
0,8
Blindness
0,013
No blindness
Disorder present
0,5
#
False negative test,
don't treat
Treatm ent Options
#
Biopsy
No disorder
#
#
True negative test,
don't treat
1
Disorder
present
0,5
Blindness
0,013
No blindness
#
No disorder
#
2008-10-27
0,12
No blindness
#
False positive te st,
treat w/ drug
Treat All
Blindness
12.555
7.536

12.555
6.933

11.556

What is the value of the
decrement in QALYs
associated with
Having the disorder?
Biopsy?
Treatment?
13.59
7.539
12.565
12.565
26
3. Blindness prevention effectiveness

Blindness
Disorder present
Treat None
0,5
0,12
No blindness
#
No disorder
#
6.936
11.56
13.6
True positive test,
treat w/ drug
0,8
Blindness
0,013
No blindness
Disorder present
0,5
#
False negative test,
don't treat
Treatm ent Options
#
Biopsy
No disorder
#
#
True negative test,
don't treat
1
Disorder
present
0,5
Blindness
0,013
No blindness
#
No disorder
#
2008-10-27
0,12
No blindness
#
False positive te st,
treat w/ drug
Treat All
Blindness
12.555
7.536

12.555

6.933
11.556

7.539

13,6-11,56 = 2,04
Biopsy?

13.59
12.565
What is the value of the
decrement in QALYs
associated with
Having the disorder?
13,6-13,59 = 0,1
Treatment?
12.565

13,6-12,565 = 0,035
27
3. Blindness prevention –
costs & effectiveness
Blindness
Disorder pre sent
Treat None
0,5
40,000 / (6.9 36)
0,12
No blindness
0 / (11.56)
#
No disorder
0 / (13.6)
#
True positive test,
treat w/ drug
0,8
Disorder pre sent
0,5
False negative test,
don't treat
#
Biopsy
No blindness
Blindness
0,12
No blindness
#
False positive test,
treat w/ drug
No disorder
#
#
True negative test,
don't treat
1
Disorder
present
0,5
Blindness
0,013
No blindness
#
No disorder
#
2008-10-27
0,013
#
Treatm ent Options
Treat All
Blindness
40,830 / (7.5 36)
830 / (12.555 )
40,150 / (6.9 33)
150 / (11.556 )
830 / (12.555 )
150 / (13.59)
40,680 / (7.5 39)
680 / (12.565 )
680 / (12.565 )
28
4. Blindness prevention –
expected QALYs
Blindness
Disorder present
0,500
Treat None
11,700 QALYs
0,120
10,400 QALYs
6,000 QALYs
No blindness
0,880
11,000 QALYs
No disorder
0,500
13,000 QALYs
True positive test,
treat w/ drug
0,800
Disorder present
0,500
Treatm ent Options
Biopsy : 12,314 QALYs
11,628 QALYs
False negative test,
don't treat
0,200
Biopsy
False positive test,
treat w/ drug
0,500
0,000
Treat All
0,500
11,967 QALYs
7,000 QALYs; P = 0,005
No blindness
0,987
12,000 QALYs; P = 0,395
Blindness
0,120
10,400 QALYs
6,000 QALYs; P = 0,012
No blindness
11,000 QALYs; P = 0,088
12,000 QALYs
TrueQALYs
negative test,
13,000
don't treat
1,000
Disorder
present
0,013
11,935 QALYs
0,880
12,314 QALYs
No disorder
Blindness
13,000 QALYs; P = 0,500
Blindness
0,013
11,935
QALYs
No blindness
0,987
7,000 QALYs
12,000 QALYs
No disorder
0,500
2008-10-27
12,000 QALYs
29
4. Blindness prevention –
expected costs
Blindness
Disorder pre sent
0,500
Treat None
$4 800
$40 000
0,120
No blindness
$2 400
$0
0,880
No disorder
$0
0,500
True positive test,
treat w/ drug
0,800
Disorder pre sent
0,500
Treatm ent Options
diseaseP rev=0,5
pB lind=0,12
pB lindTreat=0,013
testS ens=0,8
testS pec=1
False negative test,
don't treat
0,200
Biopsy
False positive test,
treat w/ drug
No disorder
0,500
0,000
negative test,
don't treat
Disorder
present
0,500
$940
$830
Blindness
0,120
$4 950
$40 150
No blindness
$150
$830
True
$150
1,000
Treat All
No blindness
0,880
$1 110
$40 830
0,013
$1 350
0,987
$2 070
Treat All : $94 0
Blindness
$150
Blindness
0,013
$1 No
200blindness
0,987
$40 680; P = 0,006
$680; P = 0,4 93
No disorder
0,500
2008-10-27
$680; P = 0,5 00
30
4. Blindness prevention –
expected costs & QALYs
Blindness
Disorder present
Treat None
0,500
$2 400 / 12, 003 QALYs
$40 000 / 6, 936 QALYs
0,120
$4 800 / 11, 005 QALYs
No blindness
$0 / 11,560 QALYs
0,880
No disorder
$0 / 13,000 QALYs
0,500
True positive test,
treat w/ drug
0,800
Disorder present
0,500
Treatm ent Options
diseaseP rev=0,5
pB lind=0,12
pB lindTreat=0,013
testS ens=0,8
testS pec=1
Treat All : $9 40 / 12,532 QALYs
Biopsy
$2 070 / 12, 192 QALYs
False negative test,
don't treat
0,200
False positive test,
treat w/ drug
0,500
0,000
Treat All
0,500
$940 / 12,53 2 QALYs
$40 830 / 7, 536 QALYs
No blindness
0,987
$830 / 12,55 5 QALYs
Blindness
$40 150 / 6, 933 QALYs
0,120
$4 950 / 11, 001 QALYs
No blindness
$150 / 11,55 6 QALYs
$830 / 12,55 5 QALYs
True
negative
test,
$150
/ 13,59
0 QALYs
don't treat
1,000
Disorder
present
0,013
$1 350 / 12, 490 QALYs
0,880
$1 110 / 12, 891 QALYs
No disorder
Blindness
$150 / 13,59 0 QALYs
Blindness
0,013
$1 No
200blindness
/ 12, 500 QALYs
0,987
$40 680 / 7, 539 QALYs
$680 / 12,56 5 QALYs
No disorder
0,500
2008-10-27
$680 / 12,56 5 QALYs
31
4. Cost-effectiveness of Strategies
for Blindness Prevention
Strategy
Cost
Treat All
$940
Biopsy
$1 110
Treat None
$2 400
2008-10-27
ΔC
Effectivene
ss
ΔE
AVG C/E
12,53
QALYs
75 $/QALY
$170
12,89
QALYs
0,36 QALYs 86 $/QALY
$1 290
12,00
QALYs
-0,89
QALYs
ΔC/ ΔE
(ICER)
474/ QALY
200 $/QALY (Dominated
)
32
5. Sensitivity analyses
Blindness
Disorder present
Treat None
diseasePrev
40000 / 6,936
pBlind
No blindness
0 / 11,56
#
No disorder
0 / (13.6)
#
True positive test,
treat w/ drug
testSens
Disorder present
diseasePrev
False negative test,
don't treat
#
Biopsy
No disorder
#
#
True negative test,
don't treat
testSpec
Disorder
present
Treat All
diseasePrev
Blindness
pBlindTreat
No blindness
#
No disorder
#
2008-10-27
No blindness
Blindness
pBlind
No blindness
#
False positive test,
treat w/ drug
40830 / 7,536
pBlindTreat
#
Treatm ent Options
diseaseP rev=0,5
pB lind=0,12
pB lindTreat=0,013
testS ens=0,8
testS pec=1
Blindness
830 / 12,555
40150 / 6,933
150 / 11,556
830 / 12,555
150 / 13,59
40680 / 7,539
680 / 12,565
680 / 12,565
33
Sensitivity Analys is on
prior dis e ase pr evalence
14, 0 QALY s
Treat N one
Biops y
Expected Value
13, 5 QALY s
Treat All
13, 0 QALY s
Threshold Values:
12, 5 QALY s
prior dis eas e prev alenc e = 0
EV = 12,5 QALYs
12, 0 QALY s
11, 5 QALY s
11, 0 QALY s
0,0
0,2
0,4
0,6
0,8
1,0
prior disease prevalence
2008-10-27
34
Sensitivity Analys is on
prob. blind, w / tr e atm e nt
13, 0 QALY s
Treat N one
Biops y
Expected Value
12, 5 QALY s
Treat All
12, 0 QALY s
Threshold Values:
11, 5 QALY s
prob. blind, w/ t reat ment = 0
EV = 12,0 QALYs
11, 0 QALY s
10, 5 QALY s
10, 0 QALY s
0,0
0,2
0,4
0,6
0,8
1,0
prob. blind, w / treatment
2008-10-27
35
prob. blind, w/ treatment
Sensitivity Analys is on
prior dise ase pr evale nce and prob. blind, w / tre atm ent
1,0
0,9
0,8
0,7
0,6
Treat N one
Biops y
Treat All
0,5
0,4
0,3
0,2
0,1
0,0
0,0
2008-10-27
0,2
0,4
0,6
0,8
1,0
prior disease prevalenc e
36
prob. blind, w/ treatment
Sensitivity Analys is on
prior dise ase pr evale nce and prob. blind, w / tre atm ent
1,0
0,9
0,8
0,7
0,6
Treat N one
Biops y
Treat All
Disease probability
is 0,9
0,5
0,4
0,3
0,2
0,1
Which strategy is
best?
0,0
0,0
2008-10-27
0,2
0,4
0,6
0,8
1,0
prior disease prevalenc e
37
Cost
Cost-Effe ctivenes s Analys is
At Tr eatm e nt Options
$2 500, 0
$2 300, 0
$2 100, 0
$1 900, 0
$1 700, 0
$1 500, 0
$1 300, 0
$1 100, 0
$900, 0
$700, 0
$500, 0
$300, 0
$100, 0
0,00 QALYs
2008-10-27
Treat N one
Biops y
Treat All
7,00 QALYs 14, 00 QALY s
Effectiveness
38
Cost
Cost-Effe ctivenes s Analys is
At Tr eatm e nt Options
$2 500, 0
$2 300, 0
$2 100, 0
$1 900, 0
$1 700, 0
$1 500, 0
$1 300, 0
$1 100, 0
$900, 0
$700, 0
$500, 0
$300, 0
$100, 0
0,00 QALYs
2008-10-27
Treat N one
Biops y
Treat All
7,00 QALYs 14, 00 QALY s
Effectiveness
39
Cost
Sensitivity Analysis on
prior dis ease pre valence
$5
$4
$4
$3
$3
000, 0
500, 0
000, 0
500, 0
000, 0
Treat N one
Biops y
Treat All
$2 500, 0
$2 000, 0
$1 500, 0
$1 000, 0
$500, 0
$0, 0
0,0 QALY s
2008-10-27
di seasePrev = 0,0
7,0 QALY s
14, 0 QALY s
Effectiveness
40
Cost
Sensitivity Analysis on
prior dis ease pre valence
$5
$4
$4
$3
$3
000, 0
500, 0
000, 0
500, 0
000, 0
Treat N one
Biops y
Treat All
$2 500, 0
$2 000, 0
$1 500, 0
$1 000, 0
$500, 0
$0, 0
0,0 QALY s
2008-10-27
di seasePrev = 0,6
7,0 QALY s
14, 0 QALY s
Effectiveness
41
Cost
Sensitivity Analysis on
prior dis ease pre valence
$5
$4
$4
$3
$3
000, 0
500, 0
000, 0
500, 0
000, 0
Treat N one
Biops y
Treat All
$2 500, 0
$2 000, 0
$1 500, 0
$1 000, 0
$500, 0
$0, 0
0,0 QALY s
2008-10-27
di seasePrev = 1,0
7,0 QALY s
14, 0 QALY s
Effectiveness
42
Decision Trees: Strengths &
Weaknesses
Strengths
 Intuitive, visual form of
the model
 Can generate rapid
response using
available data
 Permits long-term
projections
2008-10-27


Elapse of time not
explicit in decision trees
Tree format can
become unwieldy when
events repeat
43
Acknowledgements
Slides & examples greatly indebted to:
 Jeffrey S. Hoch
 Jean Lachaine
 Drummond et al. (2005)
 Briggs, Sculpher, Claxton (2006)
 Weinstein et al., (2003)
 Kuntz & Weinstein (200x)
 Tree Age
2008-10-27
44