Transcript Value for money

Value for money
How to design RCTs to ensure their
compatibility with economic
evaluation
Kevin Marsh
September 2006
Introduction and overview
Question:
• what are the methodological lessons from attempts to build economic
evaluations on RCTs?
Source of evidence
• two recent NICE public health reviews
Challenges
• measuring the cost of interventions
2 requests
• standardised measure of effects
• generalising
1 question
Why are we concerned with value for money?
Campbell Collaboration Economic Methods Group
(CCEMG) Policy Brief (in press):
Provision of evidence on cost-effectiveness, alongside
evidence on the benefits and adverse effects of
interventions, can significantly enhance […] decision-making
by managers and policy makers.
Effectiveness plane
effectiveness
-ve
+ve
y
x
Should we invest in intervention ‘x’?
• yes – ‘x’ is more effective than ‘y’
• but what is the opportunity cost of ‘x’?
Cost-effectiveness plane
+ve
effectiveness
-ve
cost
y
-ve
+ve
Cost-effectiveness plane
+ve
Should we invest in ‘x’?
• yes - ‘x’ dominates ‘y’
effectiveness
-ve
cost
y
-ve
+ve
Cost-effectiveness plane
+ve
Should we invest in ‘x’?
• no - ‘y’ dominates ‘x’
-ve
y
cost
effectiveness
-ve
+ve
Cost-effectiveness plane
+ve
effectiveness
+ve
y
Should we invest in ‘x’?
• ? – cost-effect trade off?
cost
-ve
-ve
Aren’t we already doing economic evaluation?
Number of studies identified in 2 recent reviews of
effectiveness and economic studies
Review
Effect
Econ
Ratio
studies studies
Promotion of physical activity
24
8
3:1
Community-based substance misuse prevention
in young vulnerable people
222
5
44.4:1
How do we model value for money from RCTs?
Intervention
 risk factor (e.g. cigarette use)
 probability substance misuse
 probability problematic SM
 health,  crime,  unemploy.
£ pp
£ saved pp
QALY gain pp
How do we model value for money from RCTs?
Intervention
 risk factor (e.g. cigarette use)
 probability substance misuse
Cost / 
risk factor
 probability problematic SM
 health,  crime,  unemploy.
£ pp
£ saved pp
QALY gain pp
(£ pp- £ saved pp) / QALY gained pp
How do we model value for money from RCTs?
Intervention
 risk factor (e.g. cigarette use)
 probability substance misuse
Review of
effectiveness
 probability problematic SM
(RCT)
 health,  crime,  unemploy.
£ pp
£ saved pp
QALY gain pp
How do we model value for money from RCTs?
Intervention
Step 1: cost
intervention?
 risk factor (e.g. cigarette use)
 probability substance misuse
 probability problematic SM
 health,  crime,  unemploy.
£ pp
£ saved pp
QALY gain pp
Good description of a simple intervention
Facilitate referral to an external treatment agency
1. discuss negative consequences of personal drug use
2. discuss impediments to reducing negative consequences
3. advice about appropriate services
4. discuss possible impediments to treatment
5. arrange the first appointment with a named counselor
6. reminder telephone call prior to the first appointment
7. offer of transport to first appointment
8. offer to accompany then to their first appointment
Example: intervention description
Facilitate referral to an external treatment agency
1. discuss negative consequences of personal drug use
2. discuss impediments to reducing negative consequences
3. advice about appropriate services
4. discuss possible impediments to treatment
5. arrange the first appointment with a named counselor
6. reminder telephone call prior to the first appointment
7. offer of transport to first appointment
8. offer to accompany then to their first appointment
1. Who provides the intervention?
Example: intervention description
Facilitate referral to an external treatment agency
1. discuss negative consequences of personal drug use
2. discuss impediments to reducing negative consequences
3. advice about appropriate services
4. discuss possible impediments to treatment
5. arrange the first appointment with a named counselor
6. reminder telephone call prior to the first appointment
7. offer of transport to first appointment
8. offer to accompany then to their first appointment
2. How long does each element take?
Example: intervention description
Facilitate referral to an external treatment agency
1. discuss negative consequences of personal drug use
2. discuss impediments to reducing negative consequences
3. advice about appropriate services
4. discuss possible impediments to treatment
5. arrange the first appointment with a named counselor
6. reminder telephone call prior to the first appointment
7. offer of transport to first appointment
8. offer to accompany then to their first appointment
3. % take up optional elements?
Request 1: resource use data
1. describe resource use: who does what, what equipment?
2. measure resource use: hours, units?
3. value resource use: £
How do we model value for money from RCTs?
Intervention
 risk factor (e.g. cigarette use)
 probability substance misuse
Step 2: CE?
 probability problematic SM
 health,  crime,  unemploy.
£ pp
£ saved pp
QALY gain pp
Step 2: calculating cost-effectiveness
Intervention 1:
Intervention 2:
Cost - £100
Cost - £100
Effect – YP not use
cannabis in last month
when would otherwise
Effect – YP move from
use cannabis 20 times/yr
to only 5 times/yr
How do we model value for money from RCTs?
Intervention
 risk factor (e.g. cigarette use)
 probability substance misuse
Step 3: longterm effect?
 probability problematic SM
 health,  crime,  unemploy.
£ pp
£ saved pp
QALY gain pp
Step 3: modelling long-term effect
RCT:
Epidemiological:
Change in YP risk
factors
Impact risk factors
on LR sub. misuse
Same risk factors
Different measures
E.g.
Use cannabis
in the last
month? Y/N
?
Use cannabis
> 31 times/yr?
Y/N
NICE econ review: matching effect studies to
epidemiological data
Population type
General at risk
Results of Quality and
effectiveness
effect
review
criteria
Outcome
variable
criteria
98
17
3
Behavioural / aggressive
young people
BME
7
0
0
46
8
3
Young offenders
10
0
0
Young substance users
23
3
0
18
3
0
11
0
0
9
4
0
222
35
6
Families with drug using
members
School drop-outs, truants
and underachievers
Other
6
only model fraction of the effects identified
Request 2: use standard measures of effect
•
compare between studies
•
link with epidemiological data
How generalisable are our results?
1. RCTs: conflicting + inconsistent results
2. Heterogeneity
How generalisable are our results?
Study
Intervention
Griffin et al
Life Skills Training (LST) v
(2003)
normal educational curriculum
Population / inclusion
criteria
Context
General at risk:
Low grades in school
School
Peers use substances
Location
New York,
USA
Teacher training, parent
Hawkins et al
(1999)
education, and social
competence training for
General at risk:
Seattle and
School
Washington,
USA
children v normal educational
curriculum
Zavela et al
‘Say Yes First’ v normal
(2004)
educational curriculum
Botvin et al
(1995)
Botvin et al
(2001)
Life Skills Training (LST) v
Normal educational curriculum
(information on drugs)
Life Skills Training (LST) v
General at risk
Colorado,
USA
BME group:
students in six New York
City public schools who
consented to participate
School
BME group:
Normal educational curriculum Seventh graders from 29
(information on drugs)
School
School
New York schools.
New York,
USA
New York,
USA
BME group:
Campbell et al
The Abecedarian Project v
(2002)
normal child rearing
High risk index created
from 13 sociodemographic factors
School /
home
USA
Question: how do we overcome the challenge
of heterogeneity?
Can an RCT only tell us about intervention X if implemented
in way A, for population B, in social context C?
If yes
RCT for each
combination of
intervention,
control,
population,
context?
Alternative
method?
Quasi-exp?
Other?