Transcript Outcomes in occupational health settings: The contribution of

Will It Work?
Efficacy Studies and Clinical Research
Glenn Pransky, M.D., M.Occ.H.
Director, Liberty Mutual Center
for Disability Research
Associate Professor, UMass
Medical School
How Do Doctors Decide?
• It ought to work - inductive reasoning
• Others vouch for it - abdication
• Demonstrated effect - deduction
Gastric Freezing
• Cooling to -5° C  decreased secretions
• President of ACS tried -10° C; published
case series of 20 pts:
 symptoms, X-ray healing
• 2500 gastric freezing machines sold 1962-66
• 15,000 treatments in US by 1969
Randomized Trial
(Ruffin et al, NEJM, 1969; 281, p.16)
• Double - blinded intervention - treating
MD and patient didn’t know
• Results:
– Recurrence in 30 (44%) of placebo (warm)
– Recurrence in 35 (51%) of treated patients
Efficacy
• For a given medical problem, efficacy is
the probability that treatment 
significant improvement in outcome,
under ideal conditions
• Effectiveness = efficacy in usual
conditions of practice
Efficacy Studies
• Phase I - human dosage / toxicity
• Phase II - uncontrolled trial
• Phase III - randomized, controlled trial
RCT Strengths
• Show maximum effect vs.
placebo
• Strongest design to  bias from
several potential sources
• Prove causation
• Compare treatments
RCT Weaknesses
• Infrequent side-effects missed
• Long latency effects often attenuated
• Community usage unknown
• problems with placebo definition
Home vs. Hospital Care for
Suspect MI (Hill, Lancet, 1978, l:837)
• RCT, 6 week follow-up after acute ER visit
• Sickest patients excluded
• Results:
– HOME: 20% of 79 died
– HOSPITAL: 18% of 71 died
• Need 261 / grp. - detect 25% RR difference
• Need 45 / grp. - detect 50% RR difference
RCT Sample Size
• Type I error: false optimism; usually = 5%
(p-value)
• Type II error: false pessimism; usually b= 20%
• Power = prob. of finding A if A is true
• Required parameters:
bexpected control outcome, expected
intervention effect, rate of outcome / length
follow-up
Preventive Intervention,
Uncommon Adverse Outcome
• New treatment for HTN in pregnancy
• Goal: risk from 2% 1%
• Type 1 error 5%, type 2 error 20%
• 2511 individuals needed in each group!
Selection - Exclusions
• Old, young, demented, minorities,
pregnant
• Liver or kidney disease - drug
metabolism
• Noncompliant patients (run-in period)
Selection - Inclusions
• Volunteers - sicker, compliant
• Referral (selected) population
Treatment problems in efficacy
studies
• Atypical = non-generalizable
• Randomized allocation with complex
significant factor patterns
– Propensity scoring as alternative
• Ethics of no rx. vs. comparison rx.
• Placebo effect
• Multifaceted intervention investigator attributes to one element
Control
• Blinding - ideally pt., treaters, evaluators
– When is this not feasible?
• Surgical treatments - randomization, but
patients & treaters know
Control
• Contamination type II error
– sympathy, community care
• Cointervention  type I error
(unblinding)
– ‘more is better,’ support of tertiary care center
vs usual care
Follow-up
• Adequate for stability of 1 / 2 outcomes
• Multiple assessments
• Loss - sicker, toxicity, high study burden
• Assume the worst?
Surgery vs. Medical Rx for
Bilateral Carotid Stenosis
(Fields & Maslenikov, JAMA, 1970, 211: p1993)
• 79 surg, 79 med @ F/U 1 year later
• Surgery: 27% risk stroke/death (p=0.02)
• Of 16 LTFU, 15 were allocated surgery
- all had early death or stroke
• Intention-to-treat analysis: risk 16%, p = 0.09
Outcomes
•
•
•
•
•
•
Clinical / physiologic
Function
Employment
Satisfaction, quality of life
Side-effects
Value of surrogate endpoints?
Analytic Methods
• Variable length follow-up KaplanMeier survival curve (for terminal event)
• Assumes constant probability
Problems
• Multiple comparisons
• Large trial: statistical but not practical
significance
Typical Results
% alive
100
1
50
2
0
t1
t2
t3
t4
t5
t
1 = follow-up too short (Type 2 error)
2 = follow-up too long; few subjects or overwhelming additional
influences plateau usually = low numbers
Adherence
Chola, Mortality in CDP (NEJM 303: 1038, 1980)
Post-MI secondary prevention trial with clofibrate.
Compliance
#
Took < 80%
Took > 80%
182
1813
%
Mortality
26%
16%
Risk reduction= .26-.16/.26= 38%
Convincing?
CDP Study (2)
• Same results in placebo group
• Conclusion: compliance survival effect!
Are All Relevant Results Reported?
(Oliver et al WHO trial on primary prevention IHD using
databrate Lancet 1980; 2, p. 379)
Measure
Placebo
Clofibrate
Avg. change serum chol.
+1%
-9%
Nonfatal MI/1000
7.2
5.8
Fatal MI/1000
1.7
1.6
Total deaths/1000
5.2
6.2
Generalizability
• Adequate description?
• Typical cohort, setting?
• Typical treatment; acceptable, cost, available
• Sideffects - effort to obtain information
Functional Restoration: Pitfalls in
Evaluating Efficacy
(Gatchel, Mayer, Hazard et al, Spine, 1991, 17:988)
• Staff Training
• Program duplication, same selection criteria
& patients
• Consistent evaluation methods, close follow-up
• Report important details (drop-out, total costs)
• Understand key differences - jurisdiction, etc.
Placebo Effects, Other Issues in Pain Treatment
(Turner, Deyo, et al., JAMA, 1994, 271:1609)
• Regression to mean apparent benefit
regardless of treatment
– Patients enroll @ worst point in cyclical course
– Also may reflect meas. error, random variation
• Nonspecific Rx effect
– Attention, concern; expectations of healing;
“facilitated” reporting
Placebo Effects
• “Change in illness attributable to symbolic import
of treatment.”
• Up to 70% of responses to Rx’s initially thought to
be efficacious eventually defined as placebo effects
• IMA procedure: 56% significant improvement,
42% NTG use
– (Cobb, NEJM, 1959, 260, 1115-8: skin incision
placebo)
• Diskectomy - negative surgical exploration:
37% no sciatica, 42% no LBP
– (Spangfort, Acta Orth Scand, 1972, 142:1)
Factors Influencing Placebo
Response
• Patient attitude to provider and treatment
• More compliance
• Provider attitude, unbliding
Placebo Effects in Pain Treatment
• Mimic expected dose-response relationship (pain
medications)
• Mimic expected side-effects (drowsiness, nausea)
• Perception: large or injected = strong;
yellow = stimulant
• Suggestion worse condition (Nocebo effect)
Questions
1. Randomization?
2. Blinding?
3. Adequate data collection / reporting?
4. Statistical, clinical significance?
5. All participants included
6. Generalizable
7. Feasible?