Transcript Presentation Title - University of Kentucky

Confounding and the Language of
Experimentation
Part II - Importance of Proper
Comparisons
and Randomization
This video is designed to accompany
pages 13-18
in
Making Sense of Uncertainty
Activities for Teaching Statistical Reasoning
Van-Griner Publishing Company
Arthroscopic Knee
Surgery
Traditional Design
Patients come to
doctor with knee
symptoms. Range of
motion measured
Arthroscopic surgery
used to treat the
condition
Largely:
•
•
subjects felt better
range of motion was better
Patients asked if
they felt better and
range of motion remeasured.
What’s Missing
No Placebo Comparison
How do we know that the results
were not confounded by the
placebo effect?
Seriously!
Source: N Engl J Med. 2002 Jul 11;347(2):81-8.
Title:
A controlled trial of arthroscopic surgery for
osteoarthritis of the knee.
Authors: Moseley JB, O'Malley K, Petersen NJ, Menke
TJ,
Brody BA, Kuykendall DH, Hollingsworth JC,
Ashton CM, Wray NP.
Place:
Baylor
Houston Veterans Affairs Medical Center,
College of Medicine, Houston, TX 77030,
Controlled Study w/
Placebo
METHODS:
A total of 180 patients with osteoarthritis of the knee were randomly
assigned to receive arthroscopic débridement, arthroscopic lavage, or
placebo surgery.
Patients in the placebo group received skin incisions and underwent a
simulated débridement without insertion of the arthroscope. Patients
and assessors of outcome were blinded to the treatment-group
assignment.
Outcomes were assessed at multiple points over a 24-month period
with the use of five self-reported scores--three on scales for pain and
two on scales for function--and one objective test of walking and stair
climbing. A total of 165 patients completed the trial.
Surprising Conclusion
CONCLUSIONS:
In this controlled trial involving patients with osteoarthritis of the knee,
the outcomes after arthroscopic lavage or arthroscopic débridement
were no better than those after a placebo procedure.
(N Engl J Med 2002;347:81-8.)
Copyright © 2002 Massachusetts Medical Society.
Language
• Response Variables – Level of pain and function
• Explanatory Variable – Type of Procedure (Real or
Placebo)
• Subjects – The participating patients with osteoarthritis
of the knee
• Confounding– In the traditional design, the placebo
effect created the confounding and compromised the
inference.
Diagraming the Faulty Design
Before or No
Surgery
Surgery
Placebo
Respons
e
Measure
Pain and
Function
Compare
Measure
Pain and
Function
Diagraming the Careful
Design
Placebo
Surgery
Real
Surgery
Measure
Pain and
Function
Compare
Measure
Pain and
Function
Importance of
Randomization
USE OF AN AIDS VIRUS DRUG CURBED ON THE BASIS OF DATA
By ANDREW POLLACK, Special to the New York Times
Published: April 16, 1987
SAN FRANCISCO, April 15— The Food and Drug Administration has
told a California pharmaceutical company that there is not yet enough
evidence to justify more widespread use of its antiviral drug ribavirin to
treat patients with early symptoms of infection with the AIDS virus.
Importance of
Randomization
Reported Results
Treatment
Number of
Patients
How Many Developed
Aids
800 mg
52
0
600 mg
55
6
Placebo
56
10
Looks Convincing!
FDA Looked Closely …
There was evidence that the sickest patients had been
assigned to the placebo group.
And the healthiest patients to the Ribavirin treatments.
Inference of “effective treatment” was compromised.
Language
• Response Variable – Development of AIDS (yes/no)
• Explanatory Variable – Type of Intervention (Ribavirin
level, Placebo)
• Subjects – The participating patients with pre-AIDS
symptoms
• Confounding– Lack of randomization put healthiest
patients in the Ribavirin treatment groups.
Benefits of
Randomization
Helps to keep the comparison groups – treatment groups
– as much alike as possible.
In that sense, randomization addresses confounding in a
very direct way.
One-Sentence Reflection
The placebo effect and lack of
randomization can create very real
obstacles to making credible
inferences from experimental data.