Transcript Teaching Family Physicians To Be Information Masters

Is it True? Evaluating Research about
a Therapy
Allen F. Shaughnessy, PharmD, MmedEd
Tufts University School of Medicine
Department of Family Medicine
David C. Slawson, MD
The University of Virginia,
Department of Family Medicine
2
Task 3: Clarification

Now let’s say that the same patient has heard
from a friend that there is a vitamin that will
help prevent migraines. What study design
could answer the question of whether there is
a vitamin that is useful in preventing migraine
headaches in this patient?
Study Methods
to Answer This Question





Epidemiology: Patients taking a vitamin are less likely to
have migraines
Pharmacology: Drug x affects cerebral vasculature in rat
brain isolates
Case report: “It worked on one patient”
Case-series: “It worked on a bunch of patients”
Randomized controlled trial: 1/2 get drug, 1/2 placebo.
No one knows who ‘til the end who took what
4
The “Danger” of Natural History
Studies

For each 1% decline in hemoglobin A1c,
decrease in mortality, stroke, CHD, . . ..

Women taking HRT and CVD mortality.
The “Danger” of Natural History
Studies


Compliance Effect!
Known as Factor X
Validity

Internal validity: How well was the study done?
Do the results reflect the truth?

External validity: can I apply these results to
MY patients?
8
Was it a
randomized controlled trial?
Randomization is the
best protection against
being mislead
9
The value of randomization


32 controlled trials of anticoagulation in acute MI
Results by type of study:
Historical
control
Controlled
trial
Randomized
controlled trial
Concealed
Allocation
Relative Risk
Reduction
Case fatality
rate
42%
38.3%
33%
29.2%
31%
19.6%
18%
12.1%
Chalmers TC, et al. N Engl J Med 1977;297:1091-6.
10
Was allocation assignment
“concealed”?
Did investigators know to
which group the potential
subject would be
assigned before enrolling
them?
11
Importance of concealed allocation
Trials with unconcealed allocation consistently
overestimate benefit by ~40%
Schulz KF, Chalmers I, Hayes RJ, et al. JAMA 1995;273:408-12
Schulz KF, Grimes DA. Lancet 2002;359:614-18.
Pildal J, et al. Int J Epidemiol 2007;36:847-857
Moher D, et al. Lancet 1998;352:609-13.
12
Was allocation assignment
“concealed”?

Concealed allocation  blinding
• Blinding can occur without concealed allocation
• Surfactant in the NICU
• Allocation can be concealed in an unblinded study
• PT vs surgery for knee DJD
Moseley JB, O'Malley K, Petersen NJ, et al. N Engl J Med 2002; 347:81-8.
13
Conducting a Study
Potential Subjects
Concealed
Allocation
Trial starts
Actual
Subjects
A
Randomization
Blinding, etc
B
14
Importance of concealed
assignment

Meta-analysis of trials evaluating screening
mammography
•
In studies in which allocation wasn’t concealed
•
Trials with concealed allocation = screening harmful!
• Higher SE status, education level in screened group
• Age disparity (average 6 mo older in the unscreened group)
• Richer, smarter, younger
• No effect or increased mortality
• 20% more mastectomies
Lancet Jan 8, 2000; Oct 20, 2001
15
Mammography Study Sign-up
Conducting
a Study
Number Group
Patient name
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Trial starts
A
Mamm.
Sara Smith
No Mamm
Jill Jones
Wendy Walsh
Mamm
Potential
Subjects
No Mamm
Linda Lucky
No Mamm
Mamm
Linda Lucky
Mamm
No Mamm
Mamm
No Mamm
Mamm
Actual
Mamm
No mammSubjects
No mamm
Mamm
Randomization
Blinding, etc
Concealed
Allocation
B
Technical Nitpicking? Could this
really make a difference?



Cumulative database: ~500,000 women
Current policy is based on very small differences:
•
•
•
Deaths in unscreened women 902
Deaths in screened women
837
Death difference (of 456,349) 65!
Systematic bias is not “random error” for which metaanalysis can compensate
17
Mundus Vult Decipi
“The world wishes to be
deceived”
People would rather be deceived than
have the truth cause anxiety
Caleb Carr, Killing Time
18
“YOU WANT ANSWERS??!!!
“I WANT THE TRUTH!!”
“YOU CAN’T HANDLE THE TRUTH!!”
Jack Nicholson and
Tom Cruise
“A Few Good Men”
19
Nonfebrile Seizure Incidence
20
Were all the patients properly accounted for
at its conclusion?

Complete follow-up?

“Intention to treat” analysis?
•
•
•
Patients are analyzed in the groups to which they are assigned
Attempts to reflect “real world” clinical situations in which not all
patients follow treatment recommendations
Watch when they compare only compliers with compliers and noncompliers
•
Compliant subjects always do better overall
21
Was study “double-blinded”?

Did the patients know to which group they were
assigned?

Did the treating physician know?

Did investigators assessing outcomes know (“tripleblinding” – up to 7 levels!)?
•
Judicial assessor blind + allocation concealment = surgery
RCTs
Schulz KF. Ann Int Med 2002;136:254-9.
22
Were intervention and control groups
similar?



See Table 1 of most studies
Randomization is best way to avoid bias, though
imbalances still can occur (especially if allocation
was not concealed)
Small differences sometimes are important
23
Task 4.
24