Transcript May 2: Guest Lecture on Multiple Testing

Bonferroni: Friend or Foe?
Multiple Testing in Cardiovascular Medicine
Dhruv S. Kazi, MD, MSc
AHA Cardiovascular Outcomes Research Fellow
Stanford University
[email protected]
“Off hand, I’d say you’re
suffering from an arrow
through your head, but just to
play it safe, let’s get an echo.”
Death from Cardiovascular Causes,
Nonfatal Myocardial Infarction, or Stroke
CURE
= 9 billion dollars
Yusuf, S, et al. N Engl J Med 2001;345:494-502
Liver
Cytochrome P450-dependent
oxidation
Clopidogrel
Ticagrelor
CYP2C19
Binds to P2Y12
Receptor on Platelets
Binds to P2Y12
Receptor on Platelets
Potential Strategies
 Clopidogrel
 Ticagrelor
Which one would you want?
Primary Efficacy Endpoint in the PLATO-Invasive RCT
Cannon, CP, et al. Lancet 2010; 375: 283-93.
Cannon, CP, et al. Lancet 2010; 375: 283-93.
Methods
 Cohort: 100,000 patients who present with ACS and
undergo PCI, age at entry – 65 years
 Analytic Horizon: Lifetime
 Perspective: “Ideal Insurer”
 Interventions
– DAPT 12 months from last ACS or PCI, whichever is
later
– Aspirin monotherapy for life thereafter
Possible Explanations?
Possible Explanations?
 True Difference
 Chance Finding
 Fraud?
The Multiple-Look Problem
1
0.9
0.8
Cumulative prob of
a positive association
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
10
20
30
40
50
Number of analyses
60
70
80
90
100
So how do you get around this?
Traditionally, “don’t run multiple subgroups” unless:
- The analyses are pre-specified
- The analyses are biologically plausible
And if you must, conduct rigorous statistical adjustment!
Bonferroni Adjustment
 Conservative
 Assumes independence
 1-(1-α)1/n ~ α/n
 But does this make sense?
BMJ. 1998 April 18; 316(7139): 1236–1238.
How Do We Proceed?
(Do you still want the drug?)
 Multiple testing is problematic (even if pre-specified)
 The challenges of a priori hypotheses
Conclusions
 Multiple testing is a complicated question: with real
clinical consequences
 Statistical adjustment is a necessary but imperfect
solution
Trial and Error. Kaul S, et al. J Am Coll Cardiol 2010;55:415–27
Conclusion
The p value is no substitute for a brain.
Thank You!
[email protected]