Transcript presentation_6-15-2010-12-47-7

Statistics in Drug Regulation:
The Next 10 Years
Thomas Permutt
Director, Division of Biometrics II
Center for Drug Evaluation and Research
The views expressed are those of the speaker and not necessarily of FDA.
Statutory Standards
• Substantial evidence of efficacy
• All tests reasonably applicable for safety
• Balance not explicit, but history clear
Risk/Benefit
• Formerly:
– Very good evidence about direction of mean
treatment effect
• Too good? No.
– Adverse events:
• Common: statistical but unimportant
• Rare: nonstatistical but important
What’s New?
• Rofecoxib
• Rosiglitazone
• LABA
Rofecoxib
• Heart attacks
• Large outcome trial
– which was trial in new indication
• Now need outcome studies for COX-2 and
maybe nonselective
Rosiglitazone
•
•
•
•
Nissin meta-analysis
We do meta-analysis
You do meta-analysis
You do outcome trial, maybe
Meta-analysis
•
•
•
•
Hard
Nonstatistical
Statistical
Both different in regulatory setting
Meta-analysis: Nonstatistical
• Better information, but …
• Doesn’t fit usual protocol-driven regulatory
framework, either
• Do it anyway, but …
• Nobody will believe you (or us), so … ?
– sensitivity analysis important
Meta-analysis: Statistical
• Fixed vs. random effects
– doesn’t matter much for global null, but
– this doesn’t apply to noninferiority
• Attributable vs. relative risk
– relative risk “stable” across settings
• different length of study, at least
– but attributable risk is what matters
– what about zeroes
• Nissin to Congress: “no information”
What triggers this?
• “Signal”
– Class effects
– Someone else’s meta-analysis
• For diabetes, everything
• For COX-2, probably everything
– other COX?
LABA
• Believed to cause death
– not “side effect,” death from asthma
• Effect mostly “seen” without steroid
• So, with steroid?
With Steroid, Show What?
• Noninferior to nothing?
– i.e., combination therapy vs. steroid
• Noninferior to realistic alternative?
– e.g., increased dose of steroid
– why not superior?
• because of benefit
• Interaction with steroid?
– i.e., already “know” without steroid: Is with different?
– maybe can’t do without steroid anyway
Noninferiority Margins
• Not “1.3”
– COX-2
– diabetes
– asthma!
• Risk-benefit
– for direct measures
– for surrogates
Surrogate
• Everyone likes “hard” endpoints but …
• They mostly don’t measure benefit
• They are correlated with benefit
Correlation with Benefit
• Does drug produce benefit or modify
correlation? (anti-arrythmics, maybe
glitazones)
• Qualitative validation hard enough
• Quantify benefit very hard
– estimate strength of relationship
– and hope it holds
Patient-Reported Outcomes
• Hard endpoints are “nice” but they don’t
measure utility
• PRO are squishy but relevant
• Psychometrics is not evil (now)
Linking Risk and Benefit
• Expected utility
– mean efficacy outcome
– incidence of AE
– (mean effect) X (goodness) – (AE rate) X
(badness)
• Other formulas are incorrect
– provided utility is linear wrt effect
It Isn’t Linear
• For surrogates
• For PROs
Utility Calculations: Example
• 50% symptom-free
• 50% intolerable adverse events
• Good or bad?
– How bad were symptoms?
– How bad were adverse events?
Two Drugs
• Women have efficacy
• Men have adverse
events
• Women have efficacy
• Women have adverse
events
• Men have nothing
Two Drugs
• Women have efficacy
• Men have adverse
events
• Useful drug
– provided AEs are
reversible
• Women have efficacy
• Women have adverse
events
• Men have nothing
• Useless drug
“Expected utility” does not distinguish!
Why Doesn’t Expectation Work?
• Because you don’t really measure benefit
– benefit at timepoint (or average over time) is
surrogate for long-term benefit
– don’t get long-term benefit if you drop out
– LOCF makes it worse
• “Mixing up” safety and efficacy is …
– not illegal
– not even stupid
– “individualized medicine”
• dropout is good biomarker!