Transcript active control trials - American Statistical Association

ISSUES THAT PLAGUE NONINFERIORITY TRIALS
PAST AND FUTURE
RALPH B. D’AGOSTINO, SR.
BOSTON UNIVERSITY
HARVARD CLINICAL RESEARCH
INSTITUTE
OBJECTIVES
1. REVIEW ISSUES: PAST, PRESENT
AND FUTURE IN NON-INFERIORITY
(NI) STUDIES
2. PRESENT/ DISCUSS EXAMPLES
3. MAKE SOME COMMENTS FOR
IMPROVEMENTS
4. PRESENT A PERSONAL VIEW
OUTLINE
1.
2.
3.
4.
5.
6.
7.
Early Objectives and Issues
Approaches to Non-inferiority Trials
Examples (Here are some Problems)
Non-Inferiority AND/OR Superiority
All is Non-Inferiority
Intent-to-Treat vs. Per Protocol
New Major Issues
EARLY OBJECTIVES AND
ISSUES: EQUIVALENCY
•
•
•
•
American Dental Association (ADA 1980s)
CREST equivalent to COLGATE?
Ho: A-B>= 10% or A-B<= 10%
What does the 10% mean?
– DFMS or DFMT for 2 years, 3 years?
• Study done on differences and ratio used as
descriptive measure of “effect”
– 5.0 vs 5.4 becomes (5.4-5.0)/5.0 = .4/5.= 8%
EARLY OBJECTIVES
• M = 10% CAME FROM NOWHERE, BUT
WE KNEW WHAT IT WAS, That is, 10%
• TREATMENT DIFFERENCES
CONCERNED DIFFERENCES (RATIOS)
BETWEEN ACTIVE TREATMENTS
• WE WERE LOST BUT WE BELIEVED
WE HAD A “SENSE” ABOUT IT
APPROACHES TO NI TESTS
•
•
MUST DO BETTER THAN PLACEBO
But you cannot use a Placebo (P)
1. Putative Placebo Approach
2. Test Treatment (T) vs Positive Control (C)
directly with given Margin M (Assay
Sensitivity approach)
APPROACH 1 (Putative Placebo)
Stellar Example from the Past
• CAPRIE Study. Hasselblad and Kong
(2001) present this as their major example
for using meta-analyses for dealing with
estimating assay sensitivity (T vs. P)
• Want T vs. C, C vs. P, T vs. P
21
CAPRIE STUDY (cont)
• Can we obtain effect of
Clopidogrel vs. Aspirin
• Yes, if we can locate Asprin vs.
Placebo
• Do we believe what we get?
For Aspirin vs. Placebo
Antiplatelet Trialists’ Collaboration
Meta-Analysis
• Meta-analysis of all published and
unpublished unconfounded randomized trials
available March 1990
• Trials identified by literature search, trial
registry and inquiry of investigators and
pharmaceutical manufacturers
• Clear definitions of endpoints
• Well defined statistical methodology
APPROACH
• T vs. C (from Caprie trial)
• C vs. P (from Meta-analysis)
• Obtain T vs. P (from multiplication)
• (T/C) (C/P) = (T/P)
Clopidogrel Vs. Synthetic Placebo Control
Odds Ratios and 95% Confidence Intervals
Overall Patient Population
CAPRIE: Clopidogrel Vs. Aspirin
Meta-Analysis: Aspirin Vs. Placebo
Endpoint
Estimated: Clopidogrel Vs. Placebo
All Strokes, MIs,
or Vascular Deaths
p < 0.000001
All Strokes, MIs
or Death from
Any Cause
p < 0.000001
Vascular
Deaths
p < 0.0016
All Cause
Deaths
p < 0.0045
0.4
0.6
First Drug Better
0.8
1.0
1.2
1.4
1.6
Second Drug Better
• Meta-analysis studies contain very old studies (only
up to 1990), many prior to all of the elaborate medical
interventions (procedures) now routinely provided
• Many of the studies did not have MI, IS or vascular
death as their outcomes (the meta-analysis went back
to original investigators who in turn, had to generate
data). Ever try to get data on something you did not
collect?
• None of the studies used for Clopidogrel with aspirin
comparison had PAD as an entry criteria (PAD
represented 1/3 of Clopidogrel Study)
EFFECT SIZE: Relative Risk Reduction by
Qualifying Condition (ASA vs Clopidogrel)
IS n = 6431
MI n = 6302
PAD n = 6452
Total n =19185
30
20
10
Clopidogrel Better
0
10
20
Aspirin Better
Problems With Historical Controls
• Biases
– Time Biases
• Change in recognition or diagnosis of disease
• Changing disease process
• Change in usual therapy
(Myocardial Infarctions MI, Dx, Tx)
– Selection Biases
• Patients/Health care systems
• Are we really seeing the same patients in historical studies
as those in active control trial?
Problems With Meta-Analyses
So What Is Sponsor to DO?
If we plan to use placebo controlled trials, what
should we require of the historical placebo
trials?
• Same Disease/Conditions?
• Same Population
• Same Dose and Administration Levels of Active
Control C?
• Same Outcomes?
• Combine “All” or “Some (good)” Placebo
Controlled Studies
Still Other Problems With
Meta-Analyses
• What if previous studies had multiple arms? How
to put correctly into meta-analysis?
• What if none of the individual studies achieved
significance?
• What are we to believe from meta-analyses?
• Do we believe the p-levels of the meta-analysis? (I
do not think we should.)
APPROACH 2
NON-INFERIORITY STUDIES
ACTIVE CONTROL STUDIES
NON-INFERIORITY TEST
H0: T-C >= M vs. H1: T-C < M
(Say data are event rates)
T is new treatment
C is positive control
M IS NON-INFERIORITY MARGIN
NON-INFERIORITY STUDIES
APPROACH 2
• SELECT A VALUE OF M THAT MAKES
SENSE
• WANT ASSURANCE THAT ASSAY
SENSITIVITY IS PRESENT (Placebo is
working)
• WANT CONSISTENCY WITH PAST
NON-INFERIORITY STUDIES
Statistical Approach
1. Need Active Control C vs. Placebo P data from
Historical data (C vs. P)
2. Need to test effectiveness of T vs. C
3. Need estimate of fraction of C-P preserved by T
(e.g., (T-P)/(C-P) = M) M=0.5 (C-P)
METHODS EXIST THAT ALLOW TEST TO
BUILD IN NEW AND HISTORICAL DATA
(STATISTICS IN MEDICINE, 2002)
WHAT IS NEEDED FOR 2
• CONFIDENCE INTERVAL IS OFTEN
USED. WANT M=1.11 (SAY) OUTSIDE
UPPER LIMIT OF CONFIDENCE
INTERVAL (M is relative risk)
• FDA ODAC 8/04 (non-small cell lung cancer)
1.0
1.11= M
SOME REALITIES
• Sounds nice
• What happens
Anti-infective Product
No placebo data
• Historical data is not Placebo, but C
• VRE (vancomycin resistant enterococcal)
High dose
Low dose
• MITT 60.0 % (N=65) vs. 46.2 % (N=52)
• Bacteremic
55.6 (N=18) vs. 25.0 (N=16)
• What is M? One trial OK? Any superiority?
ANOTHER EXAMPLE
Respiratory Distress
• Respiratory Distress Syndrome in
Premature Infants
– Treatments
• New Drug
• Comparator
– Outcome
• Survival at 28 Day
Respiratory Distress (cont)
• Survanta versus Sham (two studies one
positive, other negative) All Cause mortality
• Study 1: 8% vs. 23% Study2: 17% vs. 14%
• What is M? .23-.08? .180-.125?
CONSISTENCY
Example Control rate different
from historical
•
•
•
•
•
•
•
Historical Data says C=0.5 and P=0.6
Want T<=0.55
P-C=0.10, M=0.5(0.10) = 0.05
(T-C)/C = 0.05/0.50 = 10%
Data is C=0.30 and T=0.33, T-C=0.03
(T-C)/C = 0.03/0.30 = 10%
IS STUDY A SUCCESS? USE RATIOS?
ANSWER TO CONSISTENCY
• There was consistency
• Differences related to birth weight
Non-Inferiority and Inferiority at the
same time
• Sponsor falls apart
0
M
Non-Inferiority and Superiority
• Sponsor jumps for joy (Sequential test)
0
M
Switching trial design
(Cardiac Stent Trials)
• (1) New drug coated stents, we can do noninferiority study with margin set (15%)
• (2) We can do superiority study with noncoated stent as control
• With first option we have to worry about
evaluating Ms, Effect size and CREEP
• With superiority trial “clean” results
Respiratory Distress
• Compare new surfaxin to another “not so
great” one, but still used in practice
Switching from Superiority to nonInferiority
• HOW CAN WE SWITCH FROM A
SUPERIORITY TEST TO NONINFERIORITY ?
• This is a question thrown at me constantly
Assessing Efficacy NonInferiority and Safety Superiority
• Carotid artery Magnetic Resonance Imaging agent
• Imaging Agents
– Agent N (New) Agent C (Comparator)
• Non-inferiority” Outcome
– Endpoint: agent’s ability to classify correctly patients
with > 25% stenosis (sensitivity)
– Sensitivity of Comparator is .80 or 80%
– Non-inferiority margin M set to 0.10
Assessing Efficacy Non-Inferiority
and Safety Superiority (Cont’d)
• There is a specific adverse event that is
hypothesized to occur less often with New
than with Comparator
– Do we want to make the specific adverse event
rate an additional primary endpoint? WHY
NOT?
Non US STUDIES
• Forced off shore (ethical and other reasons)
The BLOB EFFECT
• Everything is suddenly Non-Inferiority
ALLHAT STUDY
• COMPARISON OF ANTIHYPERTENSIVE MEDICATIONS
(MULTIPLE ARMS)
• NOT A NON-INFERIORITY STUDY
Safety Studies
• Safety studies have become carefully
designed and executed studies
• Should they be non-inferiority studies?
SAFETY STUDIES (PHASE 4)
HISTORICAL APPROACH: NEW RATE > OLD
H01: T-C <= 0 vs. H11: T-C > 0
H02: RR=T/C <= 1 vs. H12: RR=T/C > 1
STUDY POWERED TO REJECT T/C >1.5 (SAY)
SHIFT IS TO MAKING THESE NONINFERIORITY STUDIES
• H0: T-C >= M vs. H1: T-C < M
H0: RR=T/C >= M vs. H1: RR=T/C < M
Safety Studies
• OLD
• NEW
1
M
•
•
•
•
SAFETY STUDY TO NONINFERIORITY STUDY
(QT LONGATION)
Safety issue: drug may cause QT problem
Ho: A/B = 1.0 vs H1: R = A/B > 1.0
Study powered for R > 1.0
When interest in risk fades can we suddenly
say this should be a non-inferiority study?
• Ho:R >= 1.5 vs. H1:R < 1.5 was not
original objective
• If we do not reject Ho is that enough?
Form of Interest and Sample Size
• Ho: p1-p2 >= M
• Ho: p1-p2>=Rp2
• Ho: p1/p2 >= R
• Best Choice does depend on p2 (control rates)
Intent-to-Treat vs. Per-Protocol
• In superiority trials, the primary analysis is
often on intent-to-treat (ITT) population
• Per Protocol (PP) “bigger” differences of
treatments
• In non-inferiority should we use PP?
Intent-to-Treat vs. Per-Protocol
(Cont’d)
• PP as primary not always accepted
– “the ITT analysis is as important as the PP analysis”
– “need to reconcile differences between ITT and PP
analysis”
– Perform “sensitivity” analyses. Results should be similar
in both populations (ROBUSTNESS).
– The Committee on Proprietary Medicinal Products draft
Points to Consider: “…similar conclusions from both the
ITT and PP are required in a noninferiority trial”.
• We ask sponsor to do both (ITT and PP) and
expect to achiev the sam significant result
on both.
• What is the true alpha associated with this?
NEW MAJOR ISSUES
• Missing Data
• Noncompliance
• Interim Analysis
• OUR USUAL LOGIC INCREASES
CHANCE OF ACCEPTANCE OF noninferiority
MORE NEW ISSUES
• Multiple endpoints
• Multiple groups
• Repeated Measures
WHERE ARE WE?
• NON-INFERIORITY TRIALS HAVE
MADE A BIG IMPACT
• They have brought many new problems and
challenges with them