Transcript PROCRIT Template - American Statistical Association

Case Studies of the Use of MetaAnalyses of Drug Safety Questions
in the Regulatory Setting
Jesse A. Berlin, ScD
Johnson & Johnson Pharmaceutical Research
and Development
New Jersey Chapter of the ASA
10 October 2008
2
Outline
 Case studies of the use of historical randomized
trial data to address potential safety concerns
 Emphasis on exploration of patient-level
characteristics as potential effect modifiers
– Some methodologic “heads up”
3
Is it sampling variability?
 Problem: How do we distinguish sampling
variability from “real” variability (possibly)
associated with different effects of treatment in
different subgroups of patients (or with different
dosing algorithms or other specific aspects of
treatment)?
4
Example 1: Galantamine
 Acetylcholinesterase inhibitors (AchEIs) are used as a
standard treatment for Alzheimer’s Disease (AD)
 Galantamine, an AChEI, has been extensively studied
in patients with mild to moderate AD
 Galantamine has also been studied in patients with
AD with concomitant cerebrovascular disease (CVD)
and in patients with VaD (16).
 The benefit is to slow the progress of cognitive decline
(relative to placebo)
5
Safety “signal” for Galantamine in Mild
Cognitive Impairment
 Two 2-year randomized controlled trials
– Individuals with mild cognitive impairment
– Findings replicated in both studies
– 13 deaths versus 1 death
 Higher mortality observed in galantamine-treated
patients, compared with placebo
– Overall mortality rates were low in both groups
 The findings prompted a reevaluation in patients with
dementia
6
Galantamine Methods
 All galantamine trials (J&J or Shire-sponsored) for which
J&J could access data
 Also searched MEDLINE and the Cochrane Controlled
Trials Register (2005) Issue 4
 Trials included were independently reviewed, verified by
two readers, and met the following criteria:
– a) randomized
– b) placebo-controlled
– c) parallel group
– d) blinded
– e) at least one treatment arm with galantamine
7
Meta-analysis of survival in galantamine
randomized trials (6 months duration)
OR with
Study or sub-category
AD trials upto 4 months
GAL-93-01
GAL-CAN-5
GAL-INT -2
GAL-USA-16
AD trials upto 4 months - SUBT OT AL
Fixed trial effect :
Fixed trial effect : Breslow-Day test for homogeneity
N Deaths on T otal Randomized
95%CI
Placebo
1.33
0.34
0.09
3.00
0/ 87, 0.0%
1/ 66, 1.5%
2/125, 1.6%
0/ 69, 0.0%
1/198, 0.5%
0/ 64, 0.0%
0/261, 0.0%
1/ 70, 1.4%
2/279, 0.7%
2/215, 0.9%
3/320, 0.9%
2/138, 1.4%
1/213, 0.5%
4/286, 1.4%
2/275, 0.7%
2/438, 0.5%
8/645, 1.2%
2/256, 0.8%
3/423, 0.7%
8/692, 1.2%
11/390, 2.8%
5/196, 2.6%
5/396, 1.3%
5/396, 1.3%
[0.05;32.95]
[0.01; 8.46]
[0.00; 1.98]
[0.12;74.92]
Galantamine
0.57 [0.12; 2.77]
Chisq= 5.5422 DF= 3 Pr>Chisq 0.1361
Random trial effect :
0.39 [0.06; 2.34]
AD trials equal to 5-6 months
GAL-95-05
GAL-INT -1
GAL-INT -10
GAL-JPN-3
GAL-USA-1
GAL-USA-10
1.01
0.49
1.33
0.54
1.51
0.82
AD trials equal to 5-6 months - SUBT OT AL
Fixed trial effect :
Fixed trial effect : Breslow-Day test for homogeneity
Chisq= 1.2286 DF= 5 Pr>Chisq 0.9421
Random trial effect :
[0.14; 7.25]
[0.07; 3.49]
[0.35; 5.04]
[0.07; 3.84]
[0.16;14.65]
[0.25; 2.76]
0.90 [0.46; 1.76]
0.95 [0.49; 1.83]
AD+CVD and VaD trials equal to 6 months
GAL-INT -26
GAL-INT -6
AD+CVD and VaD trials equal to 6 months - SUBT OT AL
0.44 [0.15; 1.28]
0.49 [0.14; 1.71]
Fixed trial effect :
Fixed trial effect : Breslow-Day test for homogeneity
Chisq= 0.0152 DF= 1 Pr>Chisq 0.9019
Random trial effect :
0.46 [0.20; 1.04]
0.46 [0.21; 1.01]
ALL T RIALS
Fixed trial effect :
Fixed trial effect : Breslow-Day test for homogeneity
Chisq= 8.4661 DF=11 Pr>Chisq 0.6710
Random trial effect :
0.67 [0.41; 1.10]
0.65 [0.41; 1.05]
0.001
0.01
In favor of Galantamine
0.1
1
Odds Ratio
10
In favor of Placebo
100
8
Other Galantamine Analyses
 Nested case-control study of deaths was used to
investigate potential mechanism for the mortality
increase
– Baseline ECG findings
– Comorbidities
– Concomitant medications
 Findings were inconclusive due to small sample
size
 Mortality analyses in press (Feldman et al.; Acta
Neurologica Scandinavica)
 We are doing a large, placebo-controlled study
with mortality as the primary endpoint
9
Example: Erythropoietin-stimulating
agents in oncology
 Is meta-analysis the appropriate tool?
10
“Houston, we have a problem ...”
 A randomized trial of the use of erythropoietin in
breast cancer patients found an increased risk of
mortality in the EPO arm, relative to the placebo
arm. (More on this later.)
 This led to extensive discussions with FDA and,
ultimately, a presentation in May of 2004 to the
Oncologic Drugs Advisory Committee (ODAC)
 An obvious question is, what if we look at prior
studies used in support of the current indication...
25
11
Summary of Survival Over Time
EPO-INT-76
1.0
Probability of survival
0.9
Placebo
Epoetin alfa
0.8
0.7
12-mo mortality
N
n (%)† Censored P value
470 115 (24)
92
.012
469 148 (30)
81
0.6
0.5
12-mo
hazard ratio (95% CI)
1.37 (1.07, 1.74)
0.4
0.3
0.2
0.1
0.0
0
At-risk pts
Placebo 470
Epoetin 469
4
8
12
16
20
24
28
32
36
33
39
7
11
0
3
0
0
Time, mo
453
426
419
390
348
320
168
154
79
74
Vertical line represents end of double-blind phase at 12 mo.
†Based on Kaplan-Meier estimate.
12
Definitions
 Supportive anemia care: studies evaluating the
established indication, treatment of anemia
– Goal is to obtain sufficient rise in hemoglobin
to reduce need for transfusion
 Beyond correction of anemia: investigational
studies treating to higher hemoglobin levels
– Includes most of the recent studies evaluating
effects of erythropoietic agents on cancer
treatment outcomes (response, progression,
survival)
13
Supportive Anemia Care Studies
Analysis
 Combined analysis from all completed,
randomized, double-blind trials of anemic patients
(n = 1,976; 10 trials)
– Mortality hazard ratios
– Thrombotic vascular event (TVE) frequency
 Considerations for this analysis
– Variations in design and patient heterogeneity
– Studies primarily evaluated transfusion
reduction over 12 to 24 weeks
14
Supportive Anemia Care Studies
No Survival Effect Seen in Combined Analysis
Favors
Epoetin alfa
Hazard
ratio
0.89
Tumor type/study
Mixed (non-chemo)
Mortality, n/N (%)
Placebo Epoetin alfa
13/59 (22.0) 13/65 (20)
Favors
placebo
Mixed (non-cisplatin)
1.08
9/76 (12)
10/81 (12)
Mixed (cisplatin)
0.86
9/65 (14)
8/67 (12)
CLL (J89-040)
1.68
6/79 (8)
16/142 (11)
CLL (P-174)
0.42
1/12 (8)
1/33 (3)
Ovarian (EPO-INT-1)
1.58
2/80 (3)
6/164 (4)
MM (EPO-INT-2)
0.15
7/76 (9)
1/69 (1)
Mixed (EPO-INT-3)
1.56
3/65 (5)
9/135 (7)
Mixed (EPO-INT-10)
0.81
22/124 (18)
41/251 (16)
Mixed (PR98-27-008)
1.17
26/165 (16)
31/168 (19)
OVERALL
0.99
(0.76, 1.28)
Test for heterogeneity, P = .66
0.1
1
HR (95% CI) log scale
10
15
J&J was not alone
 Two other companies presented results at the
ODAC meeting: Roche and Amgen
16
Pooled1 Analyses: Progression-free Survival Hazard
Ratios Associated with Aranesp (A) vs. Placebo (P)
by Tumor Type
Ovarian (A = 11/49, P = 3/12)
Breast (A = 16/94, P = 6/23)
GI Other (A = 15/54, P = 4/13)
SCLC (A = 40/60, P = 42/47)
NSCLC (A = 99/146, P = 109/130)
Lymphoma (A = 29/70, P = 32/60)
Myeloma (A = 66/105, P = 57/86)
CLL (A = 26/39, P = 22/28)
Other (A = 23/91, P = 5/22)
Overall (A = 325/708, P = 280/421)
0.1
1.0
Hazard Ratio (95% CI)
1Studies
980297, 20000161, 980291, 990114
10
17
Meta-analysis: Better
Survival
(Roche)
with
Better with
epoetin beta
Category
placebo
Subgroup
0.97
1409
Total
Study
N
1.01
MF4249
116
1.02
MF4250
144
MF4252
MF4253
MF4266
MF4313
MF4321
3.39
54
109
0.59
20
0.37
146
0.61
218
1.02
MF4421
1.29
MF4467
259
343
0.93
Tumor class
Solid
613
1.04
Hematological
791
Other
5
0.2 0.40.6 1
2 3 4 56 10 20 30
Risk ratio
18
Policy implications (1)
 Concerns about the meta-analyses
– Studies are too heterogeneous
 tumor types
 cancer treatment regimens
 other aspects of patient populations and
treatment protocols (2004)
– Meta-analysis may be “masking” signals in
individual studies (2007)
– Short-term follow-up, few events?
– Bias toward the null due to crossover and other
design flaws
19
Policy implications (2)
 Next question to ODAC (2004, paraphrased):
– We are asking for a large study in a single
tumor type, with restricted treatment options to
control extraneous sources of variability. Is it
fair to generalize from a single tumor type to
other sites?
– (this is part of the dilemma)
20
Now here we are in 2008
 Further studies (in head and neck cancer and in
anemia of cancer) have found a “signal”
 Conducted outside the current label (high target
Hb, or not chemotherapy-induced anemia)
 So, we were back at ODAC in 2007 and again in
March, 2008
21
22
Study-level Meta-analysis of Overall Death
46 Controlled CIA Studies (n=12,034; 6505 ESA; 5529 Control)
Study name
Dammacco (INT-2)
* Throuvalas 2000
Del Mastro
Dunphy 1999
* Vadhan-Raj
Oberhoff
Cazzola (Roche)
* EPO-GER-022
INT-3
Vansteenkiste (AMG 980297)
Henry_1995
* Blohmer (AGO-NOGGO)
Ten Bokkel (Roche)
Pirker( AMG 20010145)
Littlewood
Kotasek_2003 (AMG 980291)
Taylor 2005 (AMG 20030232)
Pangalis (P-174)
* EPO-CAN-15
Coiffer
Chang 2005 (EPO-CAN-17)
Witzig 2005
Razzouk 2006 updt
Aapro 2006 (BRAVE)
Mobus
Osterborg 2005
Osterborg 96 (Roche)
Milroy (INT-49)
Savonije 2005
* Thomas (GOG-0191)
Engert 2007 (HD 15 IA)
Thatcher combined
Case
Prozanto (INT-47)
Rose
PREPARE
Leyland-Jones (1 year ITT)
Hedenus 2003 (AMG 20000161)
Grote 2005 (N93-004)
Bamia
INT-I
O'Shaughnessy 2005
Wilkinson 2006
Random Effects Model
Odds
ratio
0.27
0.31
0.31
0.31
0.34
0.35
0.49
0.58
0.61
0.62
0.69
0.73
0.75
0.79
0.81
0.83
0.84
0.89
0.93
0.97
0.97
0.98
1.00
1.02
1.05
1.08
1.10
1.16
1.18
1.19
1.21
1.21
1.22
1.23
1.39
1.40
1.42
1.48
1.54
1.83
1.86
2.94
3.57
1.02
Lower Upper
limit
limit
0.07
1.01
0.01
7.95
0.03
3.17
0.01
8.28
0.01
8.80
0.12
1.04
0.04
5.56
0.35
0.96
0.26
1.41
0.38
1.01
0.34
1.40
0.37
1.44
0.13
4.28
0.52
1.21
0.52
1.28
0.33
2.05
0.45
1.59
0.15
5.36
0.43
2.00
0.35
2.66
0.55
1.73
0.62
1.55
0.14
7.23
0.67
1.53
0.70
1.57
0.69
1.67
0.50
2.44
0.79
1.72
0.73
1.90
0.55
2.56
0.32
4.55
0.30
4.93
0.65
2.30
0.63
2.39
0.64
2.98
0.90
2.15
1.07
1.90
0.97
2.27
0.64
3.72
0.51
6.55
0.50
6.85
0.12 73.93
0.18 70.31
0.93
1.13
0.1
I2 = 0.3%; Fixed Effects Model = Random Effects Model
If BEST LFU data are used, then I2 =0%; Random Effects Model = Fixed Effects Model = 0.97 (0.88, 1.07)
a: 3 studies (Cascinu; Hedenus; Kurtz) reported no deaths in either arm
*: Radiochemotherapy study
0.2
0.5
1
2
5
10
23
Risks of ESAs (from 2008 FDA
presentation)
 6 studies→ statistically significant evidence of ↑ tumor
promotion and/or ↓ survival
– BEST (Breast)*
– ENHANCE (Head/Neck)
– DAHANCA (Head/Neck)
– 161 (Lymphoid Ca)*
– CAN-20 (NSCLC)
– 103 (Anemia of Cancer) (Many tumor types)
 2 studies→ trends of ↑ tumor promotion and/or
– PREPARE (neoadjuvant breast)*
– GOG 191 (cervical cancer)†
* = pts receiving chemo
†=pts receiving chemoRT
↓ survival
24
Study
N
1° Endpoint ESA Adverse
Outcome
(from label)
Chemo
BEST (breast)
161 (Lymphoid)
PREPARE (breast)
939
344
733
12 mo OS
∆ Hgb
RFS, OS
↓ 12 mo OS
↓ OS
↓ RFS*, ↓ OS*
GOG 191 (cervical)
114
PFS
↓ OS*
351
522
LR PFS
LRC
↓ LR PFS, ↓ OS
↓ LRC, ↓ OS*
70
989
QOL
Transfusion
↓ OS
↓ OS
RT
ENHANCE (H/N)
DAHANCA (H/N)
No Chemo or RT
CAN-20 (NSCLC)
103 (Heterogenous)
*=trend
25
GOG-191 Cervical
Results (N=114)
Endpoint
ESA
Control
HR
95% CI
PFS (3 yr)
59%
62%
1.06
0.58, 1.91
Survival (3 yr)
61%
71%
1.28
0.68, 2.42
Local+Distant
Recurrence
33%
27%
HR = ESA : Control
26
Published meta-analysis (shortly before ODAC)
 Bennett et al. found mortality relative risk of 1.10
(95% CI: 1.01, 1.20) 27 Feb 2008 JAMA
 In an earlier presentation (Gleason et al., ASCO
2007, Dr. Bennett as senior author), analyses
separated by “label status” (defined by baseline
hemoglobin <12 versus >12:
– Off-label: RR = 1.14 (1.02, 1.27), I2 = 12.7%
– On-label: RR = 0.97 (0.86, 1.11), I2 = 0.0%
27
Mortality And ESA Responsea: (Landmark At 4 Weeks On Treatment)
All Randomized Studies, ESA-treated Subjects
Study/Contrast
Hazards Ratio
(b)
95% Confidence
Interval (b)
P value (b)
All studies, ESA-treated subjects (N=3750, # of events=1293)
ESA Response: Increase vs. Stable
0.84
(0.72, 0.98)
0.02
ESA Response: Decrease vs. Stable
1.22
(1.03, 1.45)
0.02
ESA Response: Missing vs. Stable
1.13
(0.92, 1.38)
0.24
a) Increase: Increase of >0.5 g/dL;
Stable: change of ≤ 0.5 g/dL;
Decrease: a drop of >0.5 g/dL;
In hemoglobin over the pre-treatment level, independent of transfusion
b) Analyses were based on Cox's proportional hazards model, adjusting for ECOG score,
cancer type, advanced disease, pre-treatment hemoglobin and other baseline variables,
stratified by study.
28
Mortality And ESA Responsea: (Landmark At 4 Weeks On Treatment)
Beyond-anemia-correction studies, ESA-treated subjects
Study/Contrast
Hazards Ratio
(b)
95% Confidence
Interval (b)
P value (b)
Beyond-anemia-correction studies, ESA-treated subjects (N=2089, # of
events=758)
ESA Response: Increase vs.
Stable
0.91
(0.75, 1.12)
0.38
ESA Response: Decrease vs.
Stable
1.50
(1.21, 1.86)
0.00
ESA Response: Missing vs.
Stable
1.19
(0.88, 1.60)
0.26
29
Mortality And ESA Responsea: (Landmark At 4 Weeks On Treatment)
Anemia-correction studies, ESA-treated subjects
Study/Contrast
Hazards Ratio
(b)
95% Confidence
Interval (b)
P value (b)
Anemia-correction studies, ESA-treated subjects (N=1661, # of events=535)
ESA Response: Increase vs.
Stable
0.75
(0.60, 0.95)
0.02
ESA Response: Decrease vs.
Stable
0.92
(0.69, 1.22)
0.56
ESA Response: Missing vs. Stable
1.00
(0.75, 1.32)
0.99
30
31
Methodologic issues (general)
 Standardization of definitions across studies
Endpoints
 Adverse events
 Requires standardization of data collection
across studies! (SPERT)
 What about adjudication?
 Does it depend on the endpoint? (Can we
count deaths or not?)
 Broad versus narrow definitions (numbers versus
signal strength)

32
More methodologic issues
– Long-term versus short-term follow-up
 Non-constant hazards / hazard ratio?
 When is it OK to discard data?
– Studies with small numbers of events?
 When is it OK to discard data?
 Get down-weighted in analysis anyway
– Beware of confounding of study-level
characteristics!!!!!!!
 E.g., low dose study of aspirin in women,
higher dose study in men
33
A Few Words on Endpoint Definitions
 Common view is that more sensitive definitions
– Are more “conservative” by being inclusive
– Increase power by generating more events
 Overly broad inclusion of events can lead to an underestimation
of the true relative risk
– might include events less likely to be related to the true (but
possibly unknown) mechanism of action or
– by their nature, are simply more likely to be misclassified in
clinical trials
 Implications of “non-differential” misclassification in efficacy
versus safety settings?


O’Neill RT. Assessment of safety. Biopharmaceutical Statistics for Drug Development. Peace
KE, ed. New York: Marcel Dekker;1988:543–604.
Proschan MA, Lan KKG, Wittes JT. Statistical Methods for Monitoring Clinical Trials Springer,
New York, 2006
34
Conclusions (1)
 Meta-analysis has valuable applications in
pharmacoepidemiology
– Evaluation of safety using existing randomized
trials
– Evaluation of safety using non-experimental
studies (need more time to show)
35
Conclusions (2)
 There are challenging methodologic issues in the
meta-analysis of safety data
– Rare events, multiplicity, adjudication, …
 Sensitivity analyses should always be performed
– Then more sensitivity analyses should always
be performed
 Use patient-level data when possible