Transcript Assessment of Correlate of Protection in Vaccines

Statistical Challenges and Adaptive
Design Strategies
in Evaluation of New Vaccines
Ivan S. F. Chan, Ph.D.
Late Development Statistics
Merck Research Laboratories
IMS Workshop on
Design and Analysis of Clinical Trials
National University of Singapore, Singapore
October 24-28, 2011
Collaborators
Jonathan Hartzel
Xiaoming Li
Josh Chen
Yanli Zhao
Jenny Sun
2
Outline
Introduction to Vaccines
Clinical Development of Vaccines
– Herpes Zoster Vaccine Example
Opportunities for Adaptive Design Strategy
Issues and Challenges in Adaptive
Designs
– Illustrated with Examples
3
The Ten Greatest Public Health
Achievements of the 20th Century
Vaccination
Motor-vehicle safety
Safer workplaces
Control of infectious
diseases
Decline in deaths
from coronary heart
disease and stroke
MMWR (1999);48:1141
Safer and healthier
foods
Healthier mothers and
babies
Family planning
Fluoridation of
drinking water
Recognition of
tobacco use as a
health hazard
4
What Are Vaccines?
Biological products
Typically for prophylaxis, not treatment
Use antigen or attenuated live virus to trigger
immune responses for disease protection
Administered as a single dose or series with a
potential booster dose
Highly complex immunologic milieu
– Array of humoral and cellular immune responses
5
Examples of Vaccines
Pediatric vaccines
–
–
–
–
–
–
–
Polio
Measles, mumps, rubella (MMR)
Chickenpox (Varivax)
Hepatitis B
Diphtheria, tetanus, pertussis
Rotavirus (infant gastroenteritis, RotaTeq)
Invasive pneumococcal disease (Prevnar)
Adolescents and Adult vaccines
–
–
–
–
–
HPV (cervical cancer, Gardasil)
Meningitis (Menactra)
Influenza
Invasive pneumococcal disease (Pneumovax 23)
Herpes zoster (shingles, Zostavax)
6
Benefits of Vaccines
Direct benefit
– Efficacy in clinical trials
– Risk benefit at individual level
Indirect benefit
– Herd immunity by reducing exposure and
transmission
– Public health implications
7
Types of Immunity
Humoral (antibody-mediated) immunity
– B lymphocytes,
– Plasma cells
– Immunoglobulins (Ig)
IgG, IgM, IgA, IgD and IgE
Cell-mediated immunity (CMI)
– T lymphocytes
– Cytokine/Interleukins
8
Functions of Immunoglobulins
Serve as antibodies
Neutralize viruses and bacterial toxins
– IgG accounts for ~80% of total
immunoglobulin pool
Bind antigen
Prevent or clear first infection
9
Functions of T-cells (CMI)
T lymphocytes (helper cells) stimulate B
cells to produce antibodies
T suppressor (regulatory) cells play an
inhibitory role and control the level and
quality of the immune response (CD4)
Cytotoxic T-cells recognize and destroy
infected cells (CD8)
10
Brief Overview of Clinical
Development of Vaccines
Discovery and Development
of a Successful Drug/Vaccine
YEARS
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0
MARKET LAUNCH
1
DEVELOPMENT
2
2-5
POST-MARKETING SURVEILLANCE
III
CLINICAL TEST (HUMANS)
5 - 10
10 - 20
BASIC
RESEARCH
PHASES
IV
II
I
PRECLINICAL TEST (ANIMALS)
3,000 - 10,000
SYNTHESIS,
EXAMINATION &
SCREENING
QUANTITY OF SUBSTANCES
Source: Based on PhRMA analysis, updated for data per Tufts Center for the Study of Drug Development (CSDD) database.
12
Evaluation of New Vaccines - Safety
Assess local (injection-site) and systemic adverse
experiences
Need a large database, particularly because of
giving vaccines to healthy subjects
Choice of safety parameters depend on type of
disease, population, and route of administration
Need large-scale post licensure study for additional
safety monitoring
13
Evaluation of New Vaccines - Efficacy
Measure the relative reduction (RR) of disease
incidence after vaccination compared with placebos
VE = 1 – RR = 1 – PV/PC
Require a high level of evidence and precision
– Success typically requires showing efficacy greater than
a non-zero (e.g. 20% - 50%) lower bound
– May need a very large study
Need long-term data to assess duration of efficacy
– Historical controls may be used if concurrent controls are
not available
– When is a booster dose needed?
14
Impact of VE Lower Bound
Requirement on Sample Size
Rapid increase of
sample size when VE
lower bound
increases
Example assumes
–
–
–
–
–
5/1000 incidence
90% power
60% true VE
One-sided 2.5% test
1:1 randomization
VE Lower
Bound
0
Total
Sample
Size
16,300
.10
20,800
.20
28,500
.30
43,900
15
Evaluation of New Vaccines - Immunogenicity
Important in understanding the biology
Humoral immunity
– Antibody responses
– priming, first defense
Cell-mediated immunity
– T-cell responses
– prevent virus reactivation, kill infected cells
Identify immune markers that correlates
with disease protection
16
Variability/Stability of Vaccines
Vaccines are biological products that have more
variability in than chemical compound
– Need to demonstrate consistency of manufacturing
Many vaccines contains attenuated live viruses
and will lose potency over time
– E.g., chickenpox vaccine, zoster vaccine
Need to establish a range of potency for
manufacturing and product shelf-life
– Study the safety at the high potency
– Establish efficacy at near-expiry potencies
17
An Example:
Clinical Development of
ZOSTAVAX®
A live-virus vaccine to prevent
herpes zoster (shingles)
Herpes Zoster Is a Consequence of VaricellaZoster Virus (VZV) Reactivation
Dorsal
root ganglion
Spinal cord
Site of VZV
replication
Image courtesy of Courtesy of JW Gnann.
Copyright ©2005 by Merck & Co., Inc., All rights reserved.
19
Ophthalmic Zoster
Courtesy of MN Oxman UCSD/San Diego VAMC.
20
Phase I Study for Dose Ranging
Assess the immune responses of 8 dose levels
– Potencies = 0 (placebo), 2000, 8000, 17000, 19000,
34000, and 67000 PFUs
– Evaluate both antibody and T-cell responses
– N ~40 per group
Results suggested potencies above 17000 PFUs
elicit immune responses
– Some plateau between 34,000 and 67,000 PFUs
– No safety concern
21
Phase II Study for Dose Selection
Assess the immune responses of 2 dose levels
– Potencies = 0 (placebo), 34000, and 50000 PFUs
– Evaluate T-cell responses
– N =398 total (1:3:3 ratio)
Results showed similar immune responses of two
selected potencies
– 1.9 fold higher than placebo (p<0.001)
– Confirmed the plateau observed in phase I study
22
Phase III Study for Efficacy and Safety:
The Shingles Prevention Study (SPS)
(Oxman et al., NEJM 2005)
N = 38,546 subjects ≥60 years of age randomized
1:1 to receive ZOSTAVAX® or placebo
Single dose of vaccine with potency ranging from
18,700 to 60,000 PFU (median 24,600 PFU)
– To bracket end-expiry potency
Average of 3.1 years of HZ surveillance and ≥6month follow-up of HZ pain after HZ rash onset
Conducted by Dept. of Veteran Affairs (VA) in
collaboration with the National Institutes of Health
(NIH) and Merck & Co., Inc.
23
Key Efficacy Endpoints of SPS
HZ incidence
HZ pain burden of illness (BOI)
– Composite of incidence, severity, and duration of
pain
Postherpetic neuralgia (PHN)
– Clinically significant pain persisting for or present
after 90 days of HZ rash onset
Success requires 95% CI lower bound for
vaccine efficacy >25%
24
ZOSTAVAX® Efficacy: HZ Incidence
Percent of Subjects With HZ
Estimate of the Cumulative Incidence of HZ Over Time
by Vaccination Group
6
VEHZ=51%
p<0.001
5
Placebo (n=642)
4
3
2
ZOSTAVAX (n=315)
1
0
0
1
2
3
4
Time Since the Start of Follow-Up (in Years)
Number of subjects at risk
ZOSTAVAX 19254
Placebo
19247
18994
18915
18626
18422
9942
9806
1906
1856
v211p4ACMkm_t2hz_mittv4 Dec. 1, 2005
25
ZOSTAVAX® Efficacy
25%=prespecified lower
bound success criterion
51.3%
95% CI
HZ
66.5%
PHN
61.1%
BOI
0
25
50
Vaccine Efficacy (%)
75
100
26
Phase IV/Market Expansion Studies
After ZOSTAVAX® Approval in 2006
Bridging between frozen and refrigerated
formulation of vaccines
– Allow vaccine to be distributed in markets
without freezer capacity in physician’s office
Concomitant use with flu vaccine
– Desirable for elderly population
High risk populations (HIV,
immunocompromised adults)
Another efficacy trial (~22,000 subjects) in
50-59 year olds
27
Opportunities for Adaptive
Design Strategy
Adaptive Design Strategy
New paradigm of clinical development:
Learn and Confirm
Build adaptive features in clinical trials to
provide a “window” of opportunity to adjust the
trial based on interim data
Accelerate the clinical development by
reducing the time gaps between studies
– E.g., seamless phase II/III trials
Adapt by design and not post hoc as a remedy
29
Features of Adaptive Design Strategy
Optimize dose-response curve
Interim stopping for futility or efficacy
Drop the “losers” (dose selection)
Adjust sample size
Change of populations (enrichment)
Seamless phase II/III trial
Adaptation in individual study and whole
development program
30
Adaptation Benefits
More efficient clinical development
– More accurate dose selection for last phase
development, increase the probability of final success
– Midcourse correction for trials that are off target
– Fewer patients enrolled into ineffective treatment arms
Reduce costs by stopping unsuccessful trials early
– Shift resources to more promising candidates
Reduce development time by combining phases
– Reduce white space between phases
– Reduce overall time of clinical development
31
Opportunities for Adaptive
Designs in Vaccine Development
Availability of immune markers
– Potentially predictive of protection
– Useful for assessing dose responses at phases
I & II
Large and long-duration efficacy trial
– Rare disease incidence
– High bar for success
32
Example 1: Adaptive Phase II/III Study for a new Vaccine
(Dose selection using Immune Marker)
Phase II
Phase III
Part A ( select 1 Dose )
Part B (Pivotal Efficacy)
300 patients
randomized
to each arm
Med. dose
High dose
Control
Interim decision point
Low dose
Additional 6000
patients added
to each arm
Selected
dose
F/up
Control
Non-selected
dose groups
Stop
Dose selection decision based on
safety and immune responses
33
Phase II/III Dose-Selection Trial
Dose selection based on immune responses at
phase II
Efficacy outcome followed at both phases for the
selected dose and control
– Analysis combines data from phases II and III
Overall type I error depends on
– the number of doses at phase II
– correlation between immune marker and disease
outcome
– Sample size at different phases
Type I error can be controlled using closed
testing procedure and/or use of exact test for
efficacy comparison
34
Empirical Type I Error Rate
for 4-Arm Dose-Selection Phase II/III Trial:
Continuous Immune Marker, Binary Disease Outcome,
Exact Test for Efficacy Comparison at 0.025 Level
Sample
Size
Correlation
(marker and
disease)
Disease Incidence Rate (p)
0.2
0.1
0.01
0.004
N1=300
ρ =0.8
0.017
0.023
0.025
0.022
N2=6000
ρ =0.5
0.016
0.022
0.023
0.021
ρ =0.1
0.014
0.019
0.023
0.019
Based on 100,000 simulation runs
35
Example 2: Seamless Phase II/III
Trial Using Group Sequential Design
2-Stage design comparing vaccine vs placebo
for prevention of a rare infection
Stage 1 (phase II): Enroll subjects to evaluate
proof-of-concept (POC) of the new vaccine
– If successful, continue to Stage 2
Stage 2 (phase III): Increase enrollment to
confirm the efficacy of the vaccine
– Build interim analysis strategy for early stopping due
to either futility or overwhelming efficacy
36
Fixed-Endpoint Design of
Vaccine Efficacy Trial
Total # of Subjects (Cases) Required to Conclude Vaccine Efficacy > 
(~90% power, 2% Placebo Infection Rate, one-sided  = 0.025)
True VE
VE Lower
Bound ()
50%
60%
70%
80%
0
6668 (95)
4364 (58)
2998 (37)
2106 (24)
0.10
9336 (133)
5640 (75)
3646 (45)
2458 (28)
0.20
14810 (211)
7746 (103)
4618 (57)
2984 (34)
0.25
19932 (284)
9550 (127)
5346 (66)
3336 (38)
 = success criterion of VE lower bound
VE = Vaccine Efficacy
37
Group Sequential Design (POC and Phase III):
Example Testing Strategy – 25% LB
Efficacy = 0%
Cumulativ
e
Probabilit
y of
Failure
(%)
Cumulative
Probability
of Success
(%)
Efficacy = 25%
Cumulative
Probability
of Failure
(%)
Cumulativ
e
Probabilit
y of
Success
(%)
Efficacy = 70%
Cumulativ
e
Probabilit
y of
Failure
(%)
Cumulati
ve
Probabilit
y of
Success
(%)
Analysi
s
Total
Cases
Observed
Vaccine
Cases
Futility/
Failure
1
20
10
2
35
15
6
85.85
<0.01
59.38
0.12
1.27
27.12
3
52
19
15
98.31
0.02
86.47
0.63
2.65
70.04
Final
69
22
21
99.91
0.09
97.62
2.38
6.51
93.49
Observed
Vaccine
Cases for
Success
58.81
33.47
0.77
POC demonstration at 2nd Interim Analysis (Observed VE >25%)
38
Benefit of Seamless Phase II/III Trial
Using Group Sequential Design
Interim monitoring using “well accepted”
statistical method
Allow early stopping of trial if POC is not
shown
Reduce “white-space” between phase II
and phase III
Reduce overall sample size by leveraging
data from both phases for efficacy
analysis
39
Example 3: 2-Stage Adaptive Design
with Sample Size Re-estimation
Study Objective:
– Vaccine prevention of a rare form of
disease
Main Challenge:
High uncertainty in the incidence rate of
disease
– Rare incidence
– Natural history not well studied previously
– Feasibility and cost
40
Rationale for Adaptive Design Approach
Feasibility issue:
• Uncertainty around incidence rate (IR) of disease
Options to address
Large efficacy study
based on most
conservative IR
Adaptive study with
option for sample
size re-estimation
Stand-alone
epidemiology
study first
Adaptive design with sample size re-estimation
•
•
Establish incidence rate early in study
Inform final sample size requirements for efficacy
demonstration, while maintaining scientific rigor
41
Features of the 2-Stage Design
Evaluate feasibility early (Stage I) based on
incidence rate
– Start with N=2000 with an option to increase up to 7500
Adapt sample size (total case count) requirements
for efficacy demonstration, if necessary, while
maintaining scientific rigor
Build an interim analysis using conditional rejection
probability approach to allow for
– Stopping due to futility
– Stopping due to overwhelming efficacy (Stage II)
– Sample size increase (Stage II)
42
Enroll Stage I
Stop
Follow for cases
Case accrual OK?
Too slow (not feasible)
Yes
Stop
(efficacy)
Good
efficacy
Observed Case
Split in Stage I
Bad efficacy
Not clear
Stop
(futility)
Adapt Targeted
Cases (if needed)
Project timeline to
accrue targeted cases
Acceptable
No additional
subjects required
Use: placebo incidence,
existing cases, number of
subjects still on study
Unacceptable
Add subjects to meet
Timeline
43
Potential Case Accrual Adaptation
 Based on the conditional rejection probability (CRP)
(Muller and Schafer, 2001; EastAdaptTM manual)
o For a group sequential trial, with K looks at the
information fractions t1 ,, t K .
o At interim look L , design parameters for the rest of
the study may be changed, including the maximum
sample size, the number of future interim looks, etc.
o Such changes are permissible provided the
remainder of the trial preserves the CRP of rejecting
H 0 , at look L .
44
Conditional Rejection Probability
 Let Z j be the Wald statistic at look j and z L be its
observed value at look L . Then CRP (  0 ) is defined as:
 0  Pr( H 0 is rejected at look j ( L) | z L , H 0 )
 Muller and Schafer (2001): with any data dependent
changes at an interim look, the overall unconditional type-I
error of the entire trial will be preserved, as long as the
CRP for the modified trial beyond the change point
remains  0 , under the null
 Straightforward to extend to an event-driven efficacy study:
using a single binomial distribution to model the number of
cases in the treatment group
45
Adaptive Study Design - Stage I
17 cases required to have 90% power
Interim analysis at 11 cases once feasibility
demonstrated
Cases in Vaccine
Group
Total
Cases (Out of 11 cases)
Observed
Vaccine
Efficacy
Decision
100%
Stop for efficacy
1-4
≥ 40%
Continue to Stage
II, with potential for
adaptation
≥5
< 40%
Stop for futility
0
11
46
Adaptive Study Design - Stage II
Project number of cases in Stage II to achieve 80% conditional power
Vaccine
Cases in
Stage I
Additional Cases
Targeted for
Stage II
1 or 2
(ε0=0.6563/
0.3438)
6
3
(ε0=0.1094)
12
4
(ε0=0.0156)
24
Interim
analysis at
12/24 cases
Final
analysis at
24/24 cases
Total Cases in Vaccine
Group
Decision
≤ 4 (out of 17 cases)
H0 rejected
> 4 (out of 17 cases)
H0 not rejected
≤ 6 (out of 23 cases)
H0 rejected
> 6 (out of 23 cases)
H0 not rejected
≤ 5 (out of 23 cases)
Stop for efficacy
≥ 9 (out of 23 cases)
Stop for futility
6 or 7 or 8 (out of 23 cases)
Continue for
additional 12
cases
≤ 10 (out of 35 cases)
H0 rejected
≥ 11 (out of 35 cases)
H0 not rejected
47
Evaluation of Design Characteristics
Extensive simulations were conducted to evaluate
the characteristics (study power, study duration,
average enrollment, etc.) under different scenarios:
–
–
–
–
–
–
–
–
Incidence rate
Vaccine efficacy
Initial enrollment and maximum enrollment
Preferred timeline to the interim analysis and final
analysis
Feasibility assessment
Interim analysis strategy
Lost-to-follow-up rate
Randomization ratio
48
Design Characteristics: Interim Analysis at 11 Cases
(2000 initial enrollment, 8000 maximum enrollment)
VE
0%
80%
90%
Incidence
Rate
(/100
pyrs)
Power
(%)
(1-sided
2.5%
level)
Probability
of the
Sample
Size is
Increased
Expected
Total
Number of
Subjects
Average
Study
Duration
(Months)
Expected
Total
Number of
Cases
0.2
1.9
7%
2230
68
0.5
1.9
14%
2310
0.8
1.6
4%
0.2
45
0.3
Percentage of stopping at
(percentage of stopping for efficacy)
Stage I
Stage II
Interim
Stage II
10.6
90% (1%)
9% (1%)
1% (0%)
43
14.5
74% (0%)
23% (2%)
3% (0%)
2050
34
14.3
77% (0%)
21% (1%)
2% (0%)
4%
2140
82
8.5
82% (28%)
18% (17%)
0% (0%)
65
10%
2280
80
12.4
54% (21%)
46% (43%)
0% (0%)
0.4
79
14%
2340
76
15.4
32% (15%)
67% (63%)
1% (1%)
0.5
88
14%
2320
69
17.0
22% (14%)
74% (70%)
3% (3%)
0.8
91
7%
2150
51
18.3
15% (12%)
81% (76%)
4% (4%)
0.2
63
1%
2040
83
7.8
88% (52%)
12% (12%)
0% (0%)
0.3
77
3%
2090
77
10.7
68% (45%)
32% (32%)
0% (0%)
0.4
90
5%
2110
73
13.3
50% (40%)
50% (50%)
0% (0%)
0.5
95
4%
2090
66
14.7
39% (35%)
60% (60%)
0% (0%)
0.8
99
2%
2040
49
15.6
34% (34%)
66% (65%)
0% (0%)
49
Decision Making
Numerous dynamic discussions on study
design were held within statistics, cross
functional areas, and with senior
management
– Primarily focus on scenario planning based on
simulations
Based on these simulation results, broad
agreement on the design strategy and the
selection of key design parameters was
quickly achieved
50
Summary
Adaptive trial designs are attractive and
increasingly popular in an effort to
accelerate clinical development
Good opportunities to use adaptive designs
in vaccine efficacy trials
Statistical guidance and scenario planning
is very important
Trial simulations are important in
understanding the design characteristics
51
Back-up
Antibody Responses to Vaccination
Antibody increases steeply to a plateau
and then decline
Primary responses may have a longer lag
phase and reach a lower plateau than
booster responses
53
Temporal Antibody Responses
Following Primary Immunization
WHO Immunological Basis for Immunization Series, Module 1, General Immunology.
54
Phases of Clinical Trials
Phase I
– Healthy subjects
– PK/PD of drugs
– Modeling and simulations
– Dose ranging for safety and immunogenicity of
vaccines
– Biomarker/assay development
55
Phases of Clinical Trials
Phase II
– Target population
– Dose ranging and dose selection for safety and
efficacy (or immunogenicity for vaccines)
Minimum effective dose
Optimal dose
– Proof of concept (POC) study of efficacy
– Hypothesis generating
56
Phases of Clinical Trials
Phase III
– Confirmatory trial of efficacy and safety
– Demonstration of consistency of the
manufacturing process for vaccines
– Large scale in size
– Last stage before submission for licensure
57
Phases of Clinical Trials
Phase IV
– Post-marketing studies to collect additional data
on safety, efficacy or immunogenicity
– Supports marketing or regulatory commitments
– Expansion to different populations
58
Immune Responses to a new Vaccine
Antibody Concentration (µg/mL) Log10 Scale
200
90 ug
100
30 ug
50
5 ug
Placebo
10
1
Group
7
14
28
56
Days Relative to Vaccination
Number contributing to analysis
84
59