Transcript Clinical Trial Methodology for Rare Tumors including Novel

Survival Analysis, Type I and Type II
Error, Sample Size and Positive
Predictive Value
Larry Rubinstein, PhD
Biometric Research Branch, NCI
International Clinical Trial Workshop
ASCO, FLASCA, NCI, ONS
Cordoba, Argentina September 12, 2014
Financial Disclosure
 I have nothing to disclose.
Survival Analysis: The Hazard Ratio
 The basic problem in clinical trials is
measuring and comparing time-to-death (OS)
or time-to-progression (PFS) for experimental
vs. control treatments.
 We can describe the event (death or
progression) rates, over time, by means of the
hazard functions, λe(t) and λc(t), for the
experimental and control treatments.
 We assume that the experimental treatment
decreases the event rate by a constant
proportion Δ: λe(t) = λc(t) /Δ.
The Hazard Ratio Estimator
 Δ = λc(t) / λe(t) = Median (E) / Median (C), the




ratio of the experimental to control median
times-to-event.
Δ ranges from 0 to ∞, and under the null
hypothesis of no treatment effect (H0), Δ = 1.
ln Δ ranges from -∞ to ∞, and under H0, ln Δ = 0.
The estimator ln δ of ln Δ is normal with
variance 1/De + 1/Dc ≈ 4/D (where De, Dc, and D
are the numbers of observed deaths).
The statistic Z = ln δ / √(4/D) is asymptotically
normal with variance ≈ 1.
Type I Error (α, Significance Level)
 We want to limit the type I error (α) – the
probability of calling an experimental
treatment useful when H0 is true (Δ = 1).
 Under H0, Z = ln δ / √(4/D) is normal (0,1).
 If the test statistic Z > 1.645, we are at least
95% confident that Δ > 1 (H0 is false; ln Δ > 0),
by the properties of the standard normal
distribution, and we can reject H0.
 In general, if Z > Zα, the upper α-percentile of
the standard normal, we have (100 - α) %
confidence that Δ > 1 and can reject H0.
Type II Error (β, Power = 1 - β)
 We want to limit the type II error – the
probability of calling an experimental
treatment useless when H1 is true (Δ = 2, for
example). We want 90% power, for example, to
reject H0 in this case.
 We want Z > Zα with 90% likelihood. By the
properties of the normal distribution, we want
the expectation of Z to be Zα + 1.282 (Z has
variance 1 under H1, also).
 In general, to have power 1 – β, we want the
expectation of Z = (ln Δ * √D) / 2 = Zα + Zβ.
Sample Size: Required Number of Events D
 For type I error α and type II error β, we want
the expectation of Z = (ln Δ * √D) / 2 = Zα + Zβ.
 For type I error α and type II error β, the
required D = 4 * (Zα + Zβ)2 / (ln Δ)2.
 Required D increases as α and β decrease:
(Z.1 + Z.1)2 = 6.6; (Z.025 + Z.1)2 = 10.5 (60% larger)
 Phase 3 trials must be larger than randomized
phase 2 trials, in part, because of the smaller
type I error required.
Sample Size: Phase 2 vs. Phase 3 Trials
 Generally, phase 3 trials have an OS primary
endpoint, while randomized phase 2 trials have
a PFS primary endpoint, since it is generally
much shorter and the targeted Δ can be larger.
 The required D drops as the target hazard ratio
Δ increases. For example, for α = .025, β =.1:
Target Hazard Ratio Δ
Required Number of Events D
1.4
372
1.6
191
1.8
122
2.0
88
Phase 2 and 3: Positive Predictive Value
 Positive predictive value: the likelihood that a




treatment is effective when the trial is positive
PPV = pr(H1) * (1–β) / (pr(H1) * (1–β) + pr(H0) * α)
For a likely phase 2 scenario, the pr(H1) = .1,
and α = β = .1, we can calculate that PPV = .5.
For a likely phase 3 scenario, pr(H1) = .5, α =
.025, β = .1, we can calculate that PPV = .97.
Randomized phase 2 trials with PFS endpoints
efficiently screen out most of the ineffective
treatments and enable reliable phase 3 trials
with OS endpoints (and equipoise at initiation).
References
1. Simon: Optimal two-stage designs for
phase II clinical trials. Control Clin Trials
10:1, 1989.
2. Rubinstein, Korn, Freidlin, Hunsberger,
Ivy, Smith: Design issues of randomized
phase II trials and a proposal for phase II
screening trials. J Clin Oncol 23:7199,
2005.
3. Rubinstein: Therapeutic studies.
Hematology/Oncology Clinics of North
America 14:849, 2000.