Transcript Optimal cost-effective Go-No Go decisions

Optimal cost-effective
Go-No Go decisions
Cong Chen*, Ph.D.
Robert A. Beckman, M.D.
*Director, Merck & Co., Inc.
EFSPI, Basel, June 2010
Sorry for not being able
to attend in person…
Outline



Introduction
Benefit-cost ratio analysis of POC
design strategies
Discussion
– POC strategy and risk mitigation
– Phase III futility analysis
3
How to fish smartly?
Biology and tech
revolution
Low success
rate and
predictability
Numerous POC
possibilities
Constraint on
societal cost
4
Proof-of-concept trial

A randomized double-blinded
phase II trial with type I/II error rate
(α, β) for detection of Δ based on a
surrogate marker
– Go to Phase III if p-value <α

Choice of (α, β, Δ) is based on a
heuristic argument in practice and is
under-explored in statistical literature
5
Issues to be addressed



What is a more cost-effective sample
size for a POC trial?
What is the optimal bar for a Go
decision to Phase III?
How to re-allocate resource when
there are more POC trials of similar
interest?
6
Benefit-cost ratio analysis

Probability of Go if probability of drug
truly active in the setting is POS
– (1-POS)*α+POS*(1-β)

Expected total sample size (SS)
– Phase II SS + Prob(Go)*Phase III SS

Benefit cost ratio
– Power of carrying active drug (1-β) to
Phase III divided by expected total SS
7
Two designs

Assumptions
– Same Δ of interest, e.g., 50% improvement in
median progression-free-survival
– Sample size for Phase III is fixed at 800 once a
Go decision is made after POC

Two choices of (α, β)
– (10%, 20%) or a ~160 patient/~110 events trial
– (10%, 40%) or a ~80 patient trial but higher
empirical bar (~0.8Δ vs 0.6Δ) for a Go decision
8
Results for comparison
POS
Size
10%
160
Pr(Go) Power Expected
total SS
17%
80%
300
Power/
total SS
0.27
80
15%
60%
200
0.30
20%
160
24%
80%
350
0.23
80
20%
60%
240
0.25
30%
160
31%
80%
400
0.20
80
25%
60%
280
0.21
Smaller trial is more cost-effective. More gains (1530% improvement) can be realized after optimization. 9
Optimal designs under
fixed Phase II resource
POS
(α, β)
0.1
0.2
0.3
(6.7%, 26.7%)
(7.2%, 25.3%)
(8.0%, 23.7%)
Empirical GNG
bar
0.71Δ
0.69Δ
0.66Δ
Assumptions:
1)
Phase II is resourced for (α, β)=(0.1,0.2), which
has an implicit Go bar of 0.6Δ
2)
Relative sample size of Phase II to Phase III is
20% (e.g., 160 pts vs 800 pts)
10
Resource optimization

Budgeted for conducting one 160 patient
POC trial, but has two POC trials of similar
interest
– Consensus is that one has higher POS
(P1=30%) than the other (P2=20%)
– Phase III trial uses same design once Go

Two scenarios for comparison under varying
ratio of POC budget (C2)/Phase III cost (C3)
assuming sample size is proportional to cost
– Two drugs have same value
– The one with lower POS has 50% higher value
11
Optimal resource split
under same value
12
Optimal resource split and
Go bar under same value
13
Optimal resource split and
Go bar under different value
14
Conclusions



Optimal (α, β) can be easily optimized from
benefit-cost ratio analysis
Number of POC trials and respective Go
bars depend on Phase II resource, Phase III
cost, perceived POS and projected value
Similar analysis reveals that a greater Δ has
to be considered when relationship between
surrogate marker and OS is less certain
– Uncertainty is highest in non-randomized trials!
15
POC strategy

More smaller trials, each with a higher
Go bar, are generally preferred
– Adequately powered for larger Δ of true
interest

Similar analysis shows that
simultaneous investigation is more
cost-effective than sequential
investigation
16
Avastin POC strategy
Indication #pts/arm Source
JCO 2003; 21: 60-65
Colon
33-36
RCC
37-40
NEJM 2003; 349: 427-434
NSCLC
Breast
32-34
10-18
JCO 2004; 22: 2184-91
Semin Oncol 2003; 5(suppl
16):117
All trials have 3 arms (low/high dose and placebo)
with 80% power for doubling of PFS
17
PFS effect of recently
approved innovative drugs
Trial
HR (central
review)
HR (local
review)
RCC/Sorafenib
RCC/sunitinib
CRC/panitumumab
BC/lapatinib
BC/Bev+pac vs pac
0.44
0.42
0.54
0.49
0.42
0.51
0.42
0.39
0.59
0.48
18
Pros and cons

Smaller trials
– Easier to accrue, faster to complete, and have
better quality control
– Empirical findings of large treatment effect are
more exciting, and help with Phase III accrual
– More vulnerable to baseline imbalance

More trials
– Reduces missed opportunities (type III error)
and increases overall probability of success
– May inflate program level type I error rate
19
Risk mitigation


Apply minimization or other randomization
techniques for better baseline balance
Follow-up patients for survival after primary
objective for Phase II is achieved
– Initiation of Phase III may be delayed while waiting for
Phase II OS data to mature
– May revisit a Go or No-Go decision as necessary after OS
data become available
– Strength of OS data may be used for setting futility bar of
Phase III trial as appropriate

Revisit those less promising ones from Phase II
after leading indications of same drug achieve
major milestones in Phase III
20
Futility bar at interim for
an ongoing Phase III trial

A hypothetical Phase III trial
– Designed to have 90% power for detection of Δ
in OS before accounting for any futility analysis
– Trial stops for futility at interim if one-sided pvalue > α based on survival info of fraction r
after 50% of the cost is spent

Benefit = overall power adjusted for futility
– May be further adjusted with value as needed

Expected cost = 0.5+0.5*Prob(Go)
– where Prob(Go)=(1-POS)*α+POS*(1-β)
and β satisfies Zα+Zβ=r1/2(Z0.025+Z0.1)
21
Benefit-cost ratio analysis
at 25% info for 30% POS
α (cut- Empirical Overall Expected Power/
off)
bar
power
cost
cost
1
-∞
90.0%
1.00
0.90
0.6
-0.16Δ 88.3%
0.86
1.03
0.5
0
86.8%
0.82
1.06
0.309*
0.31Δ
80.8%
0.74
1.09
0.2
0.53Δ
73.6%
0.69
1.07
22
Optimal futility bars
POS
Info (r)
30%
50%
15%
α (cut-off
p)
45.0%
Empirical
bar
0.10Δ
Overall
power
80.2%
20%
25%
36.8%
30.9%
0.23Δ
0.31Δ
80.3%
80.8%
15%
51.6%
-0.03Δ
82.8%
20%
42.5%
0.13Δ
82.6%
25%
35.5%
0.23Δ
82.8%
Optimal bar decreases with POS and increases with
information. Positive trend is generally required.
23