Transcript John Whitehead talk

Medical and Pharmaceutical
Statistics Research Unit
Meeting on Futility Analysis
London, 11 November 2008
Simple approaches to futility analysis
John Whitehead
Medical and Pharmaceutical Statistics Research Unit
Director: Professor John Whitehead
Tel: +44 1524 592350
Fax: +44 1524 592681
E-mail: [email protected]
MPS Research Unit
Department of Mathematics and Statistics
Fylde College
Lancaster University
Lancaster LA1 4YF, UK
Example: A study in stroke
Patients:
Have suffered an ischaemic stroke no more
than 6 hours earlier
Treatments:
E: Experimental drug, administered for 5
days
C: Placebo
Primary
Response:
Modified Rankin score after 90 days
SUCCESS = score of 0 or 1
FAILURE = score of 2 – 6
Success rates on E and C are pE and pC
MPS Research Unit
Futility meeting
2
Intended analysis
The data can be summarised as
Treatment
E
C
Total
SUCCESS
SE
SC
S
FAILURE
FE
FC
F
Total
nE
nC
n
and will be analysed using Pearson’s c2-test:
2
(O

E)
c2  
E
MPS Research Unit
Futility meeting
3
It can be shown that
2
Z
c2 
V
where
Z
n CSE  n ESC
n
and
V
n E n CSF
n3
To a good level of approximation, Z ~ N(qV, V)
where q is the log-odds ratio
 p E (1  p C ) 
q  log 

 p C (1  p E ) 
MPS Research Unit
Futility meeting
4
Power requirement
E will be claimed better than C if
Zu
where
P  Z  u q  0  12 
and P  Z  u q  qR   1  
and qR represents a clinically worthwhile improvement
Thus u and V must satisfy
 u  1

  2
 V
MPS Research Unit
and
  u  qR V 

  1 
V 

Futility meeting
5
This leads to
 z1 12   z1 
V

qR


where zg is the 100g percentage point of N(0, 1), and to
2
4  z1 12   z1 
n


p 1  p  
qR

2
where
p  12 pE  pC 
MPS Research Unit
Futility meeting
6
Suppose that  = 0.05, 1 –  = 0.90 and pC = 0.45
Attainment of pE = 0.55 would be clinically worthwhile
Then qR = 0.40 and
4
1.960  1.282 

n

  1051
0.5 1  0.5 
0.40

2
A sample size of 1052 could be adopted (V = 65.75)
Run the trial until 1052 patients have been recruited, treated
and followed up to 90 days
MPS Research Unit
Futility meeting
7
The one-stage design
Z
claim E > C
15.893
65.75
MPS Research Unit
Futility meeting
V
8
The two-stage design
claim E > C
Z
u1
u2
continue
V1
V2
V
1
abandon
MPS Research Unit
Futility meeting
9
The futility design
Z
claim E > C
u2
continue
V1
V2
V
1
abandon
MPS Research Unit
Futility meeting
10
Power requirement
E will be claimed better than C if
Z1  1
and
Z2  u2
where
P  Z1 
1
and Z2  u 2 q  0  12 
and
P  Z1 
MPS Research Unit
1
and Z2  u 2 q  qR   1  
Futility meeting
11
Let 2 denote the bivariate standard normal distribution function
2  x1 , x 2 ,   P X1  x1 and X2  x 2 
where
  0  1  
 X1 
 X  ~ N  0 ,   1  

 2
  
Then
  1  qV1 u1  qV2 V1 
P  Z1  1 and Z2  u 2 q   2 
,
,

V2 
V1
V2

2 is the PROBBNRM function of SAS, and so can easily be
evaluated
MPS Research Unit
Futility meeting
12
Thus 1, u2, V1 and V2 must satisfy
  1 u1 V1  1
2 
,
,
  2
 V1 V2 V2 
and
  1  qR V1 u1  qR V2 V1 
2 
,
,
  1 
V2 
V1
V2

There are 2 equations and 4 unknowns, so some constraints
can be imposed
Let us require that V1  12 V2 : futility will be assessed when half
of the information is available
MPS Research Unit
Futility meeting
13
Some feasible designs
1
2.0
1.5
1.0
0.5
0.0
0.5
1.0
1.5
2.0
 u2 = 15.890 in all cases
n2
V2

1
fut0
fut1
cpower
1052
1052
1052
1052
1054
1056
1058
1062
1066
65.75
65.75
65.75
65.75
65.88
66.00
66.13
66.38
66.63
0.0249
0.0249
0.0249
0.0248
0.0248
0.0248
0.0248
0.0249
0.0248
0.902
0.901
0.901
0.901
0.901
0.901
0.900
0.900
0.900
0.364
0.397
0.431
0.465
0.500
0.535
0.569
0.603
0.636
0.0040
0.0052
0.0067
0.0085
0.0106
0.0133
0.0164
0.0201
0.0244
0.206
0.232
0.260
0.289
0.321
0.354
0.389
0.426
0.464
MPS Research Unit
Futility meeting
14
 fut0 = P(Z1  1  q = 0);
fut1 = P(Z1  1  q = 0.40)
 u2 = 15.890 for all designs, compared with u = 15.893 for
a one-stage design
 Search is to three decimal places in u2 and to the nearest
even integer in n2
 No design exists for 1 = 2.5
MPS Research Unit
Futility meeting
15
cpower is the conditional power:
P(Z2  u2  Z1 = 1; q = 0.40)
So, when q = qR:
fut1 is the probability of falling into the hole (of false stopping)
cpower is the probability of getting out of the hole
fut1  cpower is the probability of falling into the hole and
then of getting out of it again (loss of power)
For 1 = 0.0, fut1 = 0.0106, cpower = 0.321 and
fut1  cpower = 0.0034
MPS Research Unit
Futility meeting
16
Preferred design:
1
u2
n2
V2

1
fut0
fut1
0.0 15.890 1054 65.88 0.0248 0.901 0.500 0.0106
 Only two extra patients needed
 50% chance of stopping if no effect
 Very simple futility criterion
 stop if the treatment isn’t working
MPS Research Unit
Futility meeting
17
Recommended design:
1
u2
n2
V2

1
fut0
fut1
0.0 15.893 1052 65.75 0.0247 0.8999 0.500 0.0107
 This is the one-stage design with an added futility look
 Tiny loss of power
 Avoids misunderstanding and difficulties with regulators
 Analyse as if there was no futility analysis  conservative
MPS Research Unit
Futility meeting
18
General recommendations
1.
2.
3.
4.
5.
6.
7.
In lengthy trials in serious conditions, perform an
interim analysis half way through
Abandon the study if the estimated treatment effect is
negative
Ignore the futility rule in the final analysis
Design the futility rule into the protocol, and consider
absolute properties, not conditional ones
You can adjust for prognostic factors and deal with
complicated endpoints
Also useful when there are multiple active treatments
Can check through exact calculation or simulation
MPS Research Unit
Futility meeting
19