Transcript (1) = a

Chapter 10
Data Monitoring, Monitoring
Committee Function &
Statistical Methods
1
Some References
• Texts/Chapters
1.
Friedman, Furberg & DeMets (1998) 3rd edition,
Fundamentals of Clinical Trials, Springer-Verlag, NY, NY
2.
Pocock (1983) Clinical Trials, Wiley.
3.
Ellenberg S, Fleming T and DeMets D: Data Monitoring Committees
in Clinical Trials: A Practical Perspective. John Wiley & Sons, Ltd.,
West Sussex, England, 2002.
4.
Jennison C and Turnbull B (2000) Group Sequential Methods with
Application to Cinical Trials. Chapman & Hall, NY.
5.
DeMets DL (1998) Data and Safety Monitoring Boards. In:
Encyclopedia of Biostatistics. John Wiley and Sons, West
Sussex, England, Vol. 2, pp. 1067-71.
6.
DeMets and Lan. The alpha spending function approach to interim
data analysis. In, Recent Advances in Clinical Trials Design and
Analysis. Kluwer Academic Publishers, Boston, MA, 1995.
2
Some References
•
Review Papers
1. Greenberg Report:Organization, review, and administration of
cooperative studies. Controlled Clinical Trials 9:137-148, 1988.
2. DeMets and Lan: (1994) Interim analyses: The alpha spending
function approach. Statistics in Medicine, 13(13/14):1341-52, 1994.
3. Lan and Wittes. The B-value: A tool for monitoring data.
Biometrics 44:579-585, 1988.
4. Task Force of the Working Group on Arrhythmias of the European
Society of Cardiology: The early termination of clinical trials: causes,
consequences, and control. Circulation 89(6):2892-2907, 1994.
5. Fleming and DeMets: Monitoring of clinical trials: issues and
recommendations. Controlled Clin Trials 14:183-97, 1993.
6. DeMets, Ellenberg, Fleming, Childress, et al: The Data and Safety
Monitoring Board and AIDS clinical trials. Controlled Clin Trials
16:408-21, 1995.
7. Armstrong and Furberg: Clinical trial data and safety monitoring
boards: The search for a constitution. Circulation 1, Sess:6, 1994.
3
Data Monitoring
Rationale
1. Ethical
2. Scientific
3. Economic
4
A Brief History
• A 40-year history
• Greenberg Report (1967)
• Coronary Drug Project (1968)
• NIH Experience and Guidelines
• Industry and ICH Guidelines
• Department of Health & Human Services
Policy (Shalala, 2000)
5
Greenberg Report
Recommendations
• Develop a mechanism to terminate early if
–
–
–
–
Question already answered
Trial can’t achieve its goals
Unusual circumstances
Hypothesis no longer relevant
• Sponsor decision to terminate should be
based on advice of external committee
6
Coronary Drug Project (CDP)
• References
– Design (Circulation, 1973)
– Monitoring Experience (CCT, 1981)
– Major Outcome (JAMA 1970, 1972, 1973, 1975)
• Tested several lipid lowering drugs in
post MI patients
• Multicenter study
• Mortality as primary outcome
• Began recruitment in 1965
7
Coronary Drug Project
• First trial to benefit from Greenberg Report
• Policy Advisory Board
– Senior Investigators, External Experts, NIH
– Initially reviewed interim data
• Data Coordinating and Statistical Center
• Safety Monitoring Committee formed (1968),
after trial was underway
8
Early NHLBI CT Model
Funding
Agency
Policy Advisory
Board
Data and Safety
Monitoring
Board
Data Coordinating Center
Data Management
Statistical Analysis
Steering
Committee
Multiple
Clinics
Central
Lab(s)
Working
Committees
9
NHLBI CT Model
Funding
Agency
Data Monitoring
Committee
Steering
Committee
Coordinating
Data Center
Clinics
Central
Working
Lab(s) Committees
10
NIH DMC Activity
• Ref: Statistics in Medicine (1993)
• CDP (Coronary Drug Project) became model for National
Heart, Lung, and Blood Institute (NHLBI)
– heart, lung, blood disease trials
• National Eye Institute (NEI) (1972)
– Diabetic Retinopathy Study
• National Institute Diabetes, Digestive and Kidney (NIDDK)
– Diabetes Complication and Control Trial (1980)
• National Cancer Institute (NCI)
– Prevention Trials, Cooperative Group Therapeutic Trials
• National Institute Allergy and Infectious Disease (NIAID)
– AIDS Clinical Trial Group (ACTG) (1986)
11
Industry/FDA/ICH
• Industry sponsorship of RCTs expanded dramatically
since 1990 in several disease areas
(e.g. cardiology, cancer, AIDS)
• Industry use of DMCs growing as well
• FDA 1989 guidelines very brief mention of data
monitoring and DMCs
• International Conference on Harmonization (ICH)
– ICH/E9
 Section 4.5
 Section 4.6
– ICH/E6
Interim Analyses
Independent DMCs
12
Independent DMCs
When are they Needed?
• Department of Health and Human Services Policy
– Shalala (NEJM, 2000): All NIH FDA trials must
have a monitoring plan, for some a DMC may
be required
• NIH policy (1998)
– all sponsored trials must have a monitoring
system
– safety, efficacy and validity
– DMC for Phase III trials
• FDA guidelines (Nov 2001)
13
Need for Independent DMCs
• Phase I Trials (dose)
– Monitoring usually at local level
• Phase II Trials (activity)
– Most monitoring at local level
– Some randomized, blinded, multicenter Phase II
trials may need IDMC
• Phase III & IV (effectiveness, risk, benefit)
– Most frequent user of IDMC
• Structure of monitoring depends on risk
(e.g. Phase I-IV)
14
Data Monitoring Committee
• FDA suggests a need for an
Independent DSMB for
– Pivotal Phase IIIs
– Mortality or irreversible
morbidity outcome
15
Industry-Modified NIH Model
Steering
Committee
Pharmaceutical
Industry Sponsor
Regulatory
Agencies
Independent
Data Monitoring
Committee (IDMC)
Statistical
Analysis Center
Data Management
Center
(Sponsor or CRO)
Clinical Centers
Patients
Central Units
(Labs, …)
Institutional
Review Board
DMC Relationships
and Responsibilities
• Patients
• Study Investigators
• Sponsor
• Local IRBs
• Regulatory Agencies
17
Early Administrative Analysis
DMC and Executive Committee
1.
Recruitment/Entry Criteria
2.
Baseline Comparisons
3.
Design Assumptions
a.
Control only
b.
Combined groups
18
Design Modifications
1.
Entry Criteria
2.
Treatment Dose
3.
Sample Size Adjustment
4.
Frequency of Measurements
19
DMC Data Review
Interim Analysis
1.
Recruitment
2.
Baseline Variables
-Eligibility
-Comparability
3.
Outcome Measures
-Primary
-Secondary
4.
Toxicity/Adverse Effects
5.
Compliance
6.
Specified Subgroups
20
DMC Recommendations
1. Continue Trial / Protocol
Unmodified
2. Modify Protocol
3. Terminate Trial
21
Reasons for Early Termination
1. Serious toxicity
2. Established benefit
3. Futility or no trend of interest
4. Design, logistical issues too
serious to fix
22
DMC Decision Making
Process Complex (1)
• Recruitment Goals
• Baseline risk and comparability
• Compliance
• Primary and secondary outcomes
• Safety
23
DMC Decision Making
Process Complex (2)
• Internal consistency
• External consistency
• Benefit/Risk
• Current vs future patients
• Clinical/Public impact
• Statistical issues
24
DMC Decision Making Role
• DMC makes recommendations, not final decisions
• Independent review provides basis for DMC
recommendations
• DMC makes recommendations to
– Executive Committee who recommends to
sponsor, or
– Sponsor
• DMC may, if requested, debrief Executive
Committee and/or sponsor
• Rarely are DMC recommendations rejected
25
DMC Meeting Format
• Open Session
– Progress, blinded data
– Sponsor, Executive Committee, DMC, SAC
• Closed Session
– Unblinded data
– DMC, SAC
– Sponsor Rep? (Not recommended)
• Executive Session
– DMC only
• Debriefing Session
– DMC Chair, Sponsor Rep, Executive
Committee Rep
26
DMC Relationships
• Regulatory Agencies (e.g. FDA)
– Could perhaps brief DMC about specific
concerns at Open Session
– Should not participate in DMC Closed
Sessions
– Should be briefed about DMC
recommendations/decisions ASAP
following Executive Committee
27
DMC Membership
• Monitoring is complex decision process
and requires a variety of expertise
• Needed expertise
– Clinical
– Basic science
– Clinical trial methodology
– Biostatistics
– Epidemiology
– Medical ethics
• Helpful expertise
– Regulatory
• Some experience essential
28
DMC Confidentiality
• In general, interim data must remain
confidential
– DMC may rarely release specific/limited interim
data (e.g. safety issue)
• Members must not share interim data with
anyone outside DMC
• Leaks can affect
–
–
–
–
Patient Recruitment
Protocol Compliance
Outcome Assessment
Trial Support
29
DMC Liability
• Recent events (eg Cox-IIs, Vioxx) have raised the
potential for litigation (訴訟)(Vioxx or COX-IIs
(painkillers) can raises the risk of heart attack,
stroke and death and were withdrawn from the
market)
• Members have been gotten a subpoena (傳票 )
• DMC Charters (設立) for industry trials now often
cover indemnification clauses (賠償條款)
• No indemnification yet for NIH trials
30
DMC Needs “On-Line”
Data Management and Analysis
• DMC reluctant to make decisions on “old
data”
• Minimize data delay and event verification
• Be prepared from start
• Focus on key variables, not complete
case reports (delays can be problematic)
31
Levels of Independence
• Totally Inhouse Coordinating Center
• Internal DM, Internal SAC, External DMC
• Internal DM, External SAC, External DMC
• External DM(e.g. CRO), External SAC,
External DMC
32
DMC Summary
• NIH Clinical Trial Model - long history
of success
• Adaptation for industry can be made
• SC, DMC, SAC or DM are critical
components
• Independence of DMC essential
• Best way to achieve this goal is for
external SAC and external DMC
33
Data Monitoring Process
1. DMC and the decision process
2. A brief introduction to statistical
monitoring methods
a. Group Sequential
b. Stochastic Curtailment
3. Examples
Ref: BHAT, DeMets et al.
Controlled Clin Trials,1984
34
Decision Factors
1.
Comparability
2.
Bias
3.
Compliance
4.
Main effect vs. Potential side effects
5.
Internal Consistency
a. Outcome measures
b. Subgroups
c. Centers
6.
External Consistency
7.
Impact
8.
Statistical Issues/Repeated Testing
35
Beta-blocker Heart Attack Trial
(BHAT)
Preliminary Report. JAMA 246:2073-2074, 1981
Final Report. JAMA 247:1707-1714, 1982
Design Features
Mortality Outcome
3,837 patients
Randomized
Men and women
Double-blind
30-69 years of age
Placebo-controlled
5-21 days post-M.I.
Extended follow-up
Propranolol-180 or 240 mg/day
36
BHAT
Accumulating Survival Data
Date Data Monitoring
Committee Meeting
Propranolol Placebo
Z(log rank)
May 1979
22/860
34/848
1.68
Oct 1979
29/1080
48/1080
2.24
March 1980
50/1490
76/1486
2.37
Oct 1980
74/1846
103/1841
2.30
April 1981
106/1916
141/1921
2.34
Oct 1981
135/1916
183/1921
2.82*
June 1982
* Data Monitoring Committee recommended termination
37
Beta-Blocker Heart Attack Trial
October 1, 1981
LIFE-TABLE CUMULATIVE MORALITY CURVES
38
Beta-Blocker Heart Attack Trial
Baseline Comparisons
Propranolol
(N=1,916)
Placebo
(N=1,921)
Average Age (yrs.)
55.2
55.4
Male (%)
83.8
85.2
White (%)
89.3
88.4
Systolic B.P.
112.3
111.7
Diastolic B.P.
72.6
72.3
Heart rate
76.2
75.7
212.7
213.6
57.3
56.8
Cholesterol
Current smoker (%)
39
Beta-Blocker Heart Attack Trial
Total Mortality
(Average 24-Month Follow-Up)
Propranolol
Age
Sex
Race
Placebo
30-59
5.9%
7.1%
60-69
9.6%
14.4%
Male
7.0%
9.3%
Female
7.1%
10.9%
White
6.7%
9.0%
Black
11.0%
15.2%
40
Beta-Blocker Heart Attack Trial
Total Mortality
(Average 24-Month Follow-Up)
Propranolol Placebo
Risk Group I
13.5%
16.9%
Risk Group II
7.8%
11.4%
Risk Group III
5.2%
7.1%
41
DMC Interim Analysis
• Ethical, scientific and financial
reasons
• Repeated analysis of accumulating
data causes a statistical problem
42
Data Monitoring
43
Classical Sequential Analysis
• Observations are taken sequentially
• After each observation
– Decide whether to stop sampling (one
group is significantly better, or worse,
than the other)
– Or take another observation
• Originally developed by Wald (1947)
• Applied to the clinical trial by
Armitage (1975)
44
Why Sequential Analysis?
(Armitage, 1975)
• Data reduction
• Estimation with desired precision
• Medical ethics
45
Repeated Significance Tests
• Assume X 1 , X 2 ,  ~ N(, 1)
• Let S n = X 1 +  +X n
• N is the maximum sample size
• Testing H 0 :  = 0 vs H A :   0
• Nominal significance level is 0.05
46
Repeated Significance Tests
• For each n  N , we assess if | Sn | 
1.96 n
– Stop sampling and reject H0 at the first
n  N , if any, such that | Sn |  1.96 n
– Otherwise, stop sampling at N and do
not reject H0
47
Probability of Type I Error
• *N = P {| Sn |  1.96 n for some n 
N |H0 }
• By the law of the iterated logarithm,
eventually reject H0 when in fact it is
true
• *N might be large for some N
48
The Type I Error Probability when the
Maximum Number of Observations is N
N
1
2
3
4
5
10
15
20
25
30
*N
0.050
0.083
0.107
0.126
0.142
0.193
0.225
0.248
0.266
0.280
49
The Required Critical Values and
Nominal Level Giving a Type I Error
Probability 0.05 for Various Values of N
N
1
5
Critical Value
1.96
2.42
Nominal Level
0.050
0.015
10
15
20
2.56
2.64
2.68
0.010
0.008
0.007
50
100
200
2.80
2.88
2.96
0.005
0.004
0.003
50
Group Sequential Procedures
• Repeated significance tests after every
observation are not easy to conduct
• Apply the significance test at longer
intervals
• Compute summary statistic at each
interim analyses, based on additional
group of new subjects (events)
• Compare statistic to a conservative critical
value such that α=0.05 overall
51
Group Sequential Procedures
Boundaries
• Haybittle-Peto (1971,1976)
• Pocock (1977)
• O’Brien-Fleming (1979)
• Lan-DeMets (1983)
• Slud-Wei (1982)
52
53
Group Sequential Boundaries
54
Pocock's boundary
N
 = 0.05
 = 0.01
1
1.96
2.58
2
2.18
2.77
3
2.29
2.87
4
2.36
2.94
5
2.41
2.99
55
Lan-DeMets Procedure
Criticism of “classical” group
Sequential procedure
• Number of interim analyses must be
specified in advance
• Equal increments
56
Lan-DeMets Procedure
• Specify *(t) spending function
• *(t) defines rate at which Type I error
is spent where t is the proportion of
information accumulated by calendar
time tc
• 0t1
• *(t) increasing,
*(0) = 0
*(1) = 
57
Lan & DeMets Procedure
• The function * is “arbitrary”
• Examples:
0
 t   
2  2 z 2
*
1

t

if t  0
if 0  t  1,
where z/2 is denoted such that (z/2) =

1- ,
2
and 2*(t) =  log{1 + (e - 1)t}
58
Information and Calendar Time
t =proportion of information
accumulated by tc
Example: Immediate Response
X1,X2,...,Xn,...,XN
Y1,Y2,...,Yn,...,YN
|
tc
t = 2n / (2N) = n / N
59
Information and Calendar Time
Example: Failure time (e.g., logrank)
Pr[patient dies  tc ]
t
Pr[patient dies  Tc ]
number patients dead  tc

number patients dead  Tc
60
Lan & DeMets Procedure
• Assume X1 , X2 , . . . ~ N( , 1)
• Testing H0 :   0 vs H1 :  > 0
• Let Zi be the accumulated test statistic at
calander time i at which the information
time is ti .
• Find boundary values Ci such that
P ( Z1  C1 ) = *( t1 ),
P ( Z2 < C1 , Z2  C2 ) = *( t2 ) - *( t1 ), . . . .
61
Boundary Crossing Probability
E.g., K = 5,  = 0.025
Pocock
OBF
1. P {Z1 > C1} =
Upper Boundary
C1
C2
C3
C4
C5
(2.41, 2.41, 2.41, 2.41,
2.41)
(4.56, 3.23, 2.63, 2.28,
2.04)
Pocock OBF
0.0079
(0.000)
2. P {Z1 > C1 or Z2 > C2} =
0.0079 + 0.0059 = 0.0138
(0.0006)
3. P {Z1 > C1 or Z2 > C2 or Z3 > C3} =
0.0138 + 0.0045 = 0.0183
(0.0045)
4. P {Z1 > C1, ..., Z4 > C4} =
0.0183 + 0.0036 = 0.0219
(0.0128)
5. P {Z1 > C1, ..., Z5 > C5} =
0.0219 + 0.0031 = 0.0250
(0.0250)
62
63
* (t2) - * (t1)
64
Examples of *(t)
Approximates
1.
 * ( t )  2  2  [ Z / 2 t ]
OBF
2.
2 *(t) =  ln { 1 + (e - 1)t }
Pocock
3.
3 *(t) = t
1
• Comparison of Boundaries( = .025, N = 5)
Values
C1
C2
C3
C4
C5
1.
OBF
1*(t)
4.56
4.90
3.23
3.35
2.63
2.68
2.28
2.29
2.04
2.03
2.
Pocock
2*(t)
2.41
2.44
2.41
2.43
2.41
2.41
2.41
2.40
2.41
2.38
3.
3*(t)
2.58
2.49
2.41
2.34
2.28
65
BHAT GSB
66
Cardiac Arrhythmia
Suppression Trial (CAST)
• Ref: NEJM 321(6):406-12, 1989
• Cardiac arrhythmias associated with
increased risk of sudden death
• New class of drugs (eg, encainide,
flecanide) suppressed arrhythmias
• CAST designed to test effect on sudden
death
67
CAST GSB
•  spending function approach
•
*(t) = ½  t

t<1
t=1
• for benefit  = 0.025
• Used symmetric  = 0.025 boundary for harm
68
CAST Interim Data
Sudden Death
Time
Placebo
Drug
LogRank
ZL
ZU
9/01/88
5/576
22/571
-2.82
-3.18 3.01
3/30/89
9/725
33/730
-3.22
-3.04 2.71
Initially expected 100 events/arm
69
CAST Sequential Boundaries
70
Stochastic Curtailed Sampling
• During study, whether the current
trend in the data can lead to the
acceptance or rejection of H0 ?
• Group sequential methods focus on
existing data
• Curtailed sampling in addition
considers the data which have not
yet been observed
• Lan, Simon and Halperin (1982)
71
Example
• H0 :  = 0.5 (Prob(Heads)) vs. HA :   0.5
• Flip coin 400 times
• S=total number of heads
• Reject H0 if |z|  1.96, where
S  200
Z
400  0.5  0.5
or when |S - 200|  20
After 350 coin flips and 220 heads, we
know for sure we will reject H0 .
72
Stochastic Curtailing
• Let Z(T)=statistic at end of trial
Z(t)=current value at time t
R=rejection region
• P [ Z(T)  R| H0 ] = 
• P [ Z(T)  R | HA ] = 
or P [Z(T)  R| HA ] = 1 - 
73
Stochastic Curtailing
• Lan, Simon, Halperin (1982)
reject when P [Z(T)  R| H0 , Z(t)] = 0
shows very positive trend
accept when P [Z(T) R | HA , Z(t)] = A
shows negative trend
• P [Type I error]   / 0
• P [Type II error]   / A
74
Example
• Population 1: X ~ N(mx , s2 )
Population 2: Y ~ N(my , s2 )
H0 : mx = my
HA : mx > my
• For design
HA : mx - my = 0.1, s2 = 1,  = 0.05, 1-  = 0.8
Need 1250 subjects per group
X 1250  Y1250
Z
1 1250  1 1250
Reject if Z  1.645
75
Example (continued)
• During Study
No
X Y
I
250
0.113
1.26
II
500
0.125
1.98
III
750
0.122
2.26
IV
1000
0.12
2.68
V
1250
?
?
Z
76
Example (continued)
• Conditional Probability
PZ  1.645 | x  y  0.12, D  m x  m y   ?
D = 0.12
D = 0.03
D = 0.00
P  0.999
P  0.98
P  0.95
77
B-Value:
A Method for Computing Conditional Power
Lan & Wittes (1988) Biometrics
Let t = n/N (or d/D)
Z(t) = current standardized statistic
B (t )  Z (t ) t
E(B(t))  θt ,
• Now Z(1) = B(1) and
Z(1)  Z (t ) t  ( Z (1)  Z (t ) t )
(= observed + remaining)
78
Visual Aid
H0:  = 0
HA: e.g.  = ˆ
B(1)
B(t)
79
Conditional Power
• P[Z(1)  Z | Z(t), )]


 1   Z  Z (t ) t   (1  t ) / (1  t )

80
Conditional Power
  E( Z( t  1))
1. Survival
D = total events
  D / 4 Ln (C / T )
2. Binomial
N = total sample size
PC  PT
( PC  PT ) N / 4


2 p q /( 2 / N )
pq
( PC  PT ) N
 1/ 2
pq
81
Conditional Power (2)
3. Means
N = total sample size
 m C  mT 

 N /4
 s 
 m C  mT 
 1/ 2 
 N
 s 
82
Example: BHAT
• Expected Deaths
D = 398
• Observed Deaths
183 Placebo
d = 318
135 Propranolol
D = d + 80 = 398
Zd = 2.82
t = 318/398 = .80
• Observed logrank
• Compute Conditional Power under H0
B(.8)  2.82 0.8  2.52
1.96  2.52 
H 0 : Cond Power  1 -  
 with   0
.2


= 1 -  {- 1.25}
= 0.89
83