Transcript Analyzing a viral load endpoint in an HIV vaccine trial

Demonstrating that an HIV Vaccine Lowers the
Risk and/or Severity of HIV infection
D.Mehrotra1*, X.Li1, P.Gilbert2
1Merck
2Fred
Research Laboratories, Blue Bell, PA
Hutchinson Cancer Research Center & Univ. of Washington, Seattle, WA
Adapted from Mehrotra’s talk at ENAR, Austin, TX
March 21, 2005
Biostat 578A Lecture 6
Outline
•
•
•
•
•
HIV vaccine POC efficacy trial
BOI vs. Simes’ method
Adjusting for selection bias
BOI vs. adjusted Simes’ method
Concluding remarks
2
HIV Vaccine POC Efficacy Trial
• Design
- Randomized, double-blind, multinational trial
- MRKAd5 gag/pol/nef versus placebo (1:1)
- 1500 subjects at high risk of becoming HIV+
- Continue until 50 events (HIV infections) accrue
• Co-Primary Endpoints
- HIV infection status (infected/uninfected)
- Viral load set-point (mean of log10 HIV RNA at 2 and
3 months after HIV+ diagnosis)
Note: lowering of viral load set-point will presumably
prevent or delay the onset of AIDS
3
POC Efficacy Trial (continued)
• Null Hypothesis: Vaccine is same as Placebo
VE = 0 and  = 0
• Alternative Hypothesis: Vaccine is better than Placebo
VE > 0 and/or  > 0
true HIV infection rate for VACCINE
VE = 1 
true HIV infection rate for PLACEBO
 = true difference in mean viral load set- point among
infected subjects [placebo – vaccine]
• Proof-of-concept (POC) is established if the composite
null hypothesis is rejected at  = 5%
4
POC Efficacy Trial: Data Set-Up
Number randomized
Number HIV infected
Proportion infected
Viral load set-points of
infected subjects
(log10 copies/ml)
Vaccine
Placebo
Nv
nv
nv
Nv
Np
 y1( v ) 





 yn( vv ) 
 y1( p ) 


  
 yn( p ) 
 p 
np
np
Np
5
Establishing POC: BOI vs. Simes’ Method
 Burden-of-Illness (BOI) (Chang, Guess, Heyse, 1994)
nv
Difference in BOI per subject: T 
Let Z BOI
T  E T | nv  n p , H 0 

Var (T | nv  n p , H 0 )
y
i 1
np
(v)
i
Nv

( p)
y
 i
i 1
Np
(see CGH) [unconditional test]
Reject null if p  value  Pr Z  Z BOI | H 0   .05
 Simes’ method (Simes, 1986)
p1  p-value for infection endpoint [unconditional test]
p2  p-value for viral load endpoint [conditional test]
Reject null if p  value  min(max( p1 , p2 ), 2 min( p1 , p2 ))  .05
(Similar results with Fisher’s combined p-value method)
6
Power (%) to Reject the Composite Null Hypothesis
(assuming  = 1 log10 copies/ml)
VE = 0%
VE = 30%
100
80
80
60
60
40
40
20
20
0
0
Power (%)
100
10
30
50
70
90
10
30
50
70
90
VE = 60%
100
Power (%)
80
BOI
Simes'
60
40
20
0
*Viral load reduction is 1 log10 in all plots
10
30
50
70
Number of HIV Infections
90
7
Adjusting for Selection Bias
• Simulations led us to choose Simes’ over BOI for
the POC efficacy trial. However …
• Test for viral load component in Simes’ method:
- Is restricted to subjects that are selected
based on a post-randomization outcome (HIV
infection)  can suffer from selection bias.
- Assesses mixture of (i) causal effect of vaccine
and (ii) effect of variables correlated with VL
that are unevenly distributed among the
infected subgroups.
References
Rubin (1978), Rosenbaum (1984), Robins and Greenland
(1992), Frangakis and Rubin (2002)
8
Adjusting for Selection Bias (continued)
• Proposed approach
- Adjust the viral load test for plausible levels of
selection bias such that rejection of the null
hypothesis becomes harder.
- If the adjusted test is significant, then we have
robust evidence of a causal vaccine effect.
Hudgens, Hoering, Self (Statistics in Medicine, 2003)
Gilbert, Bosch, Hudgens (Biometrics, 2003) [GBH]
Mehrotra, Li, Gilbert (Biometrics, 2006)
• Adjustment is derived via the principal stratification
framework of causal inference (Frangakis and Rubin,
Biometrics, 2002)
9
Adjusting for Selection Bias (continued)
• Subjects infected under placebo {Si(p)=1} partition into
the protected and always-infected principal strata
Principal Stratum
Potential infection
outcome under Z
Potential VL outcome under
Z given Si(z) = 1
Protected
Si(v) = 0, Si(p) = 1
undefined
Yi(p,prot.)
Always infected
Si(v) = 1, Si(p) = 1
Yi(v)
Yi(p,alw.inf.)
Z = assigned treatment
 To assess a causal vaccine effect: we need to
compare Yi(v) (= Yi(v, alw.inf.)) with Yi(p, alw.inf.), but the
placebo VLs are a mixture of Yi(p, prot.) and Yi(p, alw. inf.)
 How to identify the distribution of Yi(p, alw.inf.)?
10
Adjusting for Selection Bias (continued)
• fp(y) = (VE)fp(prot)(y) + (1-VE)fp(alw.inf)(y)
 fp(alw.inf)(y) = [w(y)/(1-VE)]fp(y)
where w(y) = Pr{Si(v)=1|Yi(p)=y, Si(p)=1} is the unknown
probability that a placebo infectee with VL set-point
y would have been infected if given vaccine.
• VE and fp(y) can be estimated from the data, but not
w(y).
Solution: assume a “known” model for w(y).
11
Adjusting for Selection Bias (continued)
• GBH (2003) assume a logistic model for w(y):
wi, = w(yi|,) = exp( + yi)/{1+exp( + yi)}, inp
where  is a fixed (pre-set) parameter:
(i)  = 0  wi, = 1 – VE for all i
(ii)  < 0  for a 1-unit decrease in Yi(p), the odds of
being in the always infected stratum increase
multiplicatively by exp(-)
(iii)  is a constant satisfying Fp(|) = 1
• For a given , fp(alw.inf) can now be estimated.
12
VL Distributions for the Protected and Always Infected
Principal Strata Implied by the Logistic Model for wi(y)
2
3
4
5
log10 VL
6
7
0.4
0.3
0.2
1-VE
VE
0.0
VE
0.0
1-VE
= -
(e- = )
0.1
0.2
probability density
0.3
0.4
1-VE
0.1
 = -2
(e- = 7.4)
0.1
probability density
0.2
0.3
VE
0.0
probability density
0.4
=0
(e- = 1)
2
3
4
5
log10 VL
6
7
2
3
4
5
6
7
log10 VL
 = 0: vaccine does not selectively protect subjects  same
distribution for Yi(p, prot.) and Yi(p, prot.)
 < 0: vaccine selectively protects subjects with higher VLs
 selection bias leads to biased estimation of the causal
effect that makes the vaccine look poorer than it is.
13
Adjusting for Selection Bias (continued)
• Adjust the viral load test in Simes’ method:
1) Fix the selection bias parameter   0.
2) Adjust (reduce) all the VLs of placebo infectees:
( p )*
i ,
y
 n p yi( p ) n p w yi | ˆ,   yi( p ) 
( p)

 yi   i1
 i1 n p
 np

w

y
|

,


ˆ
i1 i


ˆ is non-parametric m.l.e. of 
3) Let T = Wilcoxon rank sum statistic comparing
(v)
i
i nv
{y }
with
{ yi( p )*}in p
14
Adjusting for Selection Bias (continued)
• When VE = 0, T is the Wilcoxon rank sum statistic
used for the unadjusted VL test.
• The distribution of T is intractable, so the p-value for
the adjusted VL test ( = p2, ) is obtained using a nonparametric bootstrap.
• Adjusted Simes’ method: for the specified , reject
the composite null hypothesis if
max( p1 , p2, )   or min( p1 , p2, )   / 2
• Robust evidence of a causal vaccine effect on either
the infection or VL endpoint: reject the composite null
hypothesis using the adjusted Simes’ method for all
plausible values of .
15
BOI vs. Simes’ Method: Power (%)
Assuming  = 1 log10 copies/ml
VE = 0%
VE = 30%
100
80
80
60
60
40
40
20
20
0
0
Power (%)
100
10
30
50
70
90
10
30
50
70
90
VE = 60%
100
Power (%)
80
BOI
Unadjusted Simes' (Beta=0)
Adjusted Simes' (Beta=-1)
Adjusted Simes' (Beta=-2)
Adjusted Simes' (Beta=-Inf)
60
40
20
0
*Viral load reduction is 1 log10 in all plots
10
30
50
70
Number of HIV Infections
90
16
Concluding Remarks
• The selection bias-adjusted Simes’ method is more
powerful than the BOI method, unless VE is “large”
(unlikely for a CMI-based HIV vaccine).
• 50 events will provide at least 80% power to establish
POC provided:
VE  60% or   0.75 c/ml: unadjusted Simes’ method
VE  60% or   1.0 c/ml: adjusted Simes’ method.
• An -spending interim analysis after 30 events is
proposed (details omitted here). Estimated time
between 30 and 50 events is 9-15 months.
17
REFERENCES
1.
Chang MN, Guess HA, Heyse JF (1994). Reduction in the burden of illness: a new
efficacy measure for prevention trials. Statistics in Medicine, 13, 1807-1814.
2. Fisher RA (1932). Statistical Methods for Research Workers. Oliver and Boyd,
Edinburgh and London.
3. Frangakis CE, Rubin DB (2002). Principal Stratification in Causal Inference.
Biometrics, 58, 21-29.
4. Gilbert PB, Bosch RJ, Hudgens MG (2003). Sensitivity analysis for the assessment of
causal vaccine effects on viral load in HIV vaccine clinical trials. Biometrics, 59, 531541.
5. Hudgens MG, Hoering A, Self SG (2003). On the analysis of viral load endpoints in HIV
vaccine trials. Statistics in Medicine, 22, 2281-2298.
6. Mehrotra DV, Li X, Gilbert PB. Dual-endpoint evaluation of vaccine efficacy:
Application to a proof-of-concept clinical trial of a cell mediated immunity-based HIV
vaccine. Biometrics, in press.
7. Robins JM, Greenland S (1992). Identifiability and exchangeability of direct and
indirect effects. Epidemiology, 3, 143-155.
8. Rosenbaum PR (1984). The consequences of adjustment for a concomitant variable
that has been affected by the treatment. The Journal of the Royal Statistical
Society, Series A, 147, 656-666.
9. Rubin DB (1978). Bayesian inference for causal effects: the role of randomization.
The Annals of Statistics, 6, 34-58.
18
APPENDIX
• Arguments against a selection-bias adjustment:
- POC (not phase III) trial: not essential to precisely
characterize the vaccine effect.
- VE (and hence selection bias) anticipated to be small.
- If vaccine prevents infection only for less virulent
strains, then selection bias is more likely to make
placebo look better than vaccine when comparing VLs,
so the unadjusted test is already conservative from a
causal inference perspective!
• Arguments for a selection-bias adjustment:
- Will we really proceed to phase III without
robust evidence of a causal vaccine effect?
- To satisfy statisticians who are wary of any nonrandomized comparison.
19
Hypothetical Example
Infected/Enrolled
VL set-point
(log10 copies/ml)
Vaccine Group
22/750
2.26
3.98
2.55
4.02
2.57
4.17
2.68
4.17
2.82
4.44
3.13
4.69
3.17
4.83
3.41
3.45
3.64
3.65
3.74
3.82
3.92
3.95
Placebo Group
28/750
2.79
4.45
3.26
4.57
3.32
4.58
3.51
4.66
3.72
4.92
4.02
4.99
4.08
5.18
4.10
5.19
4.10
5.20
4.14
5.23
4.20
5.52
4.21
5.60
4.24
5.62
4.26
4.40
 VEobs  1  22 / 750 / 28 / 750  21% , p1 = 0.240
(binomial test)
 obs  4.43 – 3.59 = 0.84 log10 c/ml, p2 = 0.0001 (rank-sum test)
 Simes’ p-value = 0.0002*, BOI p-value = 0.062
20
Hypothetical Example (continued)
Vaccine Group
22/750
Infected/Enrolled
VL set-point
(log10 c/ml)
2.26
2.55
2.57
2.68
2.82
3.13
3.17
3.41
3.45
3.64
3.65
3.74
3.82
3.92
X
0
0
0
0
0
0
0
0
0
0
0
0
0
0
3.95
3.98
4.02
4.17
4.17
4.44
4.69
4.83
Placebo Group
28/750
X
0
0
0
1
1
0
1
1
2.79
3.26
3.32
3.51
3.72
4.02
4.08
4.10
4.10
4.14
4.20
4.21
4.24
4.26
X
0
0
0
0
0
0
0
0
0
0
0
0
0
0
4.40
4.45
4.57
4.58
4.66
4.92
4.99
5.18
5.19
5.20
5.23
5.52
5.60
5.62
X
1
1
0
1
0
1
1
0
1
1
1
1
1
1
 X = unobserved covariate (e.g., a genetic trait)
 A higher proportion of placebo infectees have X=1, and
subjects with X = 1 tend to have higher viral loads.
 Did vaccine cause lower VLs or is the observed vaccine
effect an artifact of the imbalance in the X distribution?
21
Hypothetical Example Revisited
Assigning weights to the VLs in the Placebo Group
log10
VL
2.79
3.26
3.32
3.51
3.72
4.02
4.08
4.10
4.10
4.14
4.20
4.21
4.24
4.26
 0
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
weight (wi,)
  2   
0.99
0.98
0.98
0.97
0.96
0.93
0.92
0.92
0.92
0.92
0.91
0.90
0.90
0.89
1
1
1
1
1
1
1
1
1
1
1
1
1
1
log10
VL
4.40
4.45
4.57
4.58
4.66
4.92
4.99
5.18
5.19
5.20
5.23
5.52
5.60
5.62
 0
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
0.79
weight (wi,)
  2   
0.86
0.85
0.82
0.82
0.79
0.69
0.66
0.57
0.57
0.56
0.55
0.41
0.37
0.36
1
1
1
1
1
1
1
1
0
0
0
0
0
0
22
Hypothetical Example Revisited
0.1
1-tailed p-value
0.075
Robust evidence of a causal vaccine effect on VL

0.05
0.025
= Inf
0
1
2
3
4
5
6
7
8
9
10
OR=exp
23