Transcript Analysis of multiple end points in clinical trials

By Trusha Patel and Sirisha Davuluri
“An efficient method for accommodating potentially
underpowered primary endpoints”
◦ By Jianjun (David) Li and Devan V. Mehrotra”
2
1.
2.
3.
Introduction
Procedure 1: PAAS Method
Procedure 2: 4A Method
a.
b.
4.
5.
6.
7.
8.
Independent Primary Endpoints
Correlated Primary Endpoints
Performance of both Procedures (Probability of achieving positive trial)
Conclusion
Appendix
References
Questions?
3
Clinical trials generally classify the endpoints into
primary, secondary and exploratory types.
Primary Endpoints:

Primary endpoints address primary objectives of the trial.
They are usually few but are clinically most relevant to the
disease and the treatment under study. They assess the main
clinical benefits of the treatment.
4
Secondary Endpoints:

Secondary endpoints characterize extra benefits of the
treatment under study after it has been demonstrated that the
primary endpoints show clinically meaningful benefits of the
treatment.
Explanatory Endpoints:

Exploratory endpoints are usually not prospectively planned
and are generally not rigorously evaluated like primary and
secondary endpoints.
5
Different approaches have been used to specify a
clinical decision rule for trials that have more than one
primary endpoint. Significant results can be required for
each of several primary endpoints to consider a trial
‘‘positive.”
6

To demonstrate the treatment’s superiority on at least one
endpoint.

If each multiple endpoint is independently clinical relevant,
the multiple endpoint problem can be formulated as a multiple
testing problem, and the trial is declared positive if at least one
significant effect is detected.

When significant results are required for more than one but not
all of multiple primary endpoints for a trial to be considered
positive, correction for multiplicity is also necessary, and this
must take into account the total number of endpoints and the
number required for the trial to be considered positive.
7
Weighted Bonferroni procedure
Prospective Alpha Allocation Scheme (PAAS Method)
Adaptive Alpha Allocation Approach (4A Method)
Bonferroni-type parametric procedure
Fallback-type parametric procedure
For this project, we are mainly focusing on PAAS Method and 4A
Method by considering two Primary Endpoints
(A : well powered, B : potentially underpowered).
8
Procedure:



Moye’(2000) proposed the ‘‘prospective alpha allocation
scheme” for preserving Type I error rates at acceptable levels
when there are multiple endpoints.
Assume that p-values for the individual endpoints are
independent.
The experiment-wise type I error rate = α should be capped,
say at 0.05, and the fraction of the Type I error rate α
allocating
α1* = α − ε, ε= 0.01 to endpoint A
α2* = 1 - (1−α)/(1−α1*) to endpoint B
9
Advantages:


PAAS is a simple and appealing method.
Prospectively allocating alpha in this method preserves the
experiment-wise Type I error rate and makes it possible to
consider a treatment efficacious when the null hypothesis is
not rejected for the primary endpoint but is rejected for one or
more of the secondary endpoints.
Disadvantage:
The prospective alpha allocation scheme preserves the
experiment-wise Type I error rate at a higher rate than is
customarily accepted.
10
Procedure:

Li and Mehrotra (2008) proposed a multiple testing procedure,
which they referred to as the adaptive alpha allocation
approach or 4A procedure.

Consider a clinical trial with m endpoints and assume that the
endpoints are grouped into two families. The first family
includes m1 endpoints that are adequately powered and the
second family includes m2 potentially underpowered endpoints
(m1 + m2 = m).
11
Let’s consider 2 multiple points for illustration of the procedure:



Let pA and pB denote the p-value for endpoint A and endpoint
B, respectively.
In the article, they assume that p-values are two-tailed (to
mimic common practice) and that ‘statistically significant’
results are in the direction of interest; however all the methods
discussed can also be used with one-tailed p-values.
In their proposed adaptive alpha allocation approach, endpoint
A is tested at the pre specified level α1 = α − ε, and endpoint B
is tested at the adaptive level.
12



If A achieves statistical significance (i.e. pA ≤α1), then B is tested at level α2(pA,α1,
α) = α;
If A fails to achieve statistical significance (i.e. pA>α1), then B is tested at level 0
<α2(pA,α1,α) ≤ α1, which is close (or equal) to α1 if pA is not much greater than α1,
but approaches zero as pA increases.
Sample values of α2 as a function of observed pA are provided in Table I (ρ = 0
column)
 Table I shows the alpha allocation for Primary endpoint B.
13
Table I
14

Let there be M primary end points out of which the first m are
assumed to be powered by type A endpoints and the rest of the
M-m are underpowered or type B endpoints.

Here, they suppose, the p-values for the type A endpoints are
tested for statistical significance at an overall level α1 = α – ε
using Hochberg’s method.

The p-values for the type B primary endpoints are then
measured at and overall adaptive level α2(p(m), α1, α,m) using
Hochberg’s method
15
Extending 4A to the scenario of three or more
independent primary endpoints
Note : (3) and (4) are generalizations of (1) and (2); setting m=1 in
(3) and (4) leads to (1) and (2), respectively.
Advantage:
The remaining endpoints are tested at a generally higher
significance level, which improves their power.
16
Procedure:



If the endpoints are correlated, then formula (1) and (3) cannot
be used directly because the FWER may be inflated.
Therefore, an adjustment may be needed in the formulas
presented for correlated endpoints.
The 4A procedure is readily implemented using Table I if ρ is
known.
For the case of unknown ρ, See Appendix-I.
Extending 4A to the relatively uncommon scenario of M>2
primary endpoints is notably more challenging in the case of
correlated endpoints compared with independent endpoints.
17
Let’s compare the performances of the PAAS and 4A methods for
our motivating scenario of two primary endpoints: A-well
powered and B-underpowered.
Goal:
To improve the probability of correctly achieving a
positive trial, i.e. of rejecting at least one of the two null
hypotheses, while ensuring that the family wise type I error rate
is at most α.
Table A summarizes the probabilities of achieving a positive trial
using the PAAS and 4A methods when the two endpoints are
either independent (ρ=0) or correlated (ρ=0.5); results are based
on 10,000,000 simulations. (See Table III in Appendix-II)
18
Table A
19
Case 1-7:
Primary endpoint A: Adequately Powered
Primary endpoint B: Underpowered
Case 8-17:
Primary endpoint A: Underpowered
Primary endpoint B: Underpowered
For PAAS method, α1* = 0.04 and α2* = 0.010; and
For 4A method, α1 = 0.04 and α2≡α2(pA,α1,α,ρ) is calculated
adaptively as described before.
20

Use of 4A method increased the probability of achieving a
positive trial compared with PAAS.
o For Independent endpoints, the absolute gain in power
ranged from 0.9 % (case 7; 77.9 versus 78.8%) to 3.7%
(case 8; 78.8 versus 82.5%) compared with PAAS.
o For Correlated endpoints A and B (ρ=0.5); the power for 4A
was numerically higher than that for PAAS, For example,
the power gains of 4A ranged from 0.1 per cent (case 7; 77.4
versus 77.5 per cent) to 2.9 per cent (case 8, case 14)
compared with PAAS.
21

The probability of achieving a positive trial by pre
specifying both A and B as primary endpoints and
using 4A for the analysis is about as good as (cases 7
and 13) or notably better than (cases 1 and 8) the
marginal power for endpoint A.
o In other words, 4A enables us to accommodate the
underpowered endpoint B in the primary family while
essentially preserving or substantially enhancing the likelihood
of achieving a positive trial compared with the strategy of
using only A as the primary endpoint with α=0.05.
22

There is no optimal strategy for the use of significance
testing with multiple endpoints, especially when
dealing with clinically persuasive endpoints that may
be underpowered, but based on the simulation results
that are provided in the article, we can conclude that 4A
method performs better than PAAS method.
o The power advantage of 4A over PAAS was greater when the
marginal power for endpoint B was lower (higher), regardless
of whether the endpoints were independent or correlated.
23
24
25


http://www.ncbi.nlm.nih.gov/pubmed/18759248
Multiple Testing Problems in Pharmaceutical
Statistics, by Alex Dmitrienko, Ajit Tamhane and
Frank Bretz.
26
27