Transcript Modivation
Multi-Clinic and Multi-National Trials:
A regulatory perspective
Charles Anello Sc.D.
Deputy Director, Office Of Biostatistics
Office of Pharmacovigelence and Statistical Sciences
Center for Drug Evaluation and Research
[email protected]
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Acknowledgements
I would like to express my appreciation to
Robert T. O’Neill, Satya Dubey, Jim Hung,
Robert Temple and John Lawrence for their
helpful suggestions in the preparation of
this talk.
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Disclaimer
The views expressed are those of the author
and do not necessarily represent those of the
Food and Drug Administration.
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Topics
Motivation
Regulatory Context
Changing Context
Some statistical Issues
Multi-national studies
An example
Conclusions
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Motivation
University Group Diabetes Program
Coronary Drug Project
How did multi-clinic and multi-national
trials become so common?
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Regulatory Context
Kefauver-Harris Amendments to FD &C Act.
The need to establish efficacy based on substantial
evidence
Adequately powered studies are needed to detect
clinically meaningful drug effects
As the focus shifted to modest effects the need for large
enough studies to establish efficacy led to the increase
use of multi-clinic or multi-investigator trials.
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Changing Context
Format and Content Guidance
ICH E9
Evidence Document
Emergence of a global pharmaceutical
industry and the multi-national Trial
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Format and Content Guidance
Results for individual studies should be
presented
Suggests test for interaction between
treatments and centers
No guidance on how this test is to be
conducted or the significance level to be
used
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Format and Content Guidance
Extreme or opposite results need to be
discussed
Demographic , baseline and results
presented by center
Graphic displays by center on key
characteristics and findings
Special attentions is given to age, gender,
ethnicity (or region).
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Why the emphasis on interaction?
The review of a clinical trial as several
aspects:
1. Are the treatment groups comparable
2. Are the potential sources of bias that may
account for the findings
3. Is the treatment effect consistent across the
centers
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Why the emphasis on interaction?
The review function as a detective aspect
The review function as a quality control aspect
The goal of the review is to establish efficacy and
safety and provide information to physicians in
the product label about how and for whom the
drug works
If subgroups (e.g., clinics) show different results(
i.e., interaction) this could impact the label
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
ICH E9
Suggests multi-clinic trials are needed to
speed up the time needed to conduct trials
Provide a basis for generalization
Stress the need to implement the common
protocol in a similar manner in all clinics
across all regions
Offers advice how to approach the analysis
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
ICH E9
Test main effect first , if significant
Test interaction as an exploratory analysis
If there are a large number of centers, it is
less important to consider interaction.
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Evidence Document
When only one multi-clinic trial is available
for establishing substantial evidence of
efficacy and safety
The results need to be statistically very
persuasive
Internally consistent
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Evidence Document
No single site, clinic provides an unusually large
fraction of patients
No single site, clinic is responsible for a
disproportionate favorable effect
Estimated effects are consistent across clinics
Multiple endpoints are consistent
Results highly statistically significant p<<.05 (eg.
.001)
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Lack of consistency in statistical
advice?
In an ANOVA situation the Format and Content
document suggest one tests first for interaction but
doesn’t say how to do it.
ICH E9 suggests one test main effects first only
they are significant does one examine interaction
Evidence document focuses on the need for
internal consistency but leaves open how
consistency is to be established. No advice on
statistical approach
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Looking at Interaction
There is no consistent approach
Some advocate fixed effect models
Some advocate random effect models
Some advocated mixed effect models
And so on.
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Some statistical Issues
Are centers fixed or random
If an ANOVA is used would one use the
type II or type III sums of squares
Other approaches
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Are Centers Fixed or Random?
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Centers fixed or Random
The decision to call centers fixed or random has
implications for the analysis
Arguments in support of fixed effects:
1. only approach for single center
2. gives precise answer to well defined question
3. only realistic approach with few centers,
4. definition of center is arbitrary
Senn (1998)
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Centers fixed or Random
1.
2.
3.
In support of the Random effect model
approximate answer to difficult question
affect more general
wider confidence interval
Senn (1998)
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Are Centers Fixed or Random?
There are other opinions
Fleiss(1988) favored fixed effect model with a
test for interaction
Grizzle(1988) questioned the fixed effect
model and preferred the random effect
model
Senn(1993) remained open minded on this
issue
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Are Centers Fixed or Random?
The random effect model incorporates
interaction into the error term.
The fixed effect model allows one to
explore interaction if it exists
Both approaches can be justified at some
level.
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Are Centers Fixed or Randon
In order to label a product correctly one needs to be sure
the effect is consistent across all relevant subgroups
From my perspective the fixed effect model seems more
relevant.
Whereas the random effect model may answer a question
it may answer a different question?
It may give the right answer to the wrong question.
Kimbell(1965) would have called this a “type III error”
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
What is the correct Sums of Squares
in an ANOVA analysis
The choice is between the type II and typeIII SS
for SAS GLM
Many authors support the type II SS which
involves weighting by center size
FDA has been alleged to favor the type III SS
While I can not speak for all FDA I think the
current view within CDER is with those
supporting the type II SS
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Much Discussion about variation
between centers
Is to be expected but difficult to detect
Is it quantitative (i.e., same direction only
magnitude differs) very common
Is it qualitative (i.e., treatment shows benefit in
some centers , Placebo shows benefit in others)
less common
Qualitative interaction is of concern but not found
very often and hard to establish
Peto(1982)
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Expected Center Variation
For fixed sample size as the number of centers
increases so does the chances of reversal
It has been shown that with as few as six centers
there is better than a 50% chance of having at
least one center where the placebo effect would
exceed the treatment effect.
O’Neill et. al.(1998)
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Multi-national Differences
Temple (SCT 2003) discussed the
interpretation of international differences in
clinical trail results
I will focus on a single international trial in
which a subgroup( country or region)
appeared to show a result different from the
overall finding of the study
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Multi-national Multi clinic Trials
MERIT-HF Trial a multi-national trial of
mortality in patients with chronic heart
failure.
When the regions of US versus Europe was
examined by a post hoc analysis differences
were observed although not quite at the
“qualitative difference”
one group showed a benefit another did not.
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
MERIT-HF Trial
Metoprolol succinate extended release
(Toprol XL) the MERIT-HF Trial
Placebo controlled, multi-national 3991
patient trial in NYHA Class II_III CHF
,treated added to ACEI, digitalis and other
Rx
Endpoints: all cause mortality and all cause
mortality plus hospitalization
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
MERIT-HF Trial
The differences occurred in the mortality
findings. The combined endpoint showed a
consistent finding whereas the mortality
endpoint showed a 50% reduction in the
Non US population compared to a 5%
excess in the US population. With a post
hoc p-value of .003.
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
MERIT-HF Trial
The regions were not a pre-specified subset
Approval was not an issue the study showed a
benefit for mortality plus hospitalization
The agency’s internal debate focused on “full
disclosure” this apparent unexplained qualitative
interaction in a component of composite endpoint.
Whether regions matter in this instance is an open
question.
Temple(2003)
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Summary
Multi-clinic trials are an important aspect of drug
development and regulatory review
Multi-clinic multi-national trials are becoming
more important
From a regulatory perspective consistency in
results are important
If interaction occur, they need to be identified and
understood
Interactions may have implications for the product
label and public health
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Summary
Some statistical tools for dealing with these interaction
exist, but may have limited utility due to low power
The fixed effect model may be more useful when there are
not to many clinics and the sample sizes are large and
approximately equal size and it is reasonable to assume a
constant effect across the clinics
The random effect model may be more useful when there
are large number of clinics with few observations per clinic
and one can not address the question of center by treatment
interaction.
When an ANOVA is appropriate, the type II sums of
squares is often the Sums of squares of choice
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD
Summary
Quantitative and Qualitative interactions happen
although Qualitative interaction is rare
Observed clinic or regional differences need to be
noted, explained if possible and reported but not
necessarily believed.
The potential for regional differences, may have
implications for study design and the way DSMBs
monitor trials.
FDA\
CDER\OPASS\OB
ASA-Industry Workshop,Sept.1719,2000 Bethesda, MD