Transcript slides

Design of Clinical Trials for
Treatment of Invasive Fungal
Infections
John H. Powers, MD FACP FIDSA
Senior Medical Scientist
SAIC in support of Collaborative Clinical Research Branch
Division of Clinical Research
National Institute of Allergy and Infectious Diseases
National Institutes of Health
Disclosures
Consultant for:
Acureon
Astra-Zeneca
Centegen
Cerexa
CoNCERT
Destiny
Forest
Johnson and Johnson
Merck
Methylgene
Octoplus
Takeda
Theravance
Wyeth
2
Introduction

Why is appropriate design of trials important?

How do clinical practice and clinical research
differ?

What are the principles for designing an
adequate and well-controlled, internally valid
clinical trial?

How can we do better to address these issues?
3
Why is Design Important?

Four possible reasons for the results of a trial:
1. Random error – results due to chance alone
2. Bias – systematic error that results in deviation of results from
“true” results (inaccurate measurement)
3. Confounding –
 error where the measured result is the actual measure but not
causally related to treatment received
 factors such as disease severity are not “confounders” in
randomized trials, but effect modifiers

If the above reasons ruled out then………
4
Why is Design Important?
4. Valid results – “validity” means ability of study
to measure what is purports to measure


Internal validity – ability of study to measure what it
purports to measure

External validity – ability to generalize (transfer)
results to population rather than just sample
measured
A trial that does not have internal validity
cannot have external validity
5
Why is Design Important?

Random error addressed by adequate sample size
 P value addresses probability results may be due to chance
 Does not address likelihood that hypothesis is likely true

Bias and confounding addressed by appropriate design, no
statistical fix after the study is over

Increased sample size can increase effects of bias and
confounding on results

Only way to obtain valid results is through
appropriate design, conduct and analysis of
trial
6
Why is Design Important?

Invalid clinical trial results can lead to important
clinical consequences:
 Ineffective therapies used widely in patients ( cannot
“figure it out later” since difficult to determine cause and
effect in individual patients)
 Unwarranted harms to patients in absence of benefits
 Emergence of resistance and elimination of benefits for
other patients
 Ethical issues of exposing subjects to harm in
scientifically invalid research
 Belmont Report, Ethical Principles and Guidelines for
Research Involving Human Subjects
http://ohsr.od.nih.gov/guidelines/belmont.html
7
8
Ioannidis JP PLoS Medicine 2005;2(8):e124
Clinical Trials and Clinical Practice

Clinical practice and clinical research differ

Clinical practice based on “interventions
designed solely to enhance the well-being of an
individual patient or clients and that have
reasonable expectation of success”
 Belmont Report p.3, Ethical Principles and Guidelines
for Research Involving Human Subjects

Clinical research is “activity designed to test an
hypothesis” in groups of subjects and “thereby
to develop or contribute to generalizable
knowledge”
 Belmont Report
http://ohsr.od.nih.gov/guidelines/belmont.html
9
Clinical Trials and Clinical Practice

Question not whether individual clinician believes
drug will be effective for individual patient in clinical
practice

Questions is how to study drug to demonstrate
safety and effectiveness in group of patients in a
clinical trial to then generalize to clinical practice

Medical need is reason to do a trial, not a reason to
accept invalid trials or lesser evidence

Designing trials based on previously held beliefs in
absence of evidence does not allow gathering of
evidence to validate those beliefs
10
What are the Principles?
1.
2.
3.
4.
5.
6.
7.
8.
Clear statement of objectives of the trial
Study design permits valid quantitative comparison
with a control
Select patients with disease (treatment) or at risk of
disease (prevention)
Baseline comparability (randomization)
Minimize bias (blinding, etc.)
Appropriate methods of assessment of outcomes
Appropriate methods of analysis
Appropriate measurement of potential harms
11
How Can We Do Better?
1) Clear objective:
 Define disease and clinical time course – mixing together various infections
makes interpretation of results challenging
 Differentiate treatment from prevention trials – “salvage” vs primary
treatment
 Differentiate explanatory trials from strategy/management trials
 Differentiate measurement of effectiveness from measurement of harms
 Better natural history data – what is an “invasive” infection? Does in vitro
resistance affect clinical outcomes and by how much?
 Allows for better enrollment criteria, more homogeneous population, less
variability, and appropriate timing of outcomes
2) Quantitative comparison with control
 Absence of control makes it challenging to assess causality of outcomes
 Choice of control: no treatment, placebo, dose response, active, historical
 Choice of study design: superiority, non-inferiority
12
Quantitative Comparison with a Control

Many ID clinical trials designed as “noninferiority” (NI) trials

Misconceptions about goals of NI trials
 Rule out margin by which test intervention may be less effective than
control intervention
 Does not show that experimental intervention is “as good as” or
“equivalent” to control unless shows statistical superiority
 Experimental intervention can be statistically inferior/superior and
“noninferior” at same time as long as not more inferior than margin
specified prior to trial

Designing a noninferiority trial means one is willing to accept
less effectiveness with the experimental intervention (for
what trade off?)
13
Designing a Valid Noninferiority Trial
1. Quantitative assessment that is reliable and reproducible (based on trials
that are themselves adequate and well controlled) of benefit of control
over placebo and suitably conservative evaluation examining variability
(not just point estimates)
2. Maintenance of the effect of the control from trial to trial (constancy
assumption)


Similar definition of disease, endpoints, timing of endpoints
Changes in medical practice, adjunctive therapies, antimicrobial resistance
3. Selection of margin of loss of effect of control that is less than the benefit of
control over placebo found in step 1
International Conference on Harmonization Guidance E-10, Choice of
Control Group and Related Issues in Clinical Trials, www.ich.org
14
Designing a Valid Noninferiority Trial

If these conditions not met, demonstration of similarity means
experimental and control intervention may be similarly effective
or similarly ineffective

Experimental intervention may not be any more effective than
placebo even if control agent previously effective

Link to external “negative control” data in NI trials similar to
external (historical) trials with similar biases

Other forms of bias in NI trials beyond “statistical” issues
 Not ensuring subjects have disease under study
 Blinding less effective at preventing bias since investigators know
all subjects receiving active intervention
 Greater bias due to inappropriate conduct of trials, concomitant
medications, missing data, etc.
15
How Can We Do Better?
3) Selection of subjects with disease (treatment) or at
risk of disease
 Rapid diagnostics which evaluate host response as well as
presence of organisms
 Biomarkers can be useful in diagnosis but in presence of
signs and symptoms of disease (positive predictive value of
test related to pre-test probability)
 Better current natural history data in prevention trials to
better select populations at risk
4) Baseline comparability using
 Randomization controls for selection bias as well as
measured and unmeasured confounders; basis for statistics
 Appropriate development of “severity” classifications
(comparing baseline variables to clinical outcomes) to stratify16
subjects at baseline and decrease variability
How Can We Do Better?
5) Minimizing bias
 Blinding of microbiological data to persons assessing outcome in
situations where impact of in vitro resistance on clinical outcomes is
unclear
 Could have unblinded third parties assess culture results in serious
diseases
 Will allow correlation of clinical outcomes with in vitro testing to better
define “resistance”
 Evaluate clinical outcome at time of culture result in any case
 Control for concomitant medications
 Minimize loss to follow-up and missing data
17
How Can We Do Better?
6) More accurate and sensitive outcome measures
 Effect of antimicrobials in severe disease based upon decrease in allcause mortality
 Biomarkers can make it more difficult to show effects in some diseases
since adds another criteria to assessment of outcomes
 Develop well-defined clinical outcome criteria independent of “clinician
judgment” (can cause misclassification bias and increased variability =
increased sample size) based on natural history of disease
 Expert outcome assessment does not eliminate bias and calls into
question generalizability of results
 Timing of outcomes - Time to event analyses in superiority trials can
inform duration of therapy, increase power to detect differences,
decrease sample size, and answer clinically relevant question on
magnitude of effect
18
Multiple/Composite Endpoints
All cause mortality
Non-fatal clinical events
Interested in multiple
aspects of how disease
may affect patients’
lives
Symptoms of disease
Surrogate endpoints
Lubsen J et al. Stat Med 2003;21:2159-70.
19
Multiple/Composite Endpoints
All cause mortality
Non-fatal clinical events
Symptoms of disease
Surrogate endpoints
Lubsen J et al. Stat Med 2003;21:2159-70.
Success based on
events from lower
on hierarchy
should not
supersede
failure based on events
higher up on hierarchy
that occur during
course of trial
even when surrogate
is used as part of
primary outcome
20
How Can We Do Better?
7) Appropriate analysis
 Decrease proportions of subjects who are “indeterminate” or
“unevaluable” by eliminating inappropriate exclusions from “per
protocol” analysis – all events post-randomization included
 Evaluation of the intent to treat, modified intent to treat analysis
protects against selection bias, maintains integrity of randomization
 Appropriate adjustments for multiple comparisons in secondary
endpoints and subgroup analyses
 Use of “gate-keeper” step wise hypothesis testing to control for false
positive results but requires a priori specification of order of
hypothesis testing
21
8. Analysis of Harms

Safety analysis requires an adequate number of subjects to assess
adverse events

“Rule of threes” – measurement of no events in a given trial allows
rule out rate of 3 divided by number of subjects studied (3/300 = 1%)

Not evaluating “statistical significance” of harms since not testing a
hypothesis in most clinical trials, but developing a hypothesis

Overall assessment of risks and benefits depends upon nature and
magnitude of both
 Greater risks acceptable when treatment has large effect on clinically
important endpoints like death
 Serious adverse events less acceptable when benefits small
 Unacceptable if benefits compared to placebo unclear
22
Conclusions

Need to accept that we can improve on current
level of evidence, answer questions that are still
unclear

Many opportunities to develop more clinically
relevant and more efficient clinical trials

Result can be more information for clinicians
and patients, optimal use of antimicrobials by
describing who benefits, by how much and with
quantitative comparison to risks
23