Transcript Meta-analysis vs. randomized controlled trials

Overall and subgroup analysis
• If the OVERALL results show highly
significant evidence of a worthwhile effect of
treatment, but a few subgroups of the
overview unexpectedly indicate no benefit
(which could well happen by chance), then
the appropriate question is whether there is
good evidence that this life-saving treatment
should be denied to these patients.
• REVERSAL of the usual demand that there
should be proof of worthwhile benefit.
Courtesy of Dr. K. Wheatley
Meta-analysis vs. randomized controlled
trials: internal validity vs. generalizibility
• Have complimentary roles
– RCT, large adequately powered
• If our desire is to assess the efficacy of treatment
(i.e. understand a measure of benefit of the
treatment under ideal conditions of a clinical trial
using narrow defined eligibility criteria)
– Meta-analysis (of totality of evidence)
• If our goal is to obtain reliable estimate about the
treatment effectiveness (i.e. understand the extent
to which a given treatment can produce a
beneficial effect under variety of circumstances
and eligibility criteria)
Meta-analysis vs. randomized controlled trials
Small CTs
Meta-analyses
of small RCTs
Large RCts
Meta-analyses
of large RCTs
To study mechanisms
To generate hypotheses for
more reliable RCTs
To obtain reliable overall
answers under specific
conditions of a trial
To obtain a typical and
unbiased and generalizible
estimate of treatment effect
and to explore interactions
among subgroups
Literature-based vs. individual
patient data meta-analysis?
• IPD MA gold-standard
• LMA may be misleading
– Data extraction, patient exclusion,
length of follow-up, method analysis
may be less accurate in LMA
Lancet 1993;341:418-22; Stat Med 1998;14:2057-2079
IV Ethical obligations to account
of what’s already known
• To avoid unnecessary trials if reliable
knowledge already exists
• Conversely, to determine if there is true
uncertainty about relative values of
competing treatment alternatives
– A new trial should be conducted if there is a
substantial uncertainty which of the trial
treatments would benefit the patient better
• Requirement that equipoise
(uncertainty principle) is met
Ethical obligation of building
systematically on what is already known
• Clinical trials should be preceded by a
systematic review and should be reported
with a discussion of assessing the trial’s
results in the context what is already known
– Ethical requirement for updating systematic
reviews
• UK, Denmark, Holland now mandates search
or conduct of SR before a new clinical trial is
done
JAMA 1998;280:280-282;Lancet 2001:358:1648
V Knowledge resources
Archie Cochrane
“It is surely a great criticism of our profession
that we have not organised a critical summary,
by specialty or subspecialty, adapted
periodically, of all relevant randomised
controlled trials.”
Cochrane Database of Systematic
Reviews The Cochrane Collaboration - an international
network of individuals and institutions
committed to preparing, maintaining, and
promoting the accessibility of systematic reviews
of the effects of health care interventions.
Cochrane Systematic Reviews (2,796) (January 2003)
Database of Abstracts of Reviews of Effectiveness (3,875)
Registry of Randomized Controlled Trials (353,809)
How many systematic reviews are
needed to “cover” whole medicine?
• 10,000 systematic reviews to provide
broad coverage of most health care topics
Clarke M, personal communication
Cochrane Centres
Canadian
San Francisco
San
Antonio
UK
Dutch
French
Iberoamerican
Nordic
German
Italian
New
England
Brazilian
South
African
Australasian
Chinese
Cochrane Systematic reviews
• Cochrane reviews have been shown to
be methodologically superior to nonCochrane systematic reviews
BMJ 2000;320:537-40, JAMA 1998;280:278-80
The Cancer Library
Cochrane Cancer Network
with Update Software Ltd
Courtesy of Dr. Chris Williams
Meta-analyses in radiation oncology
• 100 meta-analyses in the Cochrane
Database of Systematic Reviews
– 22 Cochrane Reviews
– 78 DARE reviews
• MEDLINE (Clinical Queries) search
– 616 systematic reviews
Meta-analyses in radiation oncology: an
example of reliable review with long-term
(20 years) follow-up
• Favourable and unfavourable effects on
long-term survival of radiotherapy for
early breast cancer: an overview of the
randomised trials
Early Breast Cancer Trialists' Collaborative Group*
Lancet 2000; 355: 175770
(20 May 2000 )
Proportional effects on all-cause mortality
in 40 trials of radiotherapy
L
a
n
c
e
t
2
0
0
0
;
3
5
5
:
Absolute effects of radiotherapy
on cause-specific survival
Absolute benefits and hazards
Part VI
Evidence and decision-making
Clinical Decision Making
Patient circumstances
Evidence
from
research
Preferences,
values and
rights
Courtesy of Dr. G. Lyman
Reporting data on benefits and
harms
• If evidence on benefits and harms are not
reported or is of poor quality, one has to
wonder how physicians make decisions
and recommendations for their patients
Eddy D. JAMA 1990;264:1737-39
Reporting data on benefits and harms:
RCTs in myeloma
• Survival outcomes
111/136 (82%)
• Survival beyond 5 years
15/111 (14%)
• Treatment-related mortality
33/136 (24%)
• Non-fatal adverse events
91/136 (67%)
Annals Oncol 2001;12:1611-1617
Porcentage of studies
Reporting harms in RTOG
randomized trials
100
90
80
70
60
50
40
30
20
10
91.5%
86.4%
74.6%
N= 51
N = 44
N = 54
Overall survival
Toxicity reported
treatment-related
mortality
Outcomes
Total number of studies: 59
HOW TO INTEGRATE BENEFITS
AND RISKS OF AVAILABLE
THERAPEUTIC OPTIONS
• Should we always use the option with the
best benefit/risk ratio?
Efficacy=80%
Toxicity=10%
E/R=8
Efficacy=20%
Toxicity=1%
E/R=20
Decision-making at the bedside
• Minimal conditions for treatment benefit at
which therapy is worth considering is met
when
– Absolute benefits>absolute harms
(adjusted for the probability of bad event,
e.g. relapse)
• Never administer treatment or order
diagnostic test if treatment harm is greater
than its efficacy
Integrating benefits and harms of
radiation therapy of breast cancer
• Threshold for administering radiation
therapy (RT):
probability of breast cancer recurrence
(without RT)>
Deaths due to (RT) (%)
Deaths due to breast cancer without RT- deaths due to breast cancer on RT
4.3%
51.4-46.6 (=4.8%)
= 89.6%
(actual relapse=30.1%)