Transcript Document

On Evidence, Medical and Legal
Donald W. Miller, Jr., M.D.
Professor, Division of Cardiothoracic Surgery
University of Washington School of Medicine
Director, Cardiothoracic Surgery, VA Puget Sound HCS
Decisions Made
Legal-justice system:
Guilty --or-- Not Guilty
Liable --or-- Not Liable
Medicine/science:
Research hypothesis --or-- Null hypothesis
True
True
Examples:
TMR relieves angina
Heparin causes thrombocytopenia
Vitamin E supplements increase mortality
TMR has no affect on angina
Heparin does not cause thrombocytopenia
Vitamin E supplements are safe
Standards of Proof
Kind
Level of Evidence
Standard
____________________________________________________________________
Regulatory
Precautionary Principle
Legal--Civil
★
More likely than not
Legal--Civil
★★
Clear and convincing
★★★
Beyond a reasonable doubt
Legal--Criminal
Scientific
★★★★
Irrefutable
Evidenced-Based Medicine
“…use of current best evidence in
making decisions about the care
of individual patients”
Sackett DL, et al. Evidence-Based Medicine: How to Practice and Teach EBM (Second
Edition). Edinburgh: Churchill Livingstone; 2000
The EBM Evidence Pyramid
RCTs on Transmyocardial Laser
Revascularization
•
•
•
•
•
•
•
Frazier OH, Tuzun E, Eichstadt H, et al. Transmyocardial laser revascularization
as an adjunct to coronary artery bypass grafting: a randomized, multicenter study
with 4-year follow-up. Tex Heart Inst J 2004;31(3):231-239.
Horvath KA, Aranki SF, Cohn LH, et al. Sustained angina relief 5 years after
transmyocardial laser revascularization with a CO(2) laser. Circulation
2001;104(Suppl I):I-81-84.
Frazier OH, March RJ, Horvath KA. Transmyocardial revascularization with a
carbon dioxide laser in patients with end-stage coronary artery disease. N Engl J
Med 1999;341:1021-1028.
Horvath KA, Cohn LH, Colley DA, et al. Transmyocardial laser
revascularization: results of a multicenter trial with transmyocardial laser
revascularization used as sole therapy for end-stage coronary artery disease. J
Thorac Cardiovasc Surg 1997;113:645-653.
Schofield PM, Sharples LD, Caine N, et al. Transmyocardial laser
revascularization in patients with refractory angina: a randomised controlled trial.
Lancet 1999;353:519-524.
Allen KB, Dowling RD, Fudge TL, et al. Comparison of transmyocardial
revascularization with medical therapy in patients with refractory angina. N Engl
J Med 1999;341:1021-1028.
Burkhoff D, Schmidt S, Schulman SP, et al, for the ATLANTIC Investigators.
Angina Treatments-Lasers and Normal Therapies in Comparison.
Transmyocardial laser revascularization compared with continued medical
therapy for treatment of refractory angina pectoris: a prospective randomised trial.
Lancet 1999;354:885-890.
Transmyocardial Laser
Revascularization (TMR)
Single high-energy CO2
laser pulse fired through
the wall of the heart.
Epicardial surface of
channel seals; 20
percent remain
patent; and new
capillaries 0.1 to .3
mm in diameter, lined
with endothelium,
extend out into the
myocardium from the
channels. Channels
spaced 1 cm2 apart.
Angina Relief with TMR
Ciruclation 2001;104(12 Suppl):181-4
Angina Relief in RCT Treatment Groups
Ciruclation 2001;104(12 Suppl):181-4
Change in Myocardial Perfusion in
RCT Treatment Groups
N Eng J Med 1999;341(14):1021-8
Event-Free Survival in RCT
Treatment Groups
N Eng J Med 1999;341(14):1028-8
Enhanced Quality of Life in RCT
Treatment Groups
Ciruclation 2001;104(12 Suppl):181-4
Biologic Plausibility of TMR
CO2 Laser
• Denervation
– Mylenated fiber
– Sympathetic fibers
• Endocardial channels
– 20 % stay remain patent
• Neoangiogenesis from channels
– New capillaries 0.1-0.3 diameter
with endothelium
– PET studies show reperfusion
Full-thickness 1 mm channel
created with a 20 joules
single pulse.
Patient Selection
Indications for TMR
? CABG to 1mm coronary arteries
? Reoperative CABG
? Poor LV function (EF <30%) with intraaortic balloon pump
? Angina-equivalents (exertional dyspnea)
Epidemiological Evidence:
Randomized Trials and Cohort Studies
• Address the incidence of disease and the
effects of therapeutic interventions at the
population level
Has limited usefulness in making clinical
decisions in individual patients
• Two things it cannot do:
Detect rare events
Prove (or disprove) that x causes y in a
specific individual
• Bradford Hill criteria for positing a causal
association
• Type I and II errors
Bradford Hill’s “Criteria” for a
Causal Association
1.
2.
3.
4.
5.
6.
7.
8.
9.
Consistency
Strength of Association
Temporal Sequence
Dose-Response
Specificity
Coherence
Biological Plausibility
Analogy
Experimental Evidence
Hill AB. The environment and disease: association or causation? Proc R Soc Med 1965;58:293-300.
Verdict on the Null Hypothesis (H0)
Decision
The Real Truth
HO True
HO False
Reject
Type I error
Correct
Retain
Correct
Type II error
(not guilty)
(HO false)
(HO true)
(guilty)
Cochrane Meta-Analysis
Albumin Administration in Critically Ill Patients
Subjects:
30 RCTs, 1419 patients
Conclusion:
Risk of death 6% higher
in patients given albumin
Funnel plot for the 24 trials in which
deaths occurred
Cochrane Injuries Group Albumin Reviewers (CIGAR). BMJ 1998;317:235-240
Cochrane Meta-Analysis
Albumin Administration in Critically Ill Patients
The Fallout
The Times (London), July 24, 1998: The review “suggests that up to
30,000 patients in Britain alone have died because they were treated
human albumin solution.”
The Observer (London), July 26, 1998 (Criticizing the UK
Department of Health on dragging its feet in responding): “300 die
as health chiefs dither.”
Response of the principal author, Ian Roberts, to this furor: “We were
amazed but totally confident we are accurate…having studied all
the evidence I am sure we are right.”
Cochrane Meta-Analysis
Albumin Administration in Critically Ill Patients
Its Flaws
• None of the (seven) authors care for critically ill
patients in the ICU
• Deaths < 24 hours excluded, > 30 days included
• Omitted relevant trials
• Included trials that gave albumin on a daily basis for
hypoalbuminemia, not hypovolemia
• Combined heterogeneous trials (adults and high-risk
neonates)
• Conflict of interest: study funded by UK’s NHS, which
could cut costs by replacing albumin with crystalloid
Cochrane Meta-Analysis
Albumin Administration in Critically Ill Patients
Subsequent Developments
• Cochrane authors update their review in 2000
• Conclusion remains the same: albumin not safe
• Wilkes and Navickis publish systematic review on this
subject in 2001 in Ann Intern Med; 55 RCTs, 3504 patients
• Conclusion: no effect of albumin on mortality, albumin is safe
• Cochrane Collaboration
• Quietly removes Albumin Review from its library of metaanalyses
Scales for Assessing the Quality of RCTs
Does Low Molecular
Weight Heparin prevent
DVT?
For your meta-analysis to
answer the question:
Yes
or
No
Choose a quality scale that
supports the answer
you want
Evidentiary Flaws in
Randomized Controlled Trials
Biases in methodology
– Faulty trial protocols
– Reporting outcome in terms of relative risk
without giving absolute risk of all-cause deaths
– Justifying intervention on surrogate outcomes
(cholesterol level) when the primary outcome
(freedom from MI and survival) is not improved
Source of funding
Kauffman JM. Bias in recent papers on diets and drugs in peer-reviewed medical journals.
J Am Phys Surg 2004;9:11-14.
EBM Guideline Approach to Clinical Problems
(Two-Dimensional) Type I Complexity
Welsby PD. Reductionism in medicine: some thoughts on medical education from the clinical front line.
Journal of Evaluation in Clinical Practice 1999;5:125-131.
Type I Complexity Guideline
NHLBI JNC 7 Clinical Practice Guideline for
Hypertension
Hypertension. 2003;42:1206
EBM Guideline Approach to Clinical Problems
(Three-Dimensional) Type II Complexity
3-vessel CAD
COPD
Hypertension
Chronic renal insufficiency
CLL
“Medicine is a science of
uncertainty and an art of
probability”
William Osler
Welsby PD. Reductionism in medicine: some thoughts on medical education from the clinical front line.
Journal of Evaluation in Clinical Practice 1999;5:125-131.
EBM Guideline Approach to Clinical Problems
(Four-Dimensional) Type III Complexity
The Art of Medicine
-- Lies within the matrix of these
interacting diseases and guidelines
Welsby PD. Reductionism in medicine: some thoughts on medical education from the clinical front line.
Journal of Evaluation in Clinical Practice 1999;5:125-131.
The EBM Evidence Pyramid
Observational Studies:
Cohort and Case Control
Statistical techniques used to construct
matched sets of treatment and control
subjects and reduce bias
– Propensity score
– Multivariate logistic regression modeling
Mangano DT and others. The Risk Associated with
Aprotinin in Cardiac Surgery. N Engl J Med 2006
(January 26);354:353-365.
Conclusion: “The association between aprotinin and
serious end-organ damage indicates that
continued use is not prudent. In contrast, the less
expensive generic medications aminocaproic acid
and tranexamic acid are safe alternatives.”
Flaws in the NEJM Aprotinin Study
• Conflict of Interest
• An observational study (using propensity
scores)
• Misapplication of earlier work to trial
results
• Biased selection of references in support
of a predetermined position
• Ideological bias of the journal’s editor
and his appointed peer reviewers
Indications for Aprotinin in Cardiac Surgery at
the Seattle VA Medical Center
Reoperations
Valve surgery
Myocardial infarction < 7 days
Plavix < 5 days
> 75 years old
The EBM Evidence Pyramid
EBM protagonists dismiss case reports as “anecdotal”
Value of Case Reports
The most essential evidence in medicine is the
patient’s story
Eyewitness testimony (i.e., a case report) can
meet the highest legal standard of proof
A single case report can meet the scientific
standard of irrefutability
“Double Hit” CDR Evidence that Heparin
Causes Thrombocytopenia
Challenge
Dechallenge
Rechallenge
NEJM 2001;344:1286-1292
Causal Significance of CDR Evidence
Institute of Medicine
“The recurrence or non recurrence of the adverse
event will often have a major impact on the
causality assessment.”
FDA
“Even a single well-documented case report can be
viewed as a signal [of causation], particularly if the
report describes a positive rechallenge.”
Stephens’ Detection of New Adverse Drug
Reactions
A positive rechallenge is “probably the strongest proof
of a causal relationship.”
Brides in Bath Case
George Joseph Smith
R v Smith, 1915, (11 Cr App R, 229)
Double Hit: CDR Evidence that MMR
Vaccine Causes Autism
Challenge
Two months
after MMR
vaccination
Dechallenge
Rechallenge
After MMR
booster shot at
age four
Standards of Proof
Kind
Level of Evidence
Standard
____________________________________________________________________
Regulatory
Precautionary Principle
Legal--Civil
★
More likely than not
Legal--Civil
★★
Clear and convincing
★★★
Beyond a reasonable doubt
Legal--Criminal
Scientific
★★★★
Irrefutable
The Precautionary Principle
Based on the:
– 1990 Bergen Declaration
– 1992 Rio Conference of Sustainable Development
– 1998 Wingspread Declaration
Increasingly governs state regulatory policy and
international environmental law
Invoked to reduce CO2 emissions, ban DDT, and
bar planting of genetically engineered crops
Calamities Resulting From the
Precautionary Principle Standard of Proof
Malaria
Starvation
Feinstein AR, Horwitz RI.* Problems in “Evidence” of
“Evidence-Based Medicine.” American Journal of
Medicine 1997;103:529-535.
* Departments of Medicine and Epidemiology, Yale University
School of Medicine
Miller DW, Miller CG. On Evidence, Medical and Legal.
Journal of American Physicians and Surgeons 2005;10:70-73.
Seattle VA Medical Center
Meta-Analysis of Sequential Trials
The Fallacy of Assumed Transitivity
• Trial design: Six numerically graded parameters for each
drug:
-- Clinical Action
-- Absorption
-- Metabolism
-- Bioavailability
-- Excretion
-- Side-effects
• Results:
– Drug A vs. B (Trial 1): A is better than B
– Drug B vs. C (Trial 2): B is better than C
– Drug C vs. D (Trial 3): C is better than D
⁂ Conclude, like C, drug A will be better than drug D
Is this correct? Not necessarily
Non-Transitive Sequences
Drug (Dice) A, B, C, and D with Six Graded Facets
The counterintuitive result
Efron B. (1990) In: Innumeracy 1(J.A. Paulos, ed.) Penguin, London. p. 100
Case Series and Case Reports
Information about a single patient or series of patients
without a control group
Case Control Studies
Retrospective studies
Cohort Studies
Prospective or retrospective (historical) incidence studies
Randomized Controlled Trials (RCTs)
Answers questions about the safety and efficacy of
different therapies
2004 ACC/AHA Guidelines for CABG
Recommendations for TMR
,Class IIA: “[The] weight of evidence is in favor of [its]
usefulness/efficacy.”
Level of Evidence: A (“data derived from multiple
randomized clinical trials.”)
Recommendation: “TMR, either alone or in combination
with CABG, is reasonable in patients with angina
refractory to medical therapy who are not candidates
for PCI or surgical revascularization.”
Candidates for TMR
• Not candidates for PCI or CABG
• PCI or CABG performed in 25 % of 6 million
people with chest pain each year (1.5 million)
• 12 % of patients with anginal chest pain not a
candidate for PCI or CABG (60,000)
• Incomplete revascularization with CABG
•
15-28 % -- (60,000 to 110,000 patients)
The Evidence-Based Medicine
Gold Standard
Meta-Analyses (Systematic Reviews)
Combining trials that address similar questions for
greater statistical precision
? Human-Caused (?) Global Warming