Transcript Keynote Powerpoint Presentation

John Ioannidis, MD
Professor, Stanford University;
Director, Stanford Prevention Research Center
#Lown2016
Evidence-based medicine has been
hijacked
Lown Conference, Chicago, 4/2016
John P.A. Ioannidis, MD, DSc
C.F. Rehnborg Chair in Disease Prevention
Professor of Medicine, of Health Research and Policy, and of Statistics
Stanford University
Director, Stanford Prevention Research Center
Co-Director, Meta-Research Innovation Center at Stanford (METRICS)
Key tools of non-evidence-based medicine in 1990
(splendidly serving vested interests)
• Ex cathedra pronouncements by prestigious
opinion leaders
• Editorials
• Non-systematic reviews
• Professional society guidelines done for the
glory of the profession
• Pamphlets from drug reps
• Other marketing material disseminated in
medical “scientific” meetings
Key tools of non-evidence-based medicine in 2016
(splendidly serving vested interests)
• Practice guidelines based on seemingly
rigorous processes
• Systematic reviews and meta-analyses
• Randomized trials
• Observational studies and risk factor
epidemiology
• X-omics, predictive medicine, precision
medicine, personalized medicine,
individualized medicine
• Physicians
A sad realization
EBM is widely tolerated only when it can
produce mostly boring evidence reports that
can be endorsed by experts and/or serve
vested interests.
The very same people who were previously
spitting when mentioning “EBM”, are now
using the very same term to buttress their
eminence-based medicine claims to prestige
by (mis)using the tools of EBM
How good is the quality of the
clinical evidence?
• All 1394 systematic reviews published on the Cochrane Database
of Systematic Reviews from January 2013 to June, 2014.
• GRADE (Grades of Recommendation, Assessment, Development,
and Evaluation) summary of findings performed in 608 (43.6%).
• Quality of the evidence for the first listed primary outcome: 13.5%
high, 30.8% moderate, 31.7% low, 24% very low level.
• Even when all outcomes listed were considered, only 19.1% had at
least one outcome with high quality of evidence.
• Of the reviews with high quality of evidence, only 25 had both
significant results and a favorable interpretation of the intervention.
Fleming et al, J Clin Epidemiol 2016
Uneven research evidence
• Spongiform encephalopathies: 2050
MEDLINE publications per 1000 patients
• Myasthenia gravis: 156 MEDLINE
publications per 1000 patients
• Cerebrovascular disease: 7.7 MEDLINE
publications per 1000 patients
• Severe varicose veins: 0.5 MEDLINE
publications per 1000 patients
Frankel and West 1993
Sub-Saharan Africa: 0.1% of all trials
for 25% of the global burden of disease
Isaakidis et al. BMJ 2002
Scientific discovery has become a boring nuisance: 96% of
the biomedical literature claims significant results
Chavalarias, Wallach, Li, Ioannidis, JAMA March 15, 2016
The glacial pace of clinical translation
Ioannidis, JAMA 2005, Contopoulos-Ioannidis et al. Science 2008
Clinical trials: Auto-looping of industry sponsors
Design of clinical research: an open world or isolated city-states (company-states)?
Lathyris et al., Eur J Clin Invest, 2010
Always getting the “right” results (!?)
• Among trials published in 2011, 55/57 of
non-inferiority trials with head to head
comparisons sponsored by the industry
demonstrated non-inferiority
• Success rate 96.5%
Flacco et al, J Clin Epidemiol 2015
Which are the best
antidepressants?
Meta-analyses
from the best
meta-analysts in
the world
Meta-analyses on antidepressants
for depression
• Using MEDLINE, we searched for metaanalyses evaluating antidepressants for
depression published in 1/2007-3/2014.
• The search starts after the big debate about
antidepressants increasing the risk of
suicides and suicide-related death
Ebrahim et al. J Clin Epidemiol 2015
Meta-analyses on antidepressants
for depression
• We identified 185 eligible meta-analyses
published over these 7 years.
• 54 (29%) had authors who were employees
of the assessed drug manufacturer and 147
(79%) had some industry link (sponsorship
or authors who were industry employees
and/or had conflicts of interest).
Meta-analyses as a marketing tool
• Only 58 of the 185 meta-analyses on
antidepressants for depression (31%) had
any negative statements in the concluding
statement of the abstract.
• Meta-analyses including an author who
were employees of the manufacturer of the
assessed drug were 22-times less likely to
have negative statements about the drug
than other meta-analyses (1/54 [2%] vs.
57/131 [44%], p<0.001).
Guidelines as a marketing tool and
as a potential threat to patients
Sharing
data –
who,
when,
and how?
Doshi, Goodman,
Ioannidis, TiPS
2013
Containing the influence of
conflicted stakeholder and authors
• Is transparency in potential conflicts of
interest enough?
• Who should be the sponsors and authors of
–
–
–
–
–
Randomized trials
Meta-analyses
Cost-effectiveness analyses
Guidelines
Other “sensitive” studies?
Who should fund research?
Type of
research
Pre-specified Utility
deliverables
Current Ideal major funder
major
funder
Discovery
No (high
Possible, but in
Public
Public (e.g. NIH)
“blue sky”
uncertainty
entirely unpredictable (e.g.
science
by default)
ways, maybe decades NIH)
later
Applied pre- Yes
Possible; substantial Public
Entrepreneurs and industry who
clinical
(uncertainty failure rate in single (e.g.
will profit if they deliver
research
is substantial) projects, accumulated NIH)
something that truly works
efforts should pay off
Clinical
Yes
Yes; results should be Industry Public (e.g. NIH, PCORI);
research
(uncertainty sufficiently useful
industry may contribute some
is
regardless of whether
funds to a common funding pool,
manageable, they are “positive” or
regulatory agencies and
explicit goals “negative”
universities/research institutions
should be set)
should safeguard the
independence of research and
may steer overall agenda
A field-wide view
of risk factor
epidemiology
Standardization of
definitions and
analyses
Serghiou, Patel, Ioannidis, J Clin
What to
adjust
for?
Serghiou et al, JCE
2015
Vibration of effects and the Janus
phenomenon
Patel, Burford, Ioannidis. JCE 2015
Risk factor epidemiology and
public health in 2016
• Major risk factors remain highly prevalent (e.g.
smoking will kill 1 billion people in the next century)
• Epidemiologists like myself are addicted to pursuing
massive amounts of insignificant significant results
• Salami-sliced data-dredged papers are becoming
more frequent
• Public policy is dictated from spurious evidence
Big data leads to proliferation of
discoveries (mostly false discoveries)
Khoury, Ioannidis, Science 2014
ἀλλὰ μάλα Σκύλλης σκοπέλῳ πεπλημένος ὦκα
νῆα παρὲξ ἐλάαν, ἐπεὶ ἦ πολὺ φέρτερόν ἐστιν
ἓξ ἑτάρους ἐν νηὶ ποθήμεναι ἢ ἅμα πάντας.
If you worry about Scylla,
Charybdis is worse
Sailing closer to Scylla is
preferable, because Charybdis
may drown the whole ship:
"Hug Scylla's crag—sail on
past her—top speed! Better by
far to lose six men and keep
your ship than lose your entire
crew"
Charybdis
• Private corporations sponsoring the
construction of doubt about the adverse
effects and toxicity of their products
• quacks ranging from television presenters
and movie stars turned into health trainers
• pure science denialists (e.g., climate, HIV,
vaccine denialists, and religious
fundamentalists)
Some entrepreneurs even think
that science and the scientific
method are obsolete
“The end of theory: The data deluge makes the scientific
method obsolete” (WIRED)
Ioannidis, JAMA, 2015
What EBM
should be
• “Evidence-based
medicine is about
integrating individual
clinical expertise with
the best external
evidence”
What it
actually is
• Clinicians are under
tremendous market pressure.
• Most discussions in
department meetings are about
money.
• This is mostly finance-based
medicine.
Clinicians are incentivized
•
•
•
•
•
•
to capture the largest possible market share
to satisfy customers
to get high satisfaction scores
to charge more
to perform more procedures
to tick off more items on charge forms
A logical consequence
• How likely is it that physicians will design
studies that threaten their jobs, decreasing
the procedures, testing and interventions
they do?
• Is EBM doomed to be heartily accepted
only when it leads to more medicine, even if
this means less health?
Concluding comments
• Science is the best thing that has happened to
human beings and well-conducted research is key
for securing and enhancing the future of mankind
• Evidence-based medicine has been a wonderful
idea, but its tools and their results have been
hijacked
• We need to think about what makes clinical
research clinically useful and reengineer the
reward system to align it with production of
credible, useful, unbiased results that can help
humans.
• Evidence-based medicine may still be possible to
implement somewhere someday.
Special
thanks
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Dan Fanelli, Stanford
Steve Goodman, Stanford
Debbie Dunne, Stanford
Shanil Ebrahim, Stanford and McMaster
Tina-Hernandez Boussard, Stanford
Ewoud Schuit, Stanford
Josh Wallach, Stanford
Despina Contopoulos-Ioannidis, Stanford
Lamberto Manzoli, U Chieti
Maria Elena Flacco, U Chieti
Kostantinos Siontis, U Michigan
Nikos Pandis, U Bern
Muin Khoury, CDC/NIH
Alvin Li, UWO
Jeanne Lenzer, BMJ
Jerome Hoffman, UCLA
Curt Furberg, Wake Forest
Petros Isaakidis, Medecins sans Frontiers
Chirag Patel, Harvard
David Chavalarias, Institute for Complex Systems
Stelios Serghiou, U Edinburgh