Transcript AURICULAR RECONSTRUCTION - Idaho Perinatal Project

DON’T BELIEVE EVERYTHING YOU READ:
AN EDITOR’S VIEW
JAMES R. SCOTT, MD
I have no conflict of interest to disclose.
OBJECTIVES
• TO CLEARLY UNDERSTAND
•
•
EVIDENCE BASED MEDICINE
TO BE ABLE TO ASSESS VALIDITY
OF NEW TREATMENTS
TO APPLY BEST-EVIDENCE IN
YOUR CLINICAL PRACTICE
JOHN IOANNIDIS
• WHY MOST PUBLISHED RESEARCH
•
FINDINGS ARE FALSE. PloS Med
2005;2(8):124
“The most downloaded document of
all time on PubMed”
SCOTT’S RULE
IT WILL NEVER WORK AS WELL IN
YOUR PRACTICE AS REPORTED IN
THE LITERATURE
"BELIEVE NOTHING
THAT YOU SEE IN
THE NEWSPAPERS
[TV, INTERNET].... IF
YOU SEE ANYTHING
IN THEM THAT YOU
KNOW IS TRUE,
BEGIN TO DOUBT IT
AT ONCE."
Sir William Osler
EVIDENCE BASED MEDICINE: A RECENT
PHENOMENON
16000
1400
14000
1200
12000
1000
10000
800
8000
600
6000
4000
400
2000
200
0
0
1973 1983 1993 2003
RCTs
2012 – 426,853 RCTs
1973 1983 1993 2003
Meta-Analyses
2012 – 53,042 Meta-Analyses
NUMBER OF PUBLISHED PAPERS WITH
“EVIDENCE BASED” MEDICINE IN TITLE
14000
12000
10000
8000
6000
4000
2000
0
1992

1996
2002
2004
2013 – NOW > 100,000
2007
2009
EVIDENCE-BASED MEDICINE
DE-EMPHASIZES
• INTUITION
• EXPERIENCE
• PATHOPHYSIOLOGY
EMPHASIZES
• PROOF
• OUTCOMES
PRINCIPLES OF
EVIDENCE-BASED MEDICINE
•
•
•
•
FIRST DO NO HARM
WHAT ARE THE RESULTS?
ARE THE RESULTS VALID?
HOW DO I APPLY THEM TO
MY PATIENTS?
LEVEL OF EVIDENCE
RANDOMIZED CONTROLLED TRIAL –
The “Gold Standard”
II-2 COHORT OR CASE CONTROL STUDY
II-3 LARGE CASE SERIES
III CASE REPORT, EXPERT COMMITTEE,
RESPECTED AUTHORITY, CLINICAL
EXPERIENCE
I
RULES TO IMPROVE PUBLISHED PAPERS
•
•
•
•
•
•
LEVELS OF EVIDENCE
GUIDELINES – CONSORT, PRISMA, STROBE
TRIAL REGISTRATION
CROSSCHECK FOR PLAGIARISM
DECLARE ALL CONFLICT OF INTERESTS
DETECT FRAUD
PREFERRED STUDIES - RANDOMIZED
CONTROLLED TRIALS (RCT) BEST
•
•
•
•
MOST RELIABLE EVIDENCE
CONSORT GUIDELINES
DESIGNED FOR EFFICACY ONLY
MY RULE - INCLUDE ABSOLUTE RISK
– ACTUAL NUMBERS, PERCENTAGES
– NUMBER NEEDED TO TREAT (NNT)
• NOTE – 25% LATER REFUTED
PROBLEMS TRANSLATING
RCTs INTO PRACTICE
•
•
•
•
•
•
DIFFERENT PATIENTS
COSTS MORE
INSURANCE WON’T COVER
PATIENT DOESN’T WANT IT
YOU ARE BETTER AT SOMETHING ELSE
UNANTICIPATED COMPLICATIONS OR
SIDE EFFECTS
CHALLENGES WITH NEW SURGICAL
AND OBSTETRIC PROCEDURES
•
•
•
•
RCTs DIFFICULT TO DO
BLINDING NOT POSSIBLE
COHORT/CASE SERIES NEXT BEST
UNETHICAL NOT TO COMPARE WITH
CURRENTLY ACCEPTED METHOD
CAUTION: IN YOUR OWN PRACTICE
USE IRB & INFORMED CONSENT
CLINICAL JUDGEMENT STILL IMPORTANT
•
WOMEN UNDERGOING ABDOMINAL
SACROCOLPOPEXY RANDOMIZED TO
BURCH OR NO BURCH (Brubaker et al.
N Engl J Med 2006;354:1557)
•
BURCH LOWERED INCIDENCE OF SUI
FROM 44% TO 24% (20%)
• REAL WORLD – PT WITH VARICOSITIES 
BLEEDING, HEMATOMA & 8 UNITS OF BLOOD
IMPORTANT TO REMEMBER
EFFICACY – UNDER IDEAL CONDITIONS
EFFECTIVENESS – IN REAL CLINICAL SETTING
(YOUR PRACTICE)
•
•
•
NO RCT FITS EVERY PATIENT
ALMOST NO RCT FITS ANY PATIENT
PERFECTLY
NOT GOOD AT DETECTING
COMPLICATIONS
META-ANALYSIS:
STATE-OF-THE-ART REVIEW
ADVANTAGES
• STANDARD RULES
• STATISTICAL POWER
DISADVANTAGES
• SUBJECTIVE ASSUMPTIONS
• COMPLICATED, ARTIFICIAL
• ONLY AS GOOD AS THE TRIALS USED
• ODDS RATIOS & CONFIDENCE INTERVALS
• TAKEN AS GOSPEL
COCHRANE REVIEWS
•
•
•
•
•
•
MOST WELL DONE
PUBLICATION BIAS
LONG & BORING
SOME OUTDATED
LIMITED ACCESS
ODDS RATIOS ONLY
STATISTICS
A FELLOW WITH ONE LEG FROZEN IN
ICE AND THE OTHER LEG IN BOILING
WATER IS COMFORTABLE – ON
AVERAGE.
J. M. Yancey
REALITY
NO RANDOMIZED TRIALS OR
META-ANALYSES FOR ABOUT
> 50-60% OF WHAT WE DO
NO RANDOMIZED CONTROLLED TRIALS
•
•
•
•
VBAC
CESAREAN ON REQUEST
SOME GYN SURGICAL PROCEDURES
MANY NEW MATERIALS & DEVICES
EXAMPLE: THERE ARE NO RANDOMIZED TRIALS
TO PROVE THAT PARACHUTES WORK
RECOMMENDATION:
EVIDENCE-BASED GURUS
SHOULD PARTICIPATE IN
A DOUBLE- BLIND RCT
Parachute Use to Prevent Death
and Major Trauma. Smith GCS
et al. BMJ 2003;327:149
COHORT STUDIES & CASE SERIES
STILL USEFUL
•
•
•
•
•
MORE CHANCE OF BIAS
STROBE GUIDELINES
HOW IT WORKS IN THE TRENCHES
LONG-TERM FOLLOWUP
FIRST REPORTS OF ADVERSE
EVENTS OR RARE COMPLICATIONS
LANDMARK OBSERVATIONAL STUDY
• FIRST DESCRIPTION OF AIDS BASED ON
•
•
CLINICAL FINDINGS IN FIVE PATIENTS
GOTTLIEB – 33 YR OLD ASSISTANT PROF.
SCHROFF – RESIDENT IN INTERNAL MED.
Gottlieb MS, Schroff et al. N Engl J Med 1981;305:1425
CITED 2,532 TIMES
SMALL CASE SERIES:
OBSTETRIC COMPLICATIONS ASSOCIATED
WITH THE LUPUS ANTICOAGULANT
D. WARE BRANCH MD
JAMES R. SCOTT MD
NEIL K. KOCHENOUR MD
ED HERSHGOLD MD
N ENGL J MED 1985;313:1322
SIGNIFICANCE: Based on 8 Patients
• NEWLY RECOGNIZED SYNDROME
• TREATABLE CAUSE OF FETAL DEATH
• MAJOR ANTIPHOSPHOLIPID ANTIBODY
RESEARCH EFFORT NEXT DECADE
•
CITED 583 TIMES
GOOD CASE REPORT
•
FIRST SUCCESSFUL EMBOLIZATION FOR
SEVERE POSTPARTUM BLEEDING*
* BY CHIEF RESIDENT @ U OF UTAH
BROWN BJ et al. Uncontrollable Postpartum Bleeding:
A New Approach to Hemostasis Through Angiographic
Embolization. Obstet Gynecol 1979;54:371.
•
CITED 132 TIMES
WORTHY OF HEALTHY SKEPTICISM
•
•
•
•
•
•
GUEST SPEAKERS
DATABASE STUDIES – Inaccurate
LARGE EPIDEMIOLOGIC STUDIES – Clinically
Irrelevant
DECISION ANALYSES – Soft data, assumptions
DRUG COMPANY SPONSORED – Ghostwriters
Bias, Overstated conclusions
ADVERTISEMENTS - Embarrassing
ALL TOO COMMON:
•
•
•
WIDELY QUOTED IN MEDIA
SAME DATABASE AS WITH
MAGNETIC FIELDS
? BIOLOGIC PLAUSIBILITY
NO INCREASED RISK IN
EPIDEMIOL. SAME MO. (no
publicity)
86 Papers – Conflicting Results
CONCLUSION – COFFEE
ASSOCIATED WITH
MISCARRIAGE
MY TAKE: ALL NONSENSE
WAKEFIELD 1998 PAPER IN LANCET RELATED
MMR VACCINATIONS IN CHILDREN TO AUTISM
•
•
•
DECLARED FRAUDULANT AND RETRACTED
IN 2010.
DID TREMENDOUS HARM
“SHOULD NEVER HAVE BEEN PUBLISHED”
Richard Smith, Former Editor of BMJ
•
SAME PATTERN OCCURING AGAIN
MULTIPLE WEAK ASSOCIATIONS ---
PUBLICIZED BY MEDIA
• LARGE DATABASE
• LOGESTIC REGRESSION
“MODELING”
• Odds Ratios (95% CI) =
1.21 (1.01-1.46)
• WIDELY PUBLICIZED BY
THE MEDIA
CONCLUSION: INDUCTION OF
LABOR AND AUGMENTATION
ASSOCIATED WITH AUTISM
INTERPRETIVE BIAS AND
OVERSTATED CONCLUSIONS
• MISLEADING RESULTS
- “3-FOLD INCREASED RISK …”
REALLY 1/MILLION VS 3/MILLION
- STATISTICALLY SIGNIFICANT
BUT CLINICALLY IRRELEVANT
OTHER BORDERLINE “ASSOCIATIONS”
WITH AUTISM DURING PAST YEAR
•
•
•
•
•
•
•
•
•
•
Flu & Fever During Pregnancy
Obese Mothers, Weight gain
Short Pregnancy Interval
Maternal Thyroid Dysfunction
ICSI (IVF)
Older Fathers
Antidepressants, Gluten Sensitivity
F.H. of Autoimmune Disease
Lyme Disease
Air Pollution, Pesticides
TRUTH
ALL OF THESE EPIDEMIOLOGIC
STUDIES WITH SMALL ODDS RATIOS
SHOULD END WITH THE STATEMENT:
“WE FOUND A WEAK ASSOCIATION
AND WE HAVE NO CLUE WHAT IT
MEANS”
David Grimes
CAUTION
•
•
•
•
•
•
75% CLINICAL TRIALS INDUSTRY FUNDED
PURPOSE - MAXIMIZE FINANCIAL RETURN
ONLY 1/3 NOW DONE IN UNIVERSITIES
WHO CONTROLLED DATA & WROTE PAPER
TIES WITH COMPANIES - MANY AUTHORS
& 59% OF EXPERTS WRITING GUIDELINES
5X > CHANCE OF FAVORABLE OUTCOME
WHEN COMMERCIALLY SPONSORED
PREDATORY OPEN ACCESS JOURNALS
• 8250 JOURNALS – 438 PUBLISHERS,
•
•
•
1/3rd IN INDIA
ADVERTISE FOR PAPERS
CHARGE AUTHOR FEE TO PUBLISH
304 VERSIONS OF FLAWED AND
FICTITIOUS WONDER DRUG PAPER
SUBMITTED  ACCEPTED BY 157
(>50%) SCIENCE 2013;342:60-65.
SUBTLE PROBLEMS
DESPITE THE RULES AS
INDUCTION AND MAINTENANCE
THERAPY FOR ULCERATIVE
COLITIS. NEJM 2013;369;699
BOTTOM LINE: MODERATE
EFFECT AT 6 & 52 WEEKS
• Complicated RCT – 211 centers in
34 countries
• Compared drug vs placebo
• 16 Authors – 10 with multiple ties to
industry – 7 Company Employees
• Company held & analyzed data
• No mention of probable cost
($100,000 per year)
• Company sponsored RCTs
• Marketing in the guise of
research – to publicize
expensive drugs
• Open Label, No Control Group,
Short-term, Physician payment
Example: VIOXX
WHAT NEEDS TO HAPPEN:
•
•
•
•
•
•
INDEPENDENT INVESTIGATORS
COMPARE TO PRESENT DRUG/Rx
INCLUDE COST COMPARISON
REJECT EPIDEMIOLOGY STUDIES
WITH ODDS RATIOS < 3-4
REQUIRE ABSOLUTE NUMBERS OR NNT
TRANSPARENCY – DISCLOSE ALL
CONFLICTS INCLUDING EDITORS
• CLINICALLY RELEVANT
• VALID STUDY DESIGN
• STRINGENT REVIEW
PROCESS
• RESULTS PHYSICIANS
CAN TRUST
DON’T WASTE YOUR TIME - OTHER
RELIABLE AND USEFUL SOURCES
•
•
•
•
COCHRANE LIBRARY
www.cochrane.co.uk
ACOG PRACTICE BULLETINS
www.greenjournal.org
UP TO DATE
www.uptodate.com
MEDICAL LETTER
www.medicalletter.com
CHALLENGE
TO TRANSLATE IMPERSONAL &
DOGMATIC STATISTICS INTO
PERSONALIZED CARE OF REAL
FLESH & BLOOD PEOPLE.
STILL IMPORTANT
•
•
•
•
COMPASSION
EMPATHY
COMMUNICATION
CLINICAL
JUDGEMENT
• COMMON SENSE
• ACCESS
REFERENCES
1.
2.
3.
4.
5.
6.
Scott JR. Show me the evidence. Obstet Gynecol 2002;100(3):403-4.
Ioannidis JPA. Why most published research findings are false. PLoS
Med 2005;2(8)e124
Scott JR. Improving systematic reviews for clinicians: a journal
editor’s view. Paediat Perinat Epidemiol 2008;22(1):38-41.
Scott JR. Evidence-based medicine under attack. Obstet Gynecol
2009;113(6):1202-3.
Grimes DA, Schulz KF. False alarms and pseudo-epidemics. The
limitations of obsevational epidemiology. Obstet Gynecol
2012;120(4):920-7.
Smith R. The Trouble With Medical Journals. The Royal Society of
Medicine Press Ltd. Edward Arnold Publishers. 2011, London, UK
7.
Bohannon J. Who’s afraid of peer review? Science 2013;342:60-7.