Transcript Evidence-Based Practice: What is it? (Hint: It`s NOT Stats)

July 11, 2015
BOC Approved EBP CEUs
Evidence-Informed Clinical Practice for the Athletic Trainer:
Evidence Based Medicine for the
Athletic Trainer: What is It?
Jennifer M. Medina McKeon, PhD, ATC, CSCS
[email protected]
…while others wandered around…and dined with
the rich and powerful …we on the other hand
labored diligently…we first learned all the things
that were creditably discovered by the ancients;
then through deeds we both tested and
practiced them.
-Galen (129-217 CE [disputed])
Method of Medicine 9.4, 10.609K
“EBP is just a fad.”
…while others wandered around…and dined with
the rich and powerful …we on the other hand
labored diligently…we first learned all the things
that were creditably discovered by the ancients;
then through deeds we both tested and
practiced them.
-Galen (129-217 CE [disputed])
Method of Medicine 9.4, 10.609K
What EBP is not…
“All this EBP stuff just means
that they think that clinicians
are all supposed to become
researchers now”
What EBP is not…
“EBP means that we
all need to be doing
stats on everything.”
We all know that you can prove
anything with statistics…
…I recently proved that nobody likes
statistics, except a few professors
- Chottiner (1990)
What EBP is not…
“EBP means that someone should read all
the research that’s out there on a topic, and
tell me what to do.”
What EBP is not…
“Ideally, we would like to see a clinician encounter an
injury, and right there, pull out a laptop or tablet, and
look up the most recent article on that injury, and
incorporate what the study says right there on the
spot.”
What is the EBP?
Evidence-Based Practice
Best Available
Evidence
Patient
Values
Clinical
Experience
“Integration of the best available research evidence with clinical experience
and patient values to make clinical decisions” – Sackett et al, BMJ 1996.
Less Risk
of Bias
More
Subjects
Involved
Meta-analyses
Systematic Reviews
Randomized Clinical Trials
Cohort Studies
(Prospective)
Case-Control Studies
(Retrospective)
Case Series
Case Studies
Unpublished Clinical Observations
HARDER TO DO
More
Control
BETTER RESEARCH EVIDENCE
Hierarchy of Clinical Research Evidence
Critical Appraisal
Critical Appraisal
Critical Appraisal
More than Peer Review
Vital Step to making
INFORMED clinical
decisions
Combination of External
Evidence (the stuff you
find) & the Internal
Evidence (the stuff you
know)
Critical Appraisal
More than Peer Review
Vital Step to making
INFORMED clinical
decisions
Combination of External
Evidence (the stuff you
find) & the Internal
Evidence (the stuff you
know)
Critical Appraisal
More than Peer Review
Vital Step to making
INFORMED clinical
decisions
Combination of External
Evidence (the stuff you
find) & the Internal
Evidence (the stuff you
know)
Critical Appraisal
More than Peer Review
Vital Step to making
INFORMED clinical
decisions
Combination of External
Evidence (the stuff you
find) & the Internal
Evidence (the stuff you
know)
RESEARCH EVIDENCE
What is the BEST SPECIAL TEST???
What is the BEST TREATMENT???
What makes a test good?
Accurate (Valid)
Consistent results (Reliable)
YUCK – that sounds like research
RESEARCH EVIDENCE: The Best Tests
What makes a test good?
Accurate (Valid)
Consistent results (Reliable)
YUCK – that sounds like research
RESEARCH EVIDENCE: The Best Tests
Accurate (Valid)
Consistent results (Reliable)
What makes a test good?
SENSITIVITY
SPECIFICTY
Definition
Proportion of patients WITH the
injury who test POSITIVE
Proportion of patients WITHOUT
the injury who test NEGATIVE
100% Sensitivity
(Perfect Score)
Test correctly identifies every
person who HAS the injury
Test correctly identifies every
person who DOES NOT HAVE
the injury
Outcome
True Positive Rate
True Negative Rate
“Ideal” Test Result
Negative
Positive
“Ideal” Test Interpretation
(What to tell the Patient)
Definitely NEGATIVE
(Patient is OK)
Definitely POSITIVE
(Patient is Injured)
“The RULE”
RULE OUT (SnOUT)
If Test is Negative, You can Rule
Out injury (Patient is OK)
RULE IN (SpIN)
If Test is Positive, You can Rule In
injury (Patient is Injured)
Poor Test
Lots of FALSE NEGATIVES
Lots of FALSE POSITIVES
Adapted from http://www.aaronswansonpt.com/sensitivity-and-specificity/
Accessed January 23, 2015
RESEARCH EVIDENCE: The Best Tests
Accurate (Valid)
Consistent results (Reliable)
What makes a test good?
SENSITIVITY
SPECIFICTY
Definition
Proportion of patients WITH the
injury who test POSITIVE
Proportion of patients WITHOUT
the injury who test NEGATIVE
100% Sensitivity
(Perfect Score)
Test correctly identifies every
person who HAS the injury
Test correctly identifies every
person who DOES NOT HAVE
the injury
Outcome
True Positive Rate
True Negative Rate
“Ideal” Test Result
Negative
Positive
“Ideal” Test Interpretation
(What to tell the Patient)
Definitely NEGATIVE
(Patient is OK)
Definitely POSITIVE
(Patient is Injured)
“The RULE”
RULE OUT (SnOUT)
If Test is Negative, You can Rule
Out injury (Patient is OK)
RULE IN (SpIN)
If Test is Positive, You can Rule In
injury (Patient is Injured)
Poor Test
Lots of FALSE NEGATIVES
Lots of FALSE POSITIVES
Adapted from http://www.aaronswansonpt.com/sensitivity-and-specificity/
Accessed January 23, 2015
RESEARCH EVIDENCE: The Best Tests
What makes a test good?
Accurate (Valid)
Consistent results (Reliable)
SENSITIVITY
SPECIFICTY
Definition
Proportion of patients WITH the
injury who test POSITIVE
Proportion of patients WITHOUT
the injury who test NEGATIVE
100% Sensitivity
(Perfect Score)
Test correctly identifies every
person who HAS the injury
Test correctly identifies every
person who DOES NOT HAVE
the injury
Outcome
True Positive Rate
True Negative Rate
“Ideal” Test Result
Negative
Positive
“Ideal” Test Interpretation
(What to tell the Patient)
Definitely NEGATIVE
(Patient is OK)
Definitely POSITIVE
(Patient is Injured)
“The RULE”
RULE OUT (SnOUT)
If Test is Negative, You can Rule
Out injury (Patient is OK)
RULE IN (SpIN)
If Test is Positive, You can Rule In
injury (Patient is Injured)
Poor Test
Lots of FALSE NEGATIVES
Lots of FALSE POSITIVES
“New Rule”
High SNNN
High SPPP
High SeNsitivity, Negative test, High SPecificity, Positive test,
Adapted from http://www.aaronswansonpt.com/sensitivity-and-specificity/
Negative pt.
Positive pt.
Accessed January 23, 2015
RESEARCH EVIDENCE: The Best Tests
QUESTIONS TO ASK
Generalizability
Comparison
Bias
RESEARCH EVIDENCE: The Best Tests
Generalizability
QUESTIONS TO ASK
Is the Test Relevant to
your practice?
Comparison
Bias
Who was the Test
performed on?
RESEARCH EVIDENCE: The Best Tests
Generalizability
Comparison
Bias
RESEARCH EVIDENCE: The Best Tests
Generalizability
Comparison
Bias
QUESTIONS TO ASK
Who did the Test?
Was the Test compared to a Gold Standard?
Was the Test tried in a group of Patients who could conceivably have
the Injury?
RESEARCH EVIDENCE: The Best Tests
Generalizability
Comparison
Bias
RESEARCH EVIDENCE: The Best Tests
Generalizability
Comparison
Bias
QUESTIONS TO ASK
Did the testers “know” if the patient had the
Injury?
Did ALL patients tested also get tested with
the Gold Standard?
RESEARCH EVIDENCE: The Best Tests
QUESTIONS TO ASK
1.
Is the Test Relevant to your practice?
Generalizability
2. Who was the Test performed on?
Comparison
3.
Who did the Test?
4.
Was the Test compared to a Gold
Standard?
5.
Was the Test tried in a group of Patients
who could conceivably have the Injury?
6.
7.
Bias
Did
the testers “know” if the patient had
the Injury?
Did ALL patients tested also get tested
with the Gold Standard?
Improve your ability to
Critically Analyze Diagnostic Test research
IMPROVE YOUR OWN PRACTICE
PREDICTION
=
BEST
PREDICTORS
Get rid of these…
they don’t add much
Clinical Prediction Rules (the other CPR)
Application, not Development
Combination of 3 or more specific patient
characteristics that provides a quantifiable
probability for diagnosis (or prognosis or benefit
for a procedure)
Best combination of S/S to predict probability of
Injury
Calculated for Maximized Prediction
Support the decisions that you make
Considerations
Must be validated
Do not guarantee n outcome (a probability)
Used fore making predictions, not recommendations (what is likely to happen, not
what to do next)
“Suggest” vs. “Recommend” next course of action Clinical Decision Rule
Clinical Prediction Rules (the other CPR)
Application, not Development
Combination of 3 or more specific patient
characteristics that provides a quantifiable
probability for diagnosis (or prognosis or benefit
for a procedure)
Best combination of S/S to predict probability of
Injury
Calculated for Maximized Prediction
Support the decisions that you make
Considerations
Must be validated
Do not guarantee an outcome (they’re a probability)
Used fore making predictions, not recommendations (what is likely to happen, not
what to do next)
“Suggest” vs. “Recommend” next course of action (vs. Clinical Decision Rule)
RESEARCH EVIDENCE: The Best Treatments
What makes a treatment good?
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question
P
I
C
O
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question
P
I
C
O
Patient or Problem or Population
What are the most important
Characteristics of your Patient?
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question
P
I
C
O
Patient or Problem or Population
What are the most important
Characteristics of your Patient?
Independent Variable
(Intervention or Assessment or
Exposure)
What is the main Intervention, Prognostic
factor, Diagnostic factor, or Exposure you
are considering for your patient?
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question
P
I
C
O
Patient or Problem or Population
What are the most important
Characteristics of your Patient?
Independent Variable
(Intervention or Assessment or
Exposure)
What is the main Intervention, Prognostic
factor, Diagnostic factor, or Exposure you
are considering for your patient?
Comparison or Control
What is the main alternative to the
Intervention / Assessment / Exposure?
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question
P
Patient or Problem or Population
What are the most important
Characteristics of your Patient?
Independent Variable
(Intervention or Assessment or
Exposure)
What is the main Intervention, Prognostic
factor, Diagnostic factor, or Exposure you
are considering for your patient?
C
Comparison or Control
What is the main alternative to the
Intervention / Assessment / Exposure?
O
Outcome of Interest
What is considered “success”
I
RESEARCH EVIDENCE: The Best Treatments
Posing a Clinical Question
P
Patient or Problem or Population
What are the most important
Characteristics of your Patient?
Independent Variable
(Intervention or Assessment or
Exposure)
What is the main Intervention, Prognostic
factor, Diagnostic factor, or Exposure you
are considering for your patient?
C
Comparison or Control
What is the main alternative to the
Intervention / Assessment / Exposure?
O
Outcome of Interest
What is considered “success”
I
RESEARCH EVIDENCE: The Best Treatments
QUESTIONS TO ASK
The PEDro Scale http://www.pedro.org.au/english/downloads/pedro-scale/
Accessed January 23, 2015
RESEARCH EVIDENCE: The Best Treatments
Generalizability
Comparison
Bias
RESEARCH EVIDENCE: The Best Treatments
Generalizability
QUESTIONS TO ASK
Who was the Treatment
performed on?
Did all of the patients present
similarly?
Comparison
Bias
RESEARCH EVIDENCE: The Best Treatments
Generalizability
Comparison
Bias
RESEARCH EVIDENCE: The Best Treatments
Generalizability
QUESTIONS TO ASK
Did they compare
the Groups?
Were they meaningfully
different?
(In other words, were the patients who
received the treatment Really better
than the controls?)
Comparison
Bias
RESEARCH EVIDENCE: The Best Treatments
Generalizability
Comparison
Bias
RESEARCH EVIDENCE: The Best Treatments
Generalizability
QUESTIONS TO ASK
Did the individuals involved
already have an “idea” of what
should happen?
In other words: No Blinding of
- Patients
- Therapists
- Investigators
Comparison
Bias
RESEARCH EVIDENCE: The Best Treatments
Generalizability
Comparison
Bias
RESEARCH EVIDENCE: The Best Treatments
QUESTIONS TO ASK
1.
Who was the Treatment performed on?
2.
Who Did all the patients present similarly?
3.
Did they compare the groups?
4.
Where they meaningfully different?
5.
Did the individuals involved have an
“idea” of what should happen
Generalizability
Comparison
Bias
Improve your ability to
Critically Analyze Intervention research
IMPROVE YOUR OWN PRACTICE
RESEARCH EVIDENCE: Summary
We focused on Foundational concepts of EBP
Specifically, we covered:
• Sackett’s 3-pronged EBP model
• Levels of Evidence
• Critical appraisal - Diagnosis (Diagnostic accuracy & CPRs)
• Critical appraisal - Intervention (PICO, Therapeutic studies)
Evidence-Based Practice
Best Available
Evidence
Patient
Clinical
Values
Experience
“Integration of the best available research evidence with clinical experience
and patient values to make clinical decisions” – Sackett et al, BMJ 1996.
Read not to contradict
and confute, nor to
believe and take for
granted…but to weigh
and consider
- Francis Bacon (1561-1626)
[email protected]
“Commitment to Excellence”