Transcript Statistics Making (Specific) Sense of It All

EBM Basics
Using an Article on Diagnosis:
Making (Specific) Sense of It All
Alex Djuricich, MD
Department of Medicine
Indiana University School of Medicine
Ambulatory Rotation 2005-2006
Case 1
• A 54 year old white male, newly diagnosed Type II DM (as
of 6 weeks ago), comes to office with foot complaints in
December. His neighbor, a long-time diabetic, just had a
BKA 1 week ago, after a long bout with PVD. The patient
just learned about diabetic complications in his DM class,
and is quite concerned about his feet. He has had “pale”
feet x 6 months. His feet seem “cool” to him. He admits
to foot pain with exercise. He gets back pain when he
walks more than 2 blocks. Meds: Glucotrol XL 5 mg qd.
On PE, he indeed has pale, cool, hairless feet bilaterally.
BP 134/82. His PT, DP, and popliteal pulses seem normal.
He has dependent rubor, and atrophic skin changes. His
ABI is 0.97.
Case 1
• You are concerned that this patient may have
peripheral vascular disease. You know that it
takes forever to get an arterial Doppler ordered,
and would rather know sooner than later.
• Given your physical examination findings, does he
have peripheral vascular disease (ischemia) or
not? Asked another way:
• In patients with risk factors for PVD, what
physical exam findings are useful diagnostic tools
for PVD?
Case 2
• A 65 year old men with no cardiac history comes to the ER
because of shortness of breath. He has had gradual
dyspnea over the past 3 weeks. He now cannot even go
upstairs to sleep without having severe dyspnea. He uses 3
pillows, which he previously did not do. He has woken up
at night, sometimes from dyspnea, sometimes from
increasing urination. He denies chest pain. PMH: BPH,
treated with doxazosin PSH: none. Rest of history,
unremarkable
• Meds: aspirin 325 mg qday, doxazosin 2 mg qhs
• On physical: RR 24 P 104 BP 178/98. No JVD. Lungs:
crackles in bases, o/w clear. Cor: RRR S1S2 no S3, +S4,
PMI laterally displaced Abd: normal Ext: no edema, O2
sat 94%. CXR: ? Haziness, borderline cardiomegaly EKG:
borderline LVH, NSSTTW changes
Case 2
• You are seeing this patient at 5 pm the Wednesday
before Thanksgiving. You are concerned about
CHF, and feel he should be admitted. However,
you realize the echo lab is closed until Monday
morning, and you have no documentation that this
is systolic dysfunction, although you are
concerned about this. You realize that you can’t
truly diagnose him as having “CHF”, or at least
systolic dysfunction, until later on Monday. You
are going into a cardiology fellowship soon, and
recently read an article on B-natriuretic peptide.
You want to order this test.
Case 2 clinical question
• In patients with suspected CHF referred for
echocardiography, what are the diagnostic
properties of B-natriuretic peptide (BNP)
levels for detecting abnormal ventricular
function?
Case 3
• You admit a 75 year old woman with community-acquired
pneumonia. She responds nicely to appropriate antibiotics but
her hemoglobin remains at 10 g/dl with a mean cell volume of
80. Her peripheral blood smear shows hypochromia, she is
otherwise well, and is on no incriminating medications. You
search her old labs and find out that her hemoglobin was 10.5
g/dl 6 months ago. She has never been investigated for anemia.
You discuss this patient on rounds and debate the use of ferritin
in the diagnosis of iron deficiency anemia. You admit to yourself
that you are unsure how to interpret a ferritin result and how
precise and accurate a serum ferritin is for diagnosing iron
deficiency anemia. You therefore form the question, "In an
elderly woman with hypochromic, microcytic anemia, can a low
ferritin diagnose iron deficiency anemia? "
Question
• You perform a MEDLINE search using
the MeSH terms "ferritin" and
"sensitivity and specificity" and find an
article on diagnosing iron deficiency
anemia in the elderly published in a
journal that your library has
• Am J Med 1990;88:205-9.
Goals
• Given a clinical scenario with a defined
question, to be able to critically appraise an
article on diagnosis by answering three
important questions
– Is is valid?
– Is it important?
– Can we use it for our patient?
Goals
• To understand definitions: sensitivity,
specificity, positive predictive value,
negative predictive value, likelihood ratio
• Make sensitivity and specificity more useful
• Distinguish between tests that rule in and
rule out disease
• Interpret likelihood ratios in the care of
patients
Valid?
• Was there an independent, blind comparison with
a reference standard of diagnosis?
• Was the test evaluated in an appropriate spectrum
of patients (like those we actually see)?
• Was the reference standard applied regardless of
the diagnostic test result?
• Was the test validated in a second group of
patients?
Was there an independent, blind
comparison with a reference
standard of diagnosis?
• Patients in study needed to have both the
diagnostic test in question AND the
reference standard
• Results of one should not be known to those
who are applying and interpreting the other
Was the diagnostic test evaluated
in an appropriate spectrum of
patients?
• Did the report include patients having all
the common presentations of the target
disorder (early and late manifestations, mild
and severe)
• Did the report apply the test to patients with
different disorders that are commonly
confused with the target disorder?
Was the reference standard
applied regardless of the
diagnostic test result?
• Investigators are tempted to forego applying
the reference standard when patients have a
negative diagnostic test result
• When the reference standard is risky or
invasive, it may be wrong to carry it out on
patients with negative test results
Was the test validated in a
second, independent
group of patients?
• Diagnostic tests are predictors, not
explainers, of diagnoses
• If a test performs well in a “test” set of
patients, we are assured about its accuracy
Important?
• Sensitivity
• Specificity
• Likelihood Ratios
What do we want when we
diagnose something using a test?
• To know what the probability of a disease is
given a positive or a negative test
• Rearranged: “given a positive test result,
what is the new probability of disease?”
Tests can be anything
• A lab
– a positive ANA for “diagnosing” lupus
• A radiology result
– CXR rib notching for diagnosing coarcation
– temporal lobe abnormality on MRI for HSV
encephalitis
• A specialty test
– echo showing aortic stenosis
Tests can be anything
• A physical finding
– heliotrope rash for dermatomyositis
– goiter for diagnosing Graves’ disease
– fixed, split S2 for atrial septal defect
• An element of the history
– fever for endocarditis
– forgetfulness for dementia
How do we get sensitivities
and specificities?
• Start with the grid
– Disease
– Test
– 2 x 2 table of these
Statistics Grid
Disease
Positive
Test
Negative
Present
Absent
True
positives
False
negatives
False
positives
True
negatives
Statistics Grid
Disease
Present
Absent
Positive
a
b
Negative
c
d
Test
Sensitivity and Specificity
• Sensitivity
• probability of a
positive test among
patients with disease
• On the graph:
• Specificity
• probability of a
negative test among
patients without
disease
Sensitivity and Specificity
• Sensitivity
• Specificity
• a/(a + c)
• d/(b + d)
Sensitivity and Specificity
• They do not give us the information we
need
• What we need is for the test to help us
diagnose a disease
• That’s why we order the test
– this helps us “confirm” (rule in) or “refute”
(rule out) the diagnosis
Sensitivity and Specificity
• Sensitivity
• Specificity
• SnNout
• in a highly Sensitive
test, a Negative test
rules out the disease
• SpPin
• in a highly Specific
test, a Positive test
rules in the disease
Predictive Values
• Positive Predictive Value:
– probability of a disease among patients with a
positive test
• Negative Predictive Value:
– probability of no disease among patients with a
negative test
Predictive Values
• Positive Predictive
Value
• Negative Predictive
Value
• a/(a + b)
• d/(c + d)
Statistics Grid
Disease
Present
Absent
Positive
a
b
Negative
c
d
Test
Statistics Grid - Alex does it
Sensitivity
0.8
Specificity
0.6
Disease
Positive Predictive Value 0.67
Negative Predictive Value 0.75 Present
Absent
Positive
40
20
Negative
10
30
Test
.
Sensitivity
Disease
Present
Absent
Positive
a
b
Negative
c
d
Test
.
Specificity
Disease
Present
Absent
Positive
a
b
Negative
c
d
Test
.
Positive Predictive Value
Disease
Present
Absent
Positive
a
b
Negative
c
d
Test
.
Negative Predictive Value
Disease
Present
Absent
Positive
a
b
Negative
c
d
Test
.
Statistics Grid
Sensitivity
Disease
Specificity
Positive Predictive Value
Negative Predictive Value
Present
Absent
Positive
a
b
Negative
c
d
Test
.
Practice 1
Negative Predictive Value
Disease
Specificity
Positive Predictive Value
Sensitivity
Present
Absent
Positive
p
q
Negative
r
s
Test
.
Practice 2
Disease
Present
Absent
Positive
80
40
Negative
20
60
Test
.
Specificity
=
60 / 60 + 40 = 0.6
Positive Predictive Value
=
80 / 80 + 40 = 0.67
Negative Predictive Value
=
60 / 60 + 20 = 0.75
Sensitivity
=
80 / 80 + 20 = 0.8
Likelihood Ratio
• When ordering a test, which tests will best
help us rule in or rule out disease?
• Initial assessment of likelihood of disease =
pre-test probability
• Final assessment of likelihood of disease =
post-test probability
Likelihood Ratio
• Pre-Test Probability
Do test
• Post-Test Probability
Likelihood Ratio
Probability of strep throat
Likelihood Ratio
Probability of patient with disease having a
given test result
Probability of patient without disease
having a given test result
Positive Likelihood Ratio (LR+)
Probability of patient with disease having a
positive test result
Probability of patient without disease
having a positive test result
Negative Likelihood Ratio (LR-)
Probability of patient with disease having a
negative test result
Probability of patient without disease
having a negative test result
Likelihood Ratios
• If ratio > 1: increases likelihood of disease
• If ratio < 1: decreases likelihood of disease
• If ratio is 1, the test did nothing to help rule
in or rule out disease
– by definition, this is anything a surgeon orders
Likelihood Ratios
• LR+
• LR-
sensitivity
1-sensitivity
1 - specificity
specificity
LRs - Example
Disease
LR+ =
LR- =
Present
Absent
Positive
40
20
Negative
10
30
Test
Summary
•
•
•
•
•
•
Sensitivity
SnNout
Specificity
SpPin
Positive Predictive Value
Negative Predictive Value
Positive Likelihood Ratio
Negative Likelihood Ratio
Can I apply this test to my patient?
• Is the diagnostic test available, affordable,
accurate, and precise in our setting?
• Can we generate a clinically sensible
estimate of our patient’s pre-test
probability?
• Will the resulting post-test probabilities
affect our management and help our
patient?
Is test available, affordable,
accurate and precise in our setting?
• Available: if no, don’t order, or find out how
to order
• Some diagnostic tests based on symptoms
or signs lose power as patients move from
primary care to secondary and tertiary care
– Only refer the positives, not the negatives
Can we generate a clinically
sensible estimate of patient’s pretest probability?
• From personal experience, prevalence
statistics, practice databases, or primary
studies
• Are study patients similar to our own?
• Is it unlikely that the disease possibilities or
probabilities have changed since this
evidence was gathered?
Will the resulting post-test
probabilities affect our
management and help our patient?
• Could it move us across a test-treatment
threshold?
• Would our patient be a willing partner in
carrying it out?
• Would the consequences of the test help our
patient reach his or her goals in all this?
Could it move us across a testtreatment threshold?
Probability of strep throat
Would our patient be a willing
partner in carrying out test?
• If patient refuses testing, difficult to order
the test
• No show (especially at Wishard)
Would the consequences of the
test help our patient reach his or
her goals?
• We find out things all the time
• Maybe the patient doesn’t want to know
Case 1
• Boyko, EJ Diagnostic utility of the history and
physical examination for peripheral vascular
disease among patients with diabetes mellitus. J
Clin Epidemiol 1997;50:659-68.
Likelihood Ratios
• Normal pulses
0.3-0.4
• Cool skin
1.2-1.5
• Color changes
0.84-0.85
• Trophic changes
1.4-1.6
Case 1
• These likelihood ratios are NOT impressive,
and therefore it is less likely that this patient
has PVD.
• An ABI is 1, and this patient’s arterial
Doppler confirms normal blood vessels and
ABIs, indicating NO peripheral vascular
disease
Case 2
• Krishnaswamy, P Utility of B-Natriuretic
Peptide Levels in Identifying Patients with
Left Ventricular Systolic or Diastolic
Dysfunction. Am J Med 2001;111:274-9.
Table
Cut point
Sens
Spec
+LR
-LR
345
36%
99%
36
0.65
160
65%
99%
65
0.35
110
75%
98%
37.5
0.26
75
85%
97%
28.3
0.15
62
89%
90%
8.9
0.12
49
91%
82%
5.1
0.11
Case 2
• Our patient’s BNP level was 220
• LR+ is very high (36-65, depending on
cutoff)
• According to article, sufficient to diagnose
“heart dysfunction”
Case 3
• Guyatt, GH. Laboratory diagnosis of iron
deficiency anemia: an overview. J Gen Int
Med 1992;7:45-53.
Case 3
Ferritin Fe def
Fe def
Result Present
Absent
< 15
474
59%
20
1.1%
LR+
52
15-34
175
22%
79
4.5%
4.8
35-64
82
10%
171
10%
1
65-94
30
3.7%
168
9.5%
0.39
>95
48
5.%
1332
75%
0.08
Case 3
• Ferritin result for patient is 13.
• This is in range with very high likelihood
ratio. Therefore, probably is correct, and
patient should be thought to have iron
deficiency anemia.
• Next: now, what should we do? (sometimes,
the answers generate more questions)
Summary of diagnostic tests
• Valid?
• Important?
–
–
–
–
Sensitivity and Specificity
Positive and Negative Predictive Value
Likelihood Ratio
SpPin and SnNout
• Can we apply to our patient?