Transcript Diagnostics sensitivity - Genomics & Bioinformatics at Purdue

Screening Test for Occult Cancer
100 patients with occult cancer:
95 have "x" in their blood
SENSITIVITY
100 patients without occult cancer: SPECIFICITY
95 do not have "x" in their blood
5 out of every 1000 randomly selected
individuals will have occult cancer
PREVALENCE
2X2
Table
"x" present "x" absent
Occult Cancer
Present
475
25
500
Occult Cancer
Absent
4,975
94,525
99,500
5,450
94,550
100,000
If a patient has “x” in his blood, chance of occult cancer
is 475 / 5450 = 8.7%
Standard Terminology
Test
positive
Test
negative
Disease
Present
True
Positives
(TP’s)
False
Negatives
(FN’s)
TP + FN
Disease
Absent
False
Positives
(FP’s)
True
Negatives
(TN’s)
FP + TN
TP + FP
FN + TN
Entire
Population
Definitions
SENSITIVITY
=
TP
= P(T +|D +)
TP + FN
SPECIFICITY
=
TN
= P(T -|D -)
FP + TN
PV + = PREDICTIVE VALUE
=
TP
= P(D +|T +)
TP + FP
Positive Predictive Value Formula
(Sens)(Prev)
PV+ =
(Sens)(Prev) + (1-Spec)(1-Prev)
Detection of Prostatic Cancer by Solid-Phase
Radioimmunoassay of Serum Prostatic Acid Phosphatase
Editorial
“The clear implication of the
accompanying report is that
mass screening on the basis of
a blood test alone can reverse
this gloomy experience [of fatal
delays in diagnosis of prostate
cancer].”
New England Journal of Medicine
December 22, 1977
Medical
Journal
Advertisement
to Physicians
Medical
Journal
Advertisement
to Physicians
Posed by a Professional Model
Medical
Journal
Advertisement
to Physicians
Advertisement
“(You should be aware of) a new
blood test called the Male-P.A.P. test
... a new, more sensitive method that
your physician can use to detect
chemical signals of a cancerous
growth in the prostate. ... And even
though all lab tests must be ordered
by a physician, we believe that you
should know the facts.”
New York Times, January 21, 1979
Sensitivity
# of
patients
Patients with prostate
cancer
# of positive
tests
sensitivity
113
79
70%
Stage I
24
8
33%
Stage II
33
26
79%
Stage III
31
22
71%
Stage IV
25
23
92%
Specificity
# of
patients
Patients without prostate 217
cancer
# of positive
tests
specificity
13
94%
Normal controls
50
0
BPH
36
2
After total prostatectomy
28
1
Other cancers
83
9
Misc. GI disorders
20
1
Use As Screening Test
• Without rectal examination:
– Sensitivity = 70% Specificity = 94%
– Prevalence 33/100,000
– PV+ = 0.41% (i.e., 1 in 244 subjects)
• With rectal examination:
– Sensitivity = 33% Specificity = 94%
– Prevalence 33/100,000
– PV+ = 0.19% (i.e., 1 in 526 subjects)
When is the test useful for screening?
Suppose patient has a nodule on rectal
examination:
– Sensitivity (Stage 2 of disease) = 79%
– Specificity = 94%
– Prevalence = 50% !!
PV+ = 93% (chance of cancer if acid
phosphatase is positive)
PV- = 82% (chance that there is no cancer if
acid phosphatase is negative)
Predictive Values in Patients with a
Nodule
PAP+
93%
PAP-
18%
50%
Combining Tests For
Screening:
If a prostate biopsy is now performed, it
needs to be considered as another test.
Specificity = 100%
Sensitivity depends on talent and statistics of
surgeon doing the procedure
Prevalence is 50% if acid phosphatase has
not been measured, but is 93% if acid
phosphatase is positive and 18% if acid
phosphatase is negative.
Sequential
Testing
BX+
PAP+
100%
93%
50%
BX+
PAP-
18%
100%
Chance of Cancer after Negative Biopsy
Sensitivity of Biopsy
50%
70%
90%
Acid Phosphatase positive
(93% chance before biopsy)
87%
80%
56%
Acid Phosphatase negative
(18% chance before biopsy)
10%
6%
2%
Sequential
Testing
PAP+
BX+
100%
BX-
56%
BX+
100%
BX-
< 2%
93%
50%
PAP-
18%
BAYES’ THEOREM
PV + =
(Sens )(Prev )
(Sens )(Prev ) + (1-Spec )(1-Prev )
OR
P(T +|D +) P(D +)
P(D |T ) =
P(T +|D +) P(D +) + [1-P(T -|D -)] [1- P(D +)]
+
+
Typical Assumptions with the
Use of Bayes' Theorem
• Completeness (for example, all men either
have or do not have prostate cancer; there
are no other possibilities)
• Mutual exclusivity (for example, if a man
has prostate cancer, he cannot
simultaneously NOT have prostate cancer)
• Conditional independence (for example,
acid phosphatase and a biopsy result ARE
conditionally independent tests; rectal
exams and acid phosphatase may NOT be
conditionally independent)
References
Foti et al. “Detection of prostate cancer by solidphase radioimmunoassay of serum prostatic
acid phosphatase.” New England Journal of
Medicine 297:1357-1361 (1977)
Watson, R.A. and Tang, D.B. “The predictive
value of prostatic acid phosphatase as a
screening test for prostatic cancer.” New
England Journal of Medicine 303:497-499
(1980)
Berwick, D.M., Fineberg, H.V., and Weinstein,
M.C. “When doctors meet numbers.”
American Journal of Medicine 71:991 (1981)
What is a “Positive Test”?
• All the analysis has assumed that it is
clear whether a test is positive or
negative
• In reality, many tests involve
continuous values so that one result
may be “more positive” than another
• How should one define the cut-off at
which a test is judged to be
abnormal?
Continuously Valued Variables
Normal
True
Negatives
“Normal”
cutoff
False
Positives
Result
False
Negatives
True
Positives
Diseased
Continuously Valued Variables
Normal
“Normal”
cutoff
• Fewer false positives
(more “conservative”)
• More false negatives
• Higher specificity
• Lower sensitivity
Result
Diseased
Continuously Valued Variables
Normal
“Normal”
cutoff
Result
• Fewer false negatives
(more “aggressive”)
• More false positives
• Higher sensitivity
• Lower specificity
Diseased
Receiver Operating Characteristic
(ROC) Curves
True Positive Rate = Sensitivity
Test B
Test A
ROC curve shifts to left,
indicating the new test
(B) is “better” – or a
better indicator to
disease presence (more
discriminatory).
False Positive Rate = 1 - Specificity
The Importance of the Gold
Standard
• Evaluating the value of a new test requires having
some other method for determining “truth”
• Methods for determining truth are called gold
standards
• Gold standards are often expensive, time
consuming, uncomfortable, or risky
–
–
–
–
Biopsies
Major invasive procedures or surgery
Autopsies
Integrated opinions of “super experts”
• We often seek simple, inexpensive, rapid, and
safe tests that can perform almost as well as the
gold standard