Transcript Descition making

Medical decision making
Predictive values
•
•
•
•
57-years old, Weight loss, Numbness, Mild fewer
What is the probability of low back cancer?
Base on demographic prevalence ~20%
Should he receive the
• low-cost ESR-test with sensitivity of 78% and specificity of
67%
• Expensive MRI scanning with sensitivity and specificity of 95%
• surgery at once
• be send home
2
The threshold model for a patient with low back pain
•
The doctor worries about cancer,
should the doctor send him home,
test, or treat him?
3
The accuracy of a test
•
•
Sensitivity; The ability to detect patients with the condition
• Definition: The probability of a positive result in patients who have the
condition: True Positive rate
• High sensitivity ↔ few false negative
Specificity; The ability to detect patients without the condition
• Definition: The probability of a negative result in patients who do not have the
condition: True negative rate or1- False Positive rate
• High specificity ↔ few false positive
4
The accuracy of a test
•
Accuracy =
(TP + TN) / total N
Known truth
Test
Positive D+
Negative D-
Positive T+
TP (true-positive)
FP (false-positive)
Negative T-
FN (false-negative)
TN (true-negative)
5
The accuracy of a ECG test for myocardial infarction
•
•
•
•
Sensitivity:
TP ratio: 6/31 = 0.19
Specificity:
TN ratio: 59/72 = 0.82
Known truth
MI Present
Test
MI Absent
ST > 5mm
TP: 6
FP: 13
ST < 5mm
FN: 25
TN: 59
31
72
Total
6
Predictive values
•
•
•
•
•
•
•
•
Assuming 20% chance for this specific
patient
ESR test has sensitivity = 78% and
specificity = 67% (Joines et al. 2001)
TP = 78% of 200 = 156
TN = 67% of 800 = 536
FN = 200 – 156 = 44
FP = 800 – 536 = 264
156 + 264 = 420 tested positive
44 + 536 = 580 tested negative
D+
Test
D-
T+
TP: 156
FP: 264
420
T-
FN: 44
TN: 536
580
200
800
1000
Total
7
Predictive values
•
•
•
•
PV+ = 156/420 = 0.37
PV- = 536/580 = 0.92
If the test is positive we are 37% sure
that it is spinal cancer
If it is negative we are 92 % sure it is
not
D+
Test
D-
T+
TP: 156
FP: 264
420
T-
FN: 44
TN: 536
580
200
800
1000
Total
8
Predictive values of MRI scan is ESR tested positive
•
•
•
•
Assuming 37% chance for this specific
patient
FMR test has sensitivity = 95% and
specificity = 95% (Joines et al. 2001)
PV+ = TP/(TP+FP) = 0.92
PV- = TN/(TN+FN) = 0.97
D+
Test
D-
T+
TP: 351.5
FP: 31.5
383
T-
FN: 18.5
TN: 598.5
617
370
630
1000
Total
9
Likelihood ratio
•
•
•
•
•
Assuming 20% chance for this specific patient
ESR test has sensitivity = 78% and specificity = 67% (Joines et al. 2001)
Pretest odds = Prior probability / (1 - Prior probability) = 0.2 / (1 – 0.2) = 0.25
Likelihood ratio (LR) = sensitivity / false positive rate = 0.78 / (1 – 0.67) = 2.36
Posttest odds = 0.25*2.36 = 0.59
10
Probability vs. odds
•
•
Posttest odds = 0.59
PV+ = 0.37
p
o
o
 p
1 p
1 o
0.59
Posterior probability =
 0.37
1+0.59
11
Negative likelihood ratio
•
•
•
•
Positive likelihood ratio (+LR) = 2.36
Is the likelihood ratio that he has the disease if he is tested positive
Negative likelihood ratio (-LR)
Is the likelihood ratio that he has the disease if he is tested negative
+
-
TP ratio
LR=
FP ratio
FN ratio 0.22
LR=

 0.33
TN ratio 0.67
12
Test with continuous outcome
•
•
•
What if the test outcome is continuous? Which threshold should be chosen?
Optimizing Specificity and sensitivity
Increasing sensitivity at to loss of specificity
13
Receiver operating characteristics (ROC) curve
•
•
•
•
•
X-axis: 1-Specificity
Y-axis: Sensitivity
The ROC curve describes the test.
Poor test → large overlap → ROC curve close
to diagonal
Good test → little overlap → ROC curve close
to vertical / horizontal
14
Receiver operating characteristics (ROC) curve
From wikipedia
15
ROC curve to test beast cancer by mammography
Status
Normal
1
Benign
2
Probably
benign
3
Suspicious
4
Malignant
5
Total
Cancer
1
0
6
11
12
30
No Cancar
9
2
11
8
0
30
16
ROC curve to test beast cancer by mammography
•
•
How good is the test?
Where to put the threshold?
Status
Normal
1
Benign
2
Probably
benign
3
Suspicious
4
Malignant
5
Total
Cancer
1
0
6
11
12
30
No Cancar
9
2
11
8
0
30
Threshold
<1
1.5
2.5
3.5
4.5
>5
TPR
(sensitivity)
30/30 =
1.00
29/30 =
0.97
29/30 =
0.97
23/30 =
0.77
12/30 =
0.40
0/30 = 0.00
FPR
(1-specificity)
30/30 =
1.00
21/30 =
0.70
19/30 =
0.63
8/30 = 0.27
0/30 = 0.00
0/30 = 0.00
17
ROC curve to test beast cancer by mammography
•
•
How good is the test?
Where to put the threshold?
Threshold
<1
1.5
2.5
3.5
4.5
>5
TPR (sensitivity)
1.00
0.97 0.97 0.77 0.40 0.00
FPR (1-specificity)
1.00
0.70 0.63 0.27 0.00 0.00
18
The area under the ROC curve
•
•
•
•
The area under the ROC gives the intrinsic accuracy of a diagnostic test and can be
interpreted in several ways (see Hanley et al.):
The average sensitivity for all values of specificity
The average specificity for all values of sensitivity
The probability that the diagnostic score of a diseased patient is more of an
indication of disease than the score of a patient without the disease.
19