Transcript Document

Diagnostic Tools
Basic Medical Statistics Course: Module C
October 2010
Wilma Heemsbergen
Diagnostic tools
Diagnostic tools are used to: classify / distinguish:
- sick people and healthy people (special case: screening);
- patients with good prognosis and bad prognosis;
- patients with different stages of the disease.
The results of the diagnostic procedures are used to:
- decide whether further diagnosis is needed;
- determine which treatment is optimal;
- estimate the further course of the disease.
-…
The accuracy of a test to discriminate diseased cases from
normal cases is essential, and the subject of many studies.
Gold standard
The gold standard, in terms of testing, is the test that most accurately
determines the presence of a disease. It may not actually be used in a clinical
setting, because of expense, time, degree if invasiveness or expertise, but it is
used to determine the sensitivity and specificity of other tests. However, it is
important to realize that most times, not even the gold standard is perfect.
Therefore there is some ambiguity to those test characteristics.
In general, "gold standard" refers to the best available means of doing
something. The "gold standard" refers to the most validated test that makes a
diagnosis. The new test is compared to the most accurate test in use at the
time.
Each cut-off is a trade-off between the %
false negative and % false positive cases.
Statistics
Sensitivity (%): probability that a test result will be positive when the disease is
present (true positive rate) = a / (a+b)
Specificity (%): probability that a test result will be negative when the disease is
not present (true negative rate) = d / (c+d)
Positive predictive value (%): probability that the disease is present when the
test is positive = a / (a+c)
Negative predictive value (%): probability that the disease is not present when
the test is negative. = d / (b+d)
Example
Exercise Stress Test (EST) for the diagnosis of Coronary Artery Disease (CAD)
CAD
y
n
930 63.4%
EST + 815 55.6% 115 7.8%
- 208 14.2% 327 22.3% 535 36.5%
1023 69.8% 442 30.1% 1465/100%
TPF = 815/1023=0.797
Sensitivity = 79.7%
TNF = 327/442=0.740
Specificity = 74.0%
PPV = 815/930=88%
NPV = 327/535=61%
ROC Curve
A Receiver Operating Characteristic Curve describes the whole
set of potential combinations that are possible (False Positive
Fraction – True Positive Fraction).
Sensitivity is plotted in function of (100Specificity) for different cut-off points. Each
point represents a sens/spec pair
corresponding to a particular cut-off.
A test with perfect discrimination (no
overlap) has a ROC plot that passes
through the upper left corner (100%
sensitivity, 100% specificity). Therefore the
closer the ROC plot is to the upper left
corner, the higher the overall accuracy of
the test.
AUC (area under the curve) summarizes
the discriminating power (0.5 – 1.00).
AUC 0.5: discriminating power is as good
as throwing a coin (50% chance for correct
answer).
Likelihood ratio
Positive likelihood ratio: ratio between the probability of a positive test result
given the presence of the disease and the probability of a positive test result
given the absence of the disease, i.e.
= True positive rate / False positive rate = Sensitivity / (1-Specificity)
Negative likelihood ratio: ratio between the probability of a negative test result
given the presence of the disease and the probability of a negative test result
given the absence of the disease, i.e.
= False negative rate / True negative rate = (1-Sensitivity) / Specificity
Association vs. Classification
A factor that is significantly associated with a disease (significant
HR/OR), is not by definition a good marker to classify this disease.
(Figure 4)
Evaluation of the diagnostic value of 2
tumor markers: separately, combined.
Example
Questions
Read the (relevant parts of the) paper. Answer the following questions.
What is the optimal cut-off with the lowest probability of false positives.
What is the optimal cut-off with the lowest probability of false negatives
Calculate for both cut-offs in a screening population of 10,000 people with 10 cancer cases:
- how many persons receive a positive test result (indicating cancer), and how many of
them will be definitely diagnosed with cancer after additional conclusive tests.
- what is the number of persons with a false positive result,
- what is the number of persons with a false negative result.
Answer the same questions for a hospital
population of 1,000 people with 500 cases.
Example
The Lancet
2002;
359: 572-77
(360: 169)
Prediction Models
Prediction model: construct a model to predict a certain
outcome by (a number of) predictive variables (e.g. the
outcome cancer is predicted based on a tumor marker level).
There is always a risk of overfitting the data.
We are interested in the model with the greatest predictive
validity (i.e. expected to perform optimally in future samples).
Therefore validation of the model is needed:
1- within the same data set: by resampling (e.g. cross validation).
2- in an external data set.
Prediction Models
Clin Cancer Res 2008;14: 5977. Taylor et al.
Validation of Biomarker based risk prediction models.
Also interesting: Int J Radiat Oncol Biol Phys. 2009 1;74(2):355-62.
Development and external validation of prognostic model for 2-year
survival of non-small-cell lung cancer patients treated with
chemoradiotherapy. Dehing-Oberije et al.
Exercise
SPSS file: trial_rt.sav
Construct a ROC curve for the following:
- predicting factors PSA, Gleason, T stage
- to classify: * death due to prostate cancer
* death due to other causes
Construct a combined factor of predictive factors for death due to
prostate cancer, what happens to the discriminating power of the
model (AUC).
RECODE cod (SYSMIS=0) (1=1) (ELSE=0) INTO cod_prost.
EXECUTE
RECODE cod (SYSMIS=0) (1=0) (ELSE=1) INTO cod_other.
EXECUTE.
Death prostate cancer