Transcript Validity & Reliability of Analytic Tests

Validity and Reliability
of
Analytical Tests
Analytical Tests include both:
• Screening Tests
• Diagnostic Tests
•Two Important Objectives
To distinguish between people in the
population who have the diseases and
those who do not
To determine how good the test is in
separating populations of people with
and without the disease in question?
Epidemiological
Surveillance
vs.
Screening
Epidemiological
Surveillance
• What is it?
• Why do it?
Epidemiological
Surveillance
• Definition - ongoing & systematic
collection, analysis & interpretation of data
related to health, disease & conditions
• Two types
Passive Surveillance – uses available data or
reporting from health care provider or
regional health officer
Active Surveillance – periodic field visits to
health care facilities to identify new cases
• The present approach is the survey
Epidemiological
Surveillance
• Why do it?
Can help discover and control the
transmission of infectious diseases
Prevention and control programs can be
planned and implemented
Screening
• Definition - use of quick and simple
testing procedures to identify and
separate persons:
who have a disease from those that do not
OR
who are apparently (appear to be) well, but
who may be at risk of a disease, from
those who probably don’t have the
disease.
Terms Related to
Screening Tests
• Validity - relates to accuracy (correctness)
• Reliability - repeatability
• Yield - the # of tests that can be done in a
time period
Terms Related to Screening
Tests (cont’d)
• Sensitivity - ability of a test to
identify those who have disease
• Specificity - ability of a test to
exclude those who don’t have
disease
Terms Related to Screening
Tests (cont’d)
• Tests with dichotomous results – tests
that give either positive or negative
results
• Tests of continuous variables – tests that
do not yield obvious “positive” or
“negative” results, but require a cutoff
level to be established as criteria for
distinguishing between “positive” and
“negative” groups
An important public health consideration,
particularly in screening free-living
populations, is:
How good is the test at
identifying people
with the disease
and without the disease?
In other words:
If we screen a population, what
proportion of people who have the
disease will be correctly identified?
POPULATION
Test Results
With Disease
Without
Disease
Positive
True Positive
(TP)
False Positive
(FP)
Negative
False Negative
(FN)
True Negative
(TN)
True positives
Sensitivity =
True positives +
false negatives
=
TP
TP + FN
True positives
=
All persons with
the disease
X 100
True negatives
Specificity =
True negatives+
false positives
=
TN
TN + FP
True negatives
=
All persons
without the
disease
X 100
Percent false negatives = % of people with the disease who
were not detected by the test
FN
FN + TP
X
100
Percent false positives = % of people without the disease
who were incorrectly labeled by the
test as having the disease
FP
FP + TN
X
100
In the clinical setting, a more important
question is:
If the test results are positive (or
negative) in a given patient, what is
the probability that this patient has (or
does not have) the disease?
In other words:
What proportion of patients who test
positive (or negative) actually have (or do
not have) the disease in question?
Predictive Value
Pos. PV =
Neg. PV =
True Positives X 100 = %
TP + FP
True Negatives
TN + FN
X 100 = %
Biologic Variation of
Human Populations &
Diagnostic Issues
Distribution of Tuberculin Reactions
Bimodal Distribution
Easy to distinguish
between exposed group
and those not exposed.
Distribution of
Systolic Blood Pressure
Unimodal Distribution
With continuous variables, a
cutoff level must be established
to separate the hypertensive
group. Could choose based on
statistics, but better to base on
biologic considerations.
Effects of
Choosing
Different Cutoff
Levels for
Diabetes
Diagnosis in
Population with
50% Prevalence
PseudoReal World
Real World
The major issue with deciding
to set a cutoff high or low
is the problem of
false positives
and false negatives.
Possible Groups
with Dichotomous Test
True Disease
Status is Known,
as with dichotomous tests.
Grouping All Positives
and All Negatives
True Disease Status
is Unknown,
as with continuous variables.
Artificial
Cutoff
•Use of Multiple Screening
Tests
Sequential (Two-stage) Testing
Simultaneous Testing
Hypothetical Two-Stage Screening
Only Pos. Test 1
are given Test 2
Hypothetical Two-Stage Screening (cont.)
TEST 2 (Glucose Tolerance Test)
Sensitivity = 90%
Specificity = 90%
DIABETES
TEST
RESULTS
+
-
+
315
190
505
-
35
1710
1745
350
1900
2250
Net Sensitivity = 315/500 = 63%
Net Specificity = 7600 + 1710 = 98%
9500
Predictive Value
Prevalence & Predictive Value
Positive
As prevalence increases,
positive predictive value increases.
Prevalence & Predictive Value
Note: Test has
95% sensitivity and
95% specificity
Specificity &
Predictive
Value
As specificity
increases,
positive predictive
value increases.
As sensitivity increases,
positive predictive value
also increases, but to a
much lesser extent.
Specificity & Predictive Value
As specificity increases,
positive predictive value increases.
Reliability (Repeatability) of Tests
Results
reliable but
NOT valid
Results
reliable and
valid