Transcript Biostatistics for High Value Testing and Treatment

Biostatistics for High Value
Testing and Treatment
Fellowship HVC Curriculum 2016-2017 • Presentation 3 of 7
Learning Objectives
• Explain basic statistical concepts for diagnostic testing: sensitivity and
specificity, predictive values, and likelihood ratios.
• Describe the impact of of pretest probability and test characteristics on
clinical decision making.
• List the elements of a high value screening test.
• Link these basic statistical concepts to the practice of high value care.
• Apply the following statistical tools to make high value treatment decisions:
risk ratios, absolute and relative measures, and numbers needed.
Brief Review of Biostatistical Concepts1
• Sensitivity: The ability to detect people who do have
disease
• Specificity: The ability to detect people who do not
have disease
• Positive Predictive Value: The likelihood that a person
with a positive test result actually has disease
• Negative Predictive Value: The likelihood that a person
with a negative test result truly does not have disease
Sensitivity
“Be sensitive to those who have disease”
Disease Positive
Disease Negative
Test Positive
True Positive
False Positive
Test Negative
False Negative
True Negative
True Positive
True Positive+ False
Negative
Specificity
“Negative people get specific”
Disease Positive
Disease Negative
Test Positive
True Positive
False Positive
Test Negative
False Negative
True Negative
True Negative
True Negative+ False
Positive
Positive Predictive Value
Disease Positive
Disease Negative
Test Positive
True Positive
False Positive
Test Negative
False Negative
True Negative
True Positive
True Positive+ False
Positive
Negative Predictive Value
Disease Positive
Disease Negative
Test Positive
True Positive
False Positive
Test Negative
False Negative
True Negative
True Negative
True Negative+ False
Negative
Diagnostic testing
Pretest probability
Disease prevalence
Diagnostic test
Sensitivity, specificity
Posttest probability
PPV, NPV
Role of Diagnostic Tests
• To reduce uncertainty regarding a specific patient’s diagnosis
• Generally most appropriate in the presence of intermediate
(10% to 90%) pretest probability of a disease (e.g., Centor
criteria for Strep pharyngitis)
• Test characteristics (i.e., likelihood ratios) should be
considered before ordering a test to help determine whether
a given test would significantly change posttest probability
(and thus affect management)
Likelihood Ratios: What do they mean?
• Likelihood ratios combine the sensitivity and specificity of a test with pretest
probability of disease in a specific patient, avoiding the need to perform
statistical calculations based on test characteristics and prevalence data.
LR+ = sensitivity / (1-specificity)
LR- = (1-sensitivity) / specificity
• They provide a sense of how “powerful” a test is in influencing our pretest
probability of disease.
• Likelihood ratios may be positive [LR(+)], which are used when assessing for
the presence of disease when a test result is positive, and negative [LR(-)],
which are used when excluding disease with a negative test result.
• Likelihood ratios may also be calculated sequentially with serial testing, if
needed.
Likelihood Ratios
Using likelihood ratios:
1. Use the estimated pretest probability
of disease as an anchor on the left side
of the graph.
2. Draw a straight line through the known
likelihood ratio, either (+) or (-).
3. Where this line intersects the graph on
the right represents the posttest
probability of disease.
Using Fagan’s Nomogram
1. Designate your pretest probability on
left line (X); example = 28%
2. Draw a line to the LR+ (or LR-) for the
test (O); example LR+ = 4
3. Draw a straight line through the two
points and extend it to the right side to
determine posttest probability (where
this line intersects the graph on the
right represents the posttest
probability of disease = 65%)
o
x
Likelihood Ratios
• A likelihood ratio of 1 indicates that the test has no influence on the pretest
probability; a likelihood ratio >1 increases the pretest probability, and a likelihood
ratio <1 decreases the pretest probability.
• In general:
•
•
•
•
•
•
A LR(+) of 10 increases the pretest probability by ~45%
A LR(+) of 5 increases the pretest probability by ~30%
A LR(+) of 2 increases the pretest probability by ~15%
A LR(-) of 0.5 decreases the pretest probability by ~15%
A LR(-) of 0.2 decreases the pretest probability by ~30%
A LR(-) of 0.1 decreases the pretest probability by ~45%
• Even if specific LR calculations are not performed for a patient, simply knowing
the LR of a test helps in making testing decisions.
Examples of Common Diseases, Tests, and Likelihood Ratios
Disease
Acute
cholecystitis
Acute
pulmonary
embolism
Test/Result
Abdominal
ultrasound
Likelihood Ratio
LR(+) = 23.8
LR(−) = 0.05
Appendicitis
= 18.8
Abdominal CT LR(+)
LR(−) = 0.06
Clostridium
difficile colitis
C. difficile
LR(+) = 19.6
toxin–positive LR(−) = 0.02
SLE
Antinuclear
antibody
Pulmonary CT LR(+) = 29.1
angiography
LR(−) = 0.05
LR(+) = 4.5
LR(−) = 0.125
Diagnostic testing
Likelihood ratio
Pretest probability
Disease prevalence
Diagnostic test
Sensitivity, specificity
Posttest probability
PPV, NPV
Using Likelihood Ratios: Small Group Exercise
• Use the nomograms and LRs given to come up with the
posttest probability of disease for the cases provided.
• Focus on the diagnostic process:
• Estimate the pretest probability of disease in your patient.
• Evaluate how testing would influence your pretest probability of
disease using the LR provided and the nomogram.
• Decide if you think the test is high value based on this exercise.
• Be prepared to briefly summarize your findings and share
them with the larger group.
Role of Screening Tests
• To detect asymptomatic and early stage disease
• Should be highly sensitive and highly specific to pick up
most cases of true disease and avoid false positives
• Targeted toward populations with a higher disease
prevalence (high positive predictive value)
• Should be relatively safe and cost-effective
• Should screen for diseases in which early identification and
treatment have been demonstrated to improve clinical
outcomes
Common Harms Associated with
Screening
False positive results
• Primary goal of screening: find specific diseases
Maximize sensitivity at cost of specificity
 False positives
• Can lead to incorrect labeling, inconvenience, expense, and
physical harm in follow-up tests
Selection bias (healthy volunteer bias)
Common Harms Associated with Screening
• Lead Time Bias: Make
diagnosis of disease
earlier → false survival
benefit
• Length Time Bias:
“Overdiagnosis”
and“Pseudodisease”3
Screening Cascade2
Value Framework2
Screening Value Cases
• Discuss the following screening cases, and use
handout to guide your decisions:
• 45-year-old woman asking about mammography
• 70-year-old man with ESRD on HD, CAD, COPD, and
uncontrolled DM asking for colonoscopy
• 35-year-old woman, new patient, had a negative Pap
smear 2 years ago, now asks for a repeat Pap test
because “that’s what she’s always had”
Screening Smarter
• Screen less frequently.
• Don’t screen patients with a life expectancy less than
10 years.
• Discuss potential downstream testing with patient
before ordering initial screening test.
• Use higher threshold for positive result.
• Understand basic test characteristics and limitations, as
well as an individual patient’s goals and values.
Biostatistical Principles in Treatment
• High value therapeutic decision
making requires understanding
the effectiveness of different
treatment options, and balancing
potential benefits with both
medical and financial costs.
Interpreting Therapeutic Statistics
• Absolute risk (AR) or Event Rate (ER): The probability of an
event occurring in a group during a specified time period.
patients with event in group/total patients in group
• Relative risk (RR): The ratio of the probability of developing a
disease with a risk factor present to the probability of
developing the disease without the risk factor present.
experimental event rate/control event rate
• Absolute risk reduction (ARR):The difference in rates of events
between experimental group (EER) and control group (CER)
experimental event rate - control event rate
Interpreting Therapeutic Statistics
• Relative risk reduction (RRR): The ratio of absolute risk reduction
to the event rate among controls
experimental event rate - control event rate/ control event rate
• Number needed to treat (NNT): Number of patients needed to
receive a treatment for one additional patient to benefit
1/absolute risk reduction
• Number needed to harm (NNH): Number of patients needed to
receive a treatment for one additional patient to be harmed
1/absolute risk increase
Most Useful Terms for Treatment Options
• Use absolute risk, absolute risk reduction, and
numbers needed whenever possible.
• Relative comparisons may exaggerate uncommon
outcomes. Interventions that reduce the rate of a
disease from 40% to 20% and 4% to 2% each have a
relative risk reduction of 50%. However, the absolute
risk reduction (ARR) for the first case is 20%, whereas
the ARR for the second case is 2%.
Most Useful Terms for Treatment Options
• Numbers needed are useful indicators of the
clinical impact of an intervention because they
provide a sense of magnitude expected from the
intervention.
• Statistical significance ≠ clinical importance,
especially for large studies with uncommon
outcomes.
Cost-effectiveness
3
“Cost-saving”
Reduces cost,
Improves health
•
•
Costs money,
Improves health
Costs money,
Worsens health
Quality-adjusted life-year (QALY) is a generic measure of disease burden, including both the quality and
the quantity of life lived. 1 QALY = 1 year in perfect health
Measures that cost money but improve health can be further categorized by their cost,
often measured in dollars per QALY
•
QALYs incorporate an estimate of the quantity of life gained by the intervention, coupled with a more
subjective assessment of the quality of that life affected by the intervention
•
Historically, payers have considered any intervention that has a cost-effectiveness ratio of
<$100K per QALY as acceptable
Cost Effectiveness of Selected Treatments4,5
• Cost-saving (ratio <$0): Aspirin for at-risk
patients, childhood immunizations
• 0 to $13,999/QALY: Chlamydia screening,
colorectal screening for all adults >50
• $14,000 to $34,999/QALY: Cervical cancer
screening, hypertension screening for all adults
Summary
• Diagnostic tests should only be used if the result is likely to
significantly affect your certainty of a disease (posttest probability)
and should rely on likelihood ratios for a given test when available.
• The goals of screening are to detect treatable, asymptomatic, or
early stage disease.
• The limitations, harms, and costs associated with screening should
be considered in the context of the patient’s goals.
• Whenever possible, treatment benefit should be expressed in
terms of absolute risk reduction (not relative risk reduction).
Commitment in Your Practice
•
Think about your approach to clinical decision
making.
•
Do you routinely make recommendations that are
not in line with the high value principles outlined in
this module?
Write down at least one thing to start doing and one
thing to stop doing.
START:
STOP:
References
1.
2.
3.
4.
5.
6.
7.
Glaser AN. High Yield Biostatistics, 3rd ed. Philadelphia:, PA: Lippincott Williams and Wilkins; 2005.
Wilt TJ, Harris RP, Qaseem A; High Value Care Task Force of the American College of Physicians.
Screening for cancer: advice for high-value care from the American College of Physicians. Ann
Intern Med. 2015 May 19;162(10):718-25. [PMID: 25984847]
Owens DK, Qaseem A, Chou R, Shekelle P; Clinical Guidelines Committee of the American College of
Physicians. High-value, cost-conscious health care: concepts for clinicians to evaluate the benefits,
harms, and costs of medical interventions. Ann Intern Med. 2011 Feb 1;154(3):174-80. [PMID:
21282697]
Institute of Medicine. The Healthcare Imperative: Lowering Costs and Improving Outcomes:
Workshop Series Summary. Washington, DC: National Academics Press; 2010.
Moynihan R, Doust J, Henry D. Preventing overdiagnosis: how to stop harming the healthy. BMJ.
2012 May 28;344:e3502. [PMID: 22645185]
Welch HG, Schwartz L, Woloshin S. Overdiagnosed: Making People Sick in the Pursuit of Health.
Boston, MA: Beacon Press; 2011.
Moriates C, Arora V, Shah, N. Understanding Value-Based Healthcare. New York, NY: McGraw-Hill
Education; 2015.