Transcript AHRQ Slide Template 2004

Agency for Healthcare Research and Quality
Advancing Excellence in Health Care • www.ahrq.gov
US Preventive Services Task Force
Association of Community Health Nurse Educators
Lucy Marion, PhD, RN
Dean, MCG School of Nursing
June 6, 2008
Advancing
Excellence in
Health Care
History of the Task Forces
 1976 – Canadian Task Force on Periodic Health Exam
 1984 – USPSTF established by PHS
 1996 – Task Force on Community Preventive Services
(Community Guide) established by CDC
 1998 - 3rd USPSTF reconvened by AHRQ
 2001 – Present – Standing USPSTF
Advancing
Excellence in
Health Care
Who is the USPSTF?
■ Experts in primary care, prevention, research
methods
 Government supported, by AHRQ, but independent
 Family medicine, internal medicine, pediatrics,
obstetrics/gynecology, nursing, behavioral health
 Scientific support from Evidence-Based Practice
Centers (EPC)
 Non-member liaisons from primary care clinician
associations, Federal agencies
Advancing
Excellence in
Health Care
Current Members
of the USPSTF
Bruce N. (Ned) Calonge, M.D., M.P.H. (Chair)
Diana B. Petitti, M.D., M.P.H. (Vice Chair)
 Thomas G. DeWitt, MD
 Allen Dietrich, MD, MPH
 Kimberly D. Gregory, MD,
MPH
 Bernadette Melnyk, PhD, RN,
CPNP/NPP
 Lucy N. Marion, PhD, RN
 Virginia A. Moyer, MD, MPH
 David Grossman, MD, MPH
 Judith K. Ockene, PhD, MSEd
 George Isham, MD, M.P.H.
 George F. Sawaya, MD
 Michael LeFevre, MD, MSPH
 J. Sanford Schwartz, MD, AB
 Rosanne Leipzig, MD, PhD
 Timothy Wilt, MD, MPH
Advancing
Excellence in
Health Care
Nurse Members
of the USPSTF
 Carolyn Williams
 Nola Pender
 Janet Allan
 Carol Loveland-
Cherry
 Lucy Marion
 Bernadette Melnyk
Advancing
Excellence in
Health Care
Task Force Activities
 Systematically reviews the evidence of
effectiveness and develops recommendations for
clinical preventive services
– Age- and risk-factor specific
– For routine use in primary care practice
 Recommendations include:
– Screening tests
– Counseling
– Preventive medications
Advancing
Excellence in
Health Care
Why Evidence-Based?
 Need transparent, systematic process to obtain
and distill best available (or best feasible) evidence
to support decision making
– Identifying, evaluating and summarizing scientific
evidence about outcomes or interventions or
policies
– Translating evidence into practice
recommendations
Advancing
Excellence in
Health Care
Topic Selection
and Prioritization
Advancing
Excellence in
Health Care
Review of Criteria for Selecting
and Prioritization of Topics
1. The Task Force solicits new topics for consideration
from public, professional orgs, and TF members.
2. The USPSTF first considers whether newly
nominated topics are within scope of
primary/secondary prevention, primary care
relevant, and with substantial health burden.
3. The USPSTF prioritizes the topics according to
public health importance, potential for impact on
clinical practice, and addressing diverse
populations.
Advancing
Excellence in
Health Care
Process for Prioritization of Topics
 USPSTF prioritizes topics on a 3 point scale
(low, moderate, high) based on:
– Impact
– Burden
– Intensity of resources
 Helps in determining the order of reviews
 Helps in allocating limited resources
Advancing
Excellence in
Health Care
Methodology for Developing
Task Force Recommendations
Advancing
Excellence in
Health Care
Steps in the Process
of Developing Recommendations
1. Define questions and outcomes of interest
2. Define and retrieve relevant evidence
3. Evaluate QUALITY of individual studies
4. Synthesize and judge STRENGTH of available
evidence
5. Determine balance of benefits and harms
6. Link recommendation to judgment about net
benefits
Advancing
Excellence in
Health Care
Step 1: Analytic Framework on
Screening for a Disease
Advancing
Excellence in
Health Care
Step 1: Example Analytic Framework Prostate Cancer
1
Treat
Screen:
Asymptomatic
Men
PSA,
DRE
2
radiation,
prostatectomy
Early Prostate
Cancer
3
Reduced prostate
cancer morbidity,
mortality
5
4
Adverse effects
of screening:
Adverse effects of Rx:
false positive, false
negative,
inconvenience,
labeling
Impotence, incontinence,
death, overtreatment
Advancing
Excellence in
Health Care
Step 1: Example Analytic Framework –
Motor Vehicle Occupant Injuries
1
Behavioral
Clinical Populations counseling
interventions
 Infant/Child
 Adolescent
 Young Adults
 Adult
2
Correct use of
age- and weightappropriate
restraints (safety
seats, booster
seats, seat belts)
3
4
Adverse effects
Reduced driving
or riding when
driver is under the
influence of
alcohol
Decreased morbidity
(injuries, severity of
injuries, length of
hospitalizations, shortand long-term disability)
and/or mortality from
MVOI
Advancing
Excellence in
Health Care
Steps in the Process
for Development of Recommendations
1. Define questions and outcomes of interest
2. Define and retrieve relevant evidence
3. Evaluate QUALITY of individual studies
4. Synthesize and judge STRENGTH of available
evidence
5. Determine balance of benefits and harms
6. Link recommendation to judgment about net
benefits
Define and Retrieve
Relevant Evidence
Advancing
Excellence in
Health Care
 For each Key Question developed from AF:
– Create inclusion/exclusion criteria based on the key
questions defined from the analytic framework
– PubMed, Cochrane, and Other database search
(CINAHL, etc)
– References from key articles, editorials, review
articles
– Expert consultation (others, TF members)
Advancing
Excellence in
Health Care
Steps in the Process
for Development of Recommendations
1. Define questions and outcomes of interest
2. Define and retrieve relevant evidence
3. Evaluate quality of individual studies
4. Synthesize and judge STRENGTH of available
evidence
5. Determine balance of benefits and harms
6. Link recommendation to judgment about net
benefits
Advancing
Excellence in
Health Care
Assess Quality of Evidence
What do we mean by
quality of evidence?
“Extent to which a study’s design, conduct, and analysis
has minimized selection, measurement, and
confounding biases.”
– Lohr, J Qual Improvement, 1999
“Extent to which one can be confident that an estimate of
effect is correct”
– GRADE , BMJ 2004
Evaluate Quality
of Individual Studies
Advancing
Excellence in
Health Care
 Good:
– Uses a credible reference standard
– Reliability of test assessed
– Includes large number of subjects
Evaluate Quality
of Individual Studies
Advancing
Excellence in
Health Care
 Fair:
– Uses reasonable although not best standard
– Interprets reference standard independent of
screening test
– Moderate sample size
Advancing
Excellence in
Health Care
Evaluate Quality
of Individual Studies
 Poor: Has fatal flaw such as:
– Uses inappropriate reference standards
– Biased ascertainment of reference standard
– Very small sample size or very selected
patients.
Advancing
Excellence in
Health Care
Steps in the Process
for Development of Recommendations
1. Define questions and outcomes of interest
2. Define and retrieve relevant evidence
3. Evaluate quality of individual studies
4. Synthesize and judge STRENGTH of overall
evidence
5. Determine balance of benefits and harms
6. Link recommendation to judgment about net
benefits
Advancing
Excellence in
Health Care
Critical Appraisal Questions
 Do the studies have the appropriate research
design to answer the key questions?
 To what extent are the existing studies high
quality?
 To what extent are the results of the studies
generalizable (or “applicable”) to the general US
primary care population and situation?
Advancing
Excellence in
Health Care
Critical Appraisal Questions
 How many studies have been conducted that
address each key question? How large are the
samples in the studies?
 How consistent/coherent are the results of the
studies?
 Are there additional factors that assist us in
drawing conclusions about the certainty of the
evidence? (e.g., presence or absence of doseresponse effects; fit within a biologic model)
Advancing
Excellence in
Health Care
Step 4: Synthesize and
Judge Strength of Overall Evidence
 Evidence reports
– Evidence tables summarizing studies
– Narrative discussing overall strength of evidence
 Meta-analysis
 Modeling
 Systematic reviews from others –
Cochrane, etc.
Advancing
Excellence in
Health Care
Step 4: Synthesize & Judge
Strength of Key Question Evidence
Convincing: Well-designed, well-conducted
studies in representative populations that
directly assess effects on health outcomes
Advancing
Excellence in
Health Care
Step 4: Synthesize & Judge
Strength of Key Question Evidence
Adequate: Evidence sufficient to determine
effects on health outcomes, but limited by
number, quality, or consistency of studies,
generalizability to routine practice, or indirect
nature of the evidence
Advancing
Excellence in
Health Care
Step 4: Synthesize & Judge
Strength of Key Question Evidence
Inadequate : Insufficient evidence to
determine effect on health outcomes due to
limited number or power of studies, important
flaws in their design or conduct, gaps in the
chain of evidence, or lack of information on
important health outcomes
Advancing
Excellence in
Health Care
USPSTF Defines Certainty
 Likelihood that the assessment of the net
benefit of a preventive service is correct.
– The net benefit is defined as benefit minus harm
of the preventive service as implemented in a
primary care population.
– The USPSTF assigns a certainty level based on
the nature of the overall evidence available to
assess the net benefit of a preventive service.
Advancing
Excellence in
Health Care
Levels of Certainty – HIGH
 The available evidence usually includes
consistent results from well-designed, wellconducted studies in representative primary
care populations.
– These studies assess the effects of the
preventive service on health outcomes.
– This conclusion is therefore unlikely to be
strongly affected by the results of future studies.
Advancing
Excellence in
Health Care
Levels of Certainty – MODERATE
 The available evidence is sufficient to determine the
effects of the preventive service on health outcomes,
but confidence in the estimate is constrained by
factors such as:
– the number, size, or quality of individual studies
– inconsistency of findings across individual studies
– limited generalizability of findings to routine primary care
practice
– or lack of coherence in the chain of evidence.
 As more information becomes available, the
magnitude or direction of the observed effect could
change, and this change may be large enough to alter
the conclusion.
Advancing
Excellence in
Health Care
Levels of Certainty – LOW
 Low: The available evidence is insufficient to
assess effects on health outcomes.
– Evidence is insufficient because of: the limited number
or size of studies; important flaws in study design or
methods; inconsistency of findings across individual
studies gaps in the chain of evidence; findings not
generalizable to routine primary care practice; or a lack
of information on important health outcomes.
 More information may allow an estimation of
effects on health outcomes.
Advancing
Excellence in
Health Care
Steps in USPSTF Process
for Development of Recommendations
1. Define questions and outcomes of interest
2. Define and retrieve relevant evidence
3. Evaluate quality of individual studies
4. Synthesize and judge STRENGTH of overall evidence
5. Determine balance of benefits and harms
6. Link recommendation to net benefits
Advancing
Excellence in
Health Care
Step 5: Determine Balance of
Benefits and Harms: Assessing Harms
 Potential harms are real, but hard to quantify
 Include psychological and physical consequences
of false-positives, false-negatives, “labeling”, over
treatment
Advancing
Excellence in
Health Care
Step 5: Determine Balance of
Benefits and Harms: Assessing Harms
 Opportunity costs
 Magnitude and duration of harm subjective,
hard to compare to benefits
– May translate into QALYs to compare
– NNH
Advancing
Excellence in
Health Care
Step 5: Determine Balance of Benefits & Harms:
Assessing Magnitude of Net Benefit
 No explicit criteria for magnitude of net benefit
 Substantial benefit: impact on high burden or
major effect on uncommon outcome
 Problems: requires evidence on harms and
common metric for benefit and harms
Advancing
Excellence in
Health Care
Step 5: Determine Balance
of Benefits and Harms
Estimate Magnitude of Net Benefit
Benefits – Harms = Net Benefit
Advancing
Excellence in
Health Care
Steps in the Process
for Development of Recommendations
1. Define questions and outcomes of interest
2. Define and retrieve relevant evidence
3. Evaluate quality of individual studies
4. Synthesize and judge STRENGTH of overall evidence
5. Determine balance of benefits and harms
6. Link recommendation to net benefits
Step 6: Link recommendation to net benefits:
Advancing
Excellence in
Health Care
USPSTF Grades Recommendations
Certainty of
Net Benefit
Magnitude of Net Benefit
Substantial
Moderate
Small Zero/negative
High
A
B
C
D
Moderate
B
B
C
D
Low
Insufficient (I Statement)
Advancing
Excellence in
Health Care
Grade
A
B
C
D
I
Statement
RS: USPSTF Conclusion about
Evidence and Net Benefit
Statement
There is high certainty that the net benefit is substantial.
There is at least moderate certainty that the net benefit is at least moderate.
There is at least moderate certainty that the net benefit is small.
There is at least moderate certainty that the service has no net benefit or that
the harms outweigh the benefits.
Evidence is lacking, of poor quality or conflicting, and the balance of
benefits and harms cannot be determined.
Grade
A
Advancing
Excellence in
Health Care
B
C
D
I
Grade Definition
Suggestion for Practice
The USPSTF recommends the service. There is high
certainty that the net benefit is substantial.
Offer or provide this service.
The USPSTF recommends the service. There is high
certainty that the net benefit is moderate or there is
moderate certainty that the net benefit is moderate to
substantial.
Offer or provide this service.
The USPSTF recommends against routinely providing
the service. There may be considerations that support
providing the service in an individual patient. There is
moderate or high certainty that the net benefit is
small.
Offer or provide this service only if
there are other considerations in
support of the offering or providing
the service in an individual patient.
The USPSTF recommends against the service. There
is moderate or high certainty that the service has no
net benefit or that the harms outweigh the benefits.
Discourage the use of this service.
The USPSTF concludes that the current evidence is
insufficient to assess the balance of benefits and
harms of the service. Evidence is lacking, of poor
quality, or conflicting, and the balance of benefits and
harms cannot be determined.
Read “Clinical Considerations”
section of Recommendation
Statement. If offered the service,
patients should understand the
uncertainty about the balance of
benefits and harms.
Advancing
Excellence in
Health Care
Step 6: Link Recommendation to Net
Benefits: Insufficient Evidence
 Lack of evidence on harms or benefits
 Poor quality of existing studies
 Good quality studies with conflicting results
Advancing
Excellence in
Health Care
Communicating the Task
Force Recommendations:
The Recommendation Statement
Advancing
Excellence in
Health Care
Template for USPSTF
Recommendation Statement (RS)
 Preamble
 Summary of Recommendation & Evidence
 Structured Rationale
 Clinical Considerations
 Discussion
Advancing
Excellence in
Health Care
RS: Preamble
• The USPSTF makes recommendations about
preventive care services for patients without
recognized signs or symptoms of the target
condition.
• It bases its recommendations on a systematic
review of the evidence of the benefits and harms and
an assessment of the net benefit of the service.
•The USPSTF recognizes that clinical or policy
decisions involve more considerations than this body
of evidence alone. Clinicians and policy-makers
should understand the evidence but individualize
decision-making to the specific patient or situation.
Advancing
Excellence in
Health Care
RS: Summary of
Recommendation & Evidence
DO
The USPSTF recommends X service for Y population.
(A recommendation)
The USPSTF recommends X service for Y population.
(B recommendation)
Advancing
Excellence in
Health Care
RS: Summary of
Recommendation & Evidence
DON’T DO
The USPTF recommends against routinely
(providing) X service for Y population. There may
be considerations that support (providing) the
service in an individual patient.
(C recommendation)
The USPSTF recommends against X service for Y
population.
(D recommendation)
Advancing
Excellence in
Health Care
RS: Summary of
Recommendation & Evidence
WE DON’T KNOW
The USPSTF concludes that the current evidence
is insufficient to assess the balance of benefits
and harms of X service in Y population.
(I statement)
See Clinical Considerations for suggestions for
practice for I recommendations and a discussion of
known risk factors, etc.
Advancing
Excellence in
Health Care
RS: Structured Rationale
Importance:
Detection:
Benefits of detection and early intervention:
– Bullets for different populations
Harms of detection and early intervetion:
– Bullets for different populations
The USPSTF concludes that for :
– Statement about certainty
– Bullets for each population
Advancing
Excellence in
Health Care
RS: Clinical Considerations
1. Patient Population Under Consideration
2. Suggestions for Practice Regarding I statement
3. Assessment of Risk
4. Screening Tests
5. Treatment
6. Screening Interval
7. Other Approaches to Prevention
8. Useful Resources
Advancing
Excellence in
Health Care
RS: Discussion
 Burden of Disease
 Scope of Review
 Accuracy of Screening Tests
 Effectiveness of Early Detection and/or Treatment
 Potential Harms of Screening and/or Treatment
 Estimate of Magnitude of Net Benefit
 How Does Evidence Fit with Biological Understanding
Advancing
Excellence in
Health Care
RS: Other Sections
Recommendations of Others
References
Date
Advancing
Excellence in
Health Care
Dissemination & Implementation
of Recommendations
Advancing
Excellence in
Health Care
Decision Support Resources
for Different Audiences
 Clinicians
 Consumers
 Businesses, Employers, and Health Care
Purchasers
Advancing
Excellence in
Health Care
Electronic Resources for
Clinicians
 ePSS – electronic Preventive Services Selector
Tool
– Search USPSTF recommendations by age, sex
and risk factors
– Available as a web-based tool or can be
downloaded to PDA
– www.epss.ahrq.gov
Advancing
Excellence in
Health Care
Advancing
Excellence in
Health Care
Print Resources for
Clinicians
Annual Pocket Guide
Publication of
Recommendations and
Evidence
Advancing
Excellence in
Health Care
One Page Clinical Summary
Resources
to Inform Consumer Decisions
Advancing
Excellence in
Health Care
 Print materials: based on recommendations of the
USPSTF
 Men: Stay Healthy at Any Age – Your Checklist for
Health*
 Women: Stay Healthy at Any Age –Your Checklist
for Health*
 Adult Preventive Care Timeline – Wall chart
*Available in English and Spanish
Advancing
Excellence in
Health Care
Resources
to Inform Consumer Decisions
[email protected]
Resources for
Clinicians and
Consumers
Advancing
Excellence in
Health Care
 Based on
recommendations of the
USPSTF.
 At-a-glance wall chart for
appropriate preventive
services based on age,
sex, and risk status.
 To be used in prompting
shared decision-making
between consumers and
their primary care
clinician.
Electronic Resources
for All Audiences
Advancing
Excellence in
Health Care
www.preventiveservices.ahrq.gov
 Access to
– USPSTF Recommendations, Reports, and Methods
Papers
– Prevention Dissemination and Implementation
Information and Materials
– Links to our partners and their information, such as
the NBGH Purchaser’s Guide to Clinical Preventive
Services, NCI’s Cancer Control P.L.A.N.E.T., and
the National Commission on Prevention Priorities
Advancing
Excellence in
Health Care
Resource for
Employers and Policy Makers
 A Purchaser’s Guide to
Clinical Preventive
Services: Moving Science
into Coverage
– Promotes coverage and
promotion of preventive
services health benefits that
are based on evidence
– Published November 2006
– Partnership between NBGH,
AHRQ and CDC.
Advancing
Excellence in
Health Care
Potential Barriers
to Delivering Preventive Services
 System barriers
 Clinician
 Patient/consumer
Advancing
Excellence in
Health Care
Implications for Community Health
Nurse Educators
 Learn and teach rigorous methods of TFs
 Guide student applications: case studies,
practicum experiences, discussions
 Demonstrate how to integrate Community Guide
recommendations with USPSTF
– Put clinical guidelines in context of community and
family
– Consider “primary care referrable” for Community
Guide recommendations as well as referrals from
community health settings for primary care
Advancing
Excellence in
Health Care
Implications for Community Health
Nurse Educators: Example
 In collaboration with PHNs and CHNs in the
community, the CHNE with students conducts
– Needs assessment to identify priority health needs
– Planning to include USPSTF recommendations for
effective preventive interventions and CGTF
recommendations for effective strategies for
increasing the number of people screened
 Example:
– Clinical Guide recommends the breast, cervical, and
colorectal screening test
– Community Guide recommends client reminders, one
on one education, and provider reminders
Advancing
Excellence in
Health Care
US Preventive Services Task Force
 USPSTF Homepage -
http://www.ahrq.gov/clinic/uspstfix.htm
 Electronic Preventive Services Selector (ePSS)
http://epss.ahrq.gov/PDA/index.jsp
 USPSTF Fact Sheet
http://www.ahrq.gov/clinic/uspstfab.htm
 Guide to Clinical Preventive Services, 2007
http://www.ahrq.gov/clinic/pocketgd.htm
Advancing
Excellence in
Health Care





USPSTF Publications (Selected)
Ockene, J., Edgerton, E., Teutsch, S., Marion, L., Miller, T., Genevro, J.,
Loveland-Cherry, C., Fielding, J., Briss, P. (March 2007). Integrating
Evidence-Based Clinical and Community Strategies to Improve Health.
American Journal of Preventive Medicine, 32(3), 244-252.
Whitlock EP., Orleans CT., Pender N., Allan J. (May 2002). Evaluating primary
care behavioral counseling interventions: an evidence-based approach.
American Journal of Preventive Medicine, 22(4), 267-84.
Meyers D., Wolff T., Gregory K., Marion L., Moyer V., Nelson H., Petitti D.,
Sawaya GF. (May 2008). USPSTF. USPSTF recommendations for STI
screening. American Family Physician, 77(6), 819-24.
Sawaya GF., Guirguis-Blake J., LeFevre M., Harris R., Petitti D. (Dec 2007).
U.S. Preventive Services Task Force. Update on the methods of the U.S.
Preventive Services Task Force: estimating certainty and magnitude of net
benefit. Annals of Internal Medicine, 147(12), 871-5.
Lin K., Watkins B., Johnson T., Rodriguez JA., Barton MB. (Apr 2008).
Screening for chronic obstructive pulmonary disease using spirometry:
summary of the evidence for the U.S. Preventive Services Task Force. Annals
of Internal Medicine, 148(7), 535-43.
Agency for Healthcare Research and Quality
Advancing Excellence in Health Care • www.ahrq.gov
Thank you.
Questions?