Transcript reliability

A System for Great Asthma Care
Chapter Quality Network Asthma Project
Ohio AAP Chapter
Learning Session 1
Keith Mandel, M.D.
Vice President of Medical Affairs, Physician-Hospital Organization
Cincinnati Children’s Hospital Medical Center
AAP/CQN Improvement Advisor
October 9, 2009
I have no relevant financial relationships with
the manufacturers(s) of any commercial
products(s) and/or provider of commercial
services discussed in this CME activity.
Objective
• Discuss key drivers for improving system of care
for children with asthma.
Where Change Happens
Informed,
Empowered Patient
and Family
Productive
Interactions
Improved Outcomes
Prepared,
Proactive
Practice Team
What Does a “Productive Interaction”
Look Like for Chronic Illness Care?
• Systematic assessment at point of care
–
–
–
–
Clinical status
Evidence-based care
Confidence
Self-management skills
• Tailoring clinical management to family needs and
preferences
• Active, sustained follow-up
Why is this so hard to do?
It’s the system!
Every system is perfectly designed to achieve
exactly the results it gets
What will your system
look like in 1-2 years?
A short film by Jesse Dylan
Evidence that System Change Works
(Cochrane Review; JAMA 2002; Diabetes 2001)
• 40 studies (85% RCTs) (mostly in primary care)
• Four categories of interventions:
–
–
–
–
Decision support
Delivery system design
Changes to information systems
Self-management
• 19/20 that included self-management had a positive
effect
• The five studies that included all 4 categories had a
positive effect
Conclusions
• No “magic bullet” – no single intervention made
a major difference
• Self-management is necessary, but not sufficient
• More intervention categories addressed, greater
impact on patient outcomes
• Comprehensive system changes are needed to
improve outcomes
What is a System?
• “A network of interdependent components that
work together to accomplish a shared aim.”
(Deming)
• Overall aim of the CQN asthma collaborative
– To achieve measurable improvement in outcomes for
asthma populations by applying NHLBI guidelines and
making key practice changes
Improving Care for Populations:
Need to Work at Multiple “System” Levels
Broader Environment (ABP-MOC, payors/P4P, hospitals,
specialists, schools, community agencies, etc.)
AAP Chapters/National AAP office (improvement
collaborative/resources)
Primary Care Practice (practice leadership/engagement,
registry implementation)
Patient-Provider Interaction (AAP/CQN asthma form)
Recipe for Improvement
Evidence-Based Changes
(“What”)
Process Improvement Model
(“How”)
What are we trying to accomplish?
How will we know that a change is an improvement?
What changes can we make that will result in
improvement?
System Change Concepts
(“What” + Suggestions of “How”)
Version 2.0
CQN Asthma Project Practice Key Driver Diagram
Key Drivers
GLOBAL CQN AIM
We will build a sustainable quality
improvement infrastructure within our practice
to achieve measurable improvements in
asthma outcomes
Specific Aim
From fall 2009 to fall 2010, we will achieve
measurable improvements in asthma
outcomes by implementing the NHLBI
guidelines and making CQN’s key practice
changes
Measures/Goals
Outcome Measures:
 >90% of patients well controlled
Process Measures
 >90% of patients have “optimal” asthma care (all
of the following)
 assessment of asthma control using a
validated instrument
 stepwise approach to identify treatment
options and adjust therapy
 written asthma action plan
 patients >6 mos. Of age with flu shot
(or flu shot recommendation)
 >90% of practice’s asthma patients have at least
an annual assessment using a structured encounter
form
Engaging Your QI Team and
Your Practice
*The QI team and practice is active and
engaged in improving practice processes
and patient outcomes
Using a Registry to Manage
Your Asthma Population
*Identify each asthma patient at every visit
*Identify needed services for each patient
*Recall patients for follow-up
Using a Planned Care
Approach to Ensure Reliable
Asthma Care in the Office
* CQN Encounter Form
* Care team is aware of patient needs and
work together to ensure all needed
services are completed
Developing an Approach to
Employing Protocols
* Standardize Care Processes
* Practice wide asthma guidelines
implemented
Providing Self management
Support
* Realized patient and care team
relationship
Interventions





























Form a 3-5 person interdisciplinary QI Team
Formally communicate to entire practice the importance and goal of this
project
Meet regularly to work on improvement
All physicians and team members complete QI Basics on EQIPP
Collect and enter baseline data
Generate performance data monthly
Communicate with the state chapter and leaders within the organization
Turn in all necessary data and forms
Attend all necessary meetings and phone conferences
Select and install a registry tool
Determine staff workflow to support registry use
Populate registry with patient data
Routinely maintain registry data
Use registry to manage patient care & support population management
Select template tool from registry or create a flow sheet
Determine workflow to support use of encounter form at time of visit
Use encounter form with all asthma patients
Ensure registry updated each time encounter form used
Monitor use of encounter form
Select & customize evidence-based protocols for your office
Determine staff workflow to support protocol, including standing orders
Use protocols with all patients
Monitor use of protocols
Obtain patient education materials
Determine staff workflow to support SMS
Provide training to staff in SMS
Assess and set patient goals and degree of control collaboratively
Document & Monitor patient progress toward goals
Link with community resources
Ed Wagner, MD, MPH: MacColl Institute; Associates in Process Improvement;
Institute for Healthcare Improvement
Act
Plan
Study
Do
Network for Learning
(Framework for “How”)
Chronic Care Model (Wagner)
Version 2.0
CQN Asthma Project Practice Key Driver Diagram
Key Drivers
GLOBAL CQN AIM
We will build a sustainable quality
improvement infrastructure within our practice
to achieve measurable improvements in
asthma outcomes
Specific Aim
From fall 2009 to fall 2010, we will achieve
measurable improvements in asthma
outcomes by implementing the NHLBI
guidelines and making CQN’s key practice
changes
Measures/Goals
Outcome Measures:
 >90% of patients well controlled
Process Measures
 >90% of patients have “optimal” asthma care (all
of the following)
 assessment of asthma control using a
validated instrument
 stepwise approach to identify treatment
options and adjust therapy
 written asthma action plan
 patients >6 mos. Of age with flu shot
(or flu shot recommendation)
 >90% of practice’s asthma patients have at least
an annual assessment using a structured encounter
form
Engaging Your QI Team and
Your Practice
*The QI team and practice is active and
engaged in improving practice processes
and patient outcomes
Using a Registry to Manage
Your Asthma Population
*Identify each asthma patient at every visit
*Identify needed services for each patient
*Recall patients for follow-up
Using a Planned Care
Approach to Ensure Reliable
Asthma Care in the Office
* CQN Encounter Form
* Care team is aware of patient needs and
work together to ensure all needed
services are completed
Developing an Approach to
Employing Protocols
* Standardize Care Processes
* Practice wide asthma guidelines
implemented
Providing Self management
Support
* Realized patient and care team
relationship
Interventions





























Form a 3-5 person interdisciplinary QI Team
Formally communicate to entire practice the importance and goal of this
project
Meet regularly to work on improvement
All physicians and team members complete QI Basics on EQIPP
Collect and enter baseline data
Generate performance data monthly
Communicate with the state chapter and leaders within the organization
Turn in all necessary data and forms
Attend all necessary meetings and phone conferences
Select and install a registry tool
Determine staff workflow to support registry use
Populate registry with patient data
Routinely maintain registry data
Use registry to manage patient care & support population management
Select template tool from registry or create a flow sheet
Determine workflow to support use of encounter form at time of visit
Use encounter form with all asthma patients
Ensure registry updated each time encounter form used
Monitor use of encounter form
Select & customize evidence-based protocols for your office
Determine staff workflow to support protocol, including standing orders
Use protocols with all patients
Monitor use of protocols
Obtain patient education materials
Determine staff workflow to support SMS
Provide training to staff in SMS
Assess and set patient goals and degree of control collaboratively
Document & Monitor patient progress toward goals
Link with community resources
Key Drivers of Focus (in near-term)
• Using a planned care approach to assure reliable
asthma care at time of visit (reliability).
• Implementing a registry to improve outcomes
at patient and population level. (deferred to later
presentation)
• Engaging QI team and practice. (deferred to
breakout session)
Value of Highly Reliable Use of
AAP/CQN Encounter Form
• Changes nature of patient/family-provider interaction
through more active patient/parent engagement.
• Generates disconfirming data that surfaces
issues/challenges.
• Triggers improvement interventions at point of care.
• Brings evidence-based guideline tables “forward” to
point of care.
• Provides data for: driving improvement in populationbased measures, populating registry, and engaging
practice colleagues.
• Provides data to identify “high-risk” patients.
Definition of Reliability: General
• The measurable capacity of a process to
perform intended function in required time
under commonly occurring conditions.
• The extent of failure-free operation over time.
• Reliability involves industrial engineering, human
factors, and reliability science.
Definition of Reliability:
AAP/CQN Asthma Project
• Reducing the number of missed opportunities to capture
information on, and address, key aspects of asthma care
(using AAP/CQN encounter form) for practices’ total
asthma population.
Measuring “Reliability”/
Defining “System Failure”
• “Reliability” = # of opportunities where form
utilized ÷ total # of potential opportunities
• “Defect rate” = 1 minus “reliability”
# of missed opportunities ÷
total # of potential opportunities
• Defect rate often expressed as an order of
magnitude (e.g., 10-1, 10-2, 10-3).
Levels of Reliability
• 10-1 (Level 1) = missed opportunity occurs 1
time in 10 (90% capture rate)
• 10-2 (Level 2) = missed opportunity occurs 1
time in 100 (99% capture rate)
• 10-3 (Level 3) = missed opportunity occurs 1
time in 1000 (99.9% capture rate)
• Nearly all studies assessing reliability of applying
clinical evidence conclude it is at 10-1.
Importance of Reliability
Cumulative Percentage of Asthma Population
with Data Captured
100.0
y = mx + b
90.0
m = 30-5/12 = 2%/month
80.0
70.0
60.0
At this rate, it would take another 3 years to
capture data on 100% of population—need to
accelerate slope
50.0
40.0
30.0
20.0
10.0
0.0
2003Oct
2003Nov
2003Dec
2004Jan
2004Feb
2004Mar
2004Apr
2004May
2004Jun
2004Jul
2004Aug
2004Sept
2004Oct
2004Nov
2004Dec
Level 1 (10-1) Reliability:
Change Concepts and Examples
• Vigilance (“stay alert”) and hard work (“try harder”).
• Examples:
– Data feedback on compliance.
– Training/education/awareness.
– Personal reminders by “opinion leader”.
• Complicating factors:
– “Fatigue” (at physician, nurse, staff level).
– Competing demands for time/attention.
– “Environmental conditions” (e.g., less time available/less focus at
certain visit types).
Level 2 (10-2) Reliability:
Change Concepts and Examples
• Checklists/reminders built into system.
– Nurse/MA checks chart of asthma patients to assure data
collection form inserted and ultimately completed.
– “Reminders” built into EMR.
• Desired action (based on the evidence) is the “default.”
– “Standing orders” that all asthma patients receive written
management plan, controller medications (if “persistent”) and flu
shots—nurses screen patients at beginning or end of office visit.
• Scheduling.
– Data captured at time of regular follow-up phone call to parents
of asthma patients.
– Data captured via regular mailing to parents of asthma patients
to reassess status.
Level 2 (10-2) Reliability:
Change Concepts and Examples
• Redundancy (i.e., multiple opportunities to complete
form).
– If physician fails to complete form, nurse/MA works with family
to complete prior to their leaving office.
– If form not placed in chart prior to visit, staff adds form to chart
at time of visit.
– Parent completes form in waiting area or while in exam room.
– “Hold point” to review status of form prior to departure of
asthma patients from office (e.g., nurse reviews chart of asthma
patients prior to departure to see if form completed,
management plan provided/revised, controller medications
prescribed, flu shot administered).
Level 2 (10-2) Reliability:
Change Concepts and Examples
• Taking advantage of habits/patterns.
– Parent indicates if patient has asthma at time of check-in.
– Parent completes asthma form while updating demographic data in
waiting area.
– Nurse asks parent if patient has asthma when taking/confirming history.
• Standardization of processes/essential tasks.
– Process for getting forms completed is standardized across
nurses/physicians/office sites (e.g., process mapping of workflow).
– All patients screened for asthma, flu shot status, ED/urgent care visits,
admissions at time of visit.
• Differentiation (e.g., color coding of patient charts).
• “Real-time” identification of “failures” (missed opportunities for
using form at point of care).
Level 3 Reliability (10-3 and greater):
Change Concepts
• Preoccupation with failures.
– Circumstances underlying each missed opportunity discussed among
physicians and staff.
• Reluctance to simplify interpretations.
• Commitment to resilience.
– “Contingency plan” exists if patient not identified prior to visit or patient
newly diagnosed at time of visit.
• Deference to expertise.
– Recruit improvement/design ideas from multiple stakeholders, including
patient/family and office staff at all levels.
• High degree of cooperation, coordination, communication, and
collaboration among staff/team members.
“Prevent-Identify-Mitigate” Framework for
Designing Highly Reliable Systems
• Prevent: design system to prevent failure (at time of visit).
–
–
–
–
Identify population.
Flag charts.
Pre-populate charts with form.
Ask parents to self-identify at check-in that child has asthma.
• Identify: design processes to make failures visible so that they can
be addressed (at the time of the visit).
– Prior to checkout, nurse/MA checks chart to see if patient has asthma
and assures that form completed.
• Mitigate: design processes to “mitigate harm” caused by failures
when not detected/intercepted (at time of visit).
– Identify missed opportunities via billing system query and mail form to
parent.
What did you learn from testing
AAP/CQN form?
What challenges do you anticipate
around reliably implementing form
into workflow?
What reliability change concepts
might you test?
Overcoming Challenges
• Ask questions of Collaborative faculty
• Share challenges and learnings on Listserv
• Use tools and resources posted on Extranet
• Seek input from other practice teams
Transformation:
“Being The Best At
Getting Better”
(Lee Carter, former Board Chair,
Cincinnati Children’s)
Questions for Discussion
• What system challenges are you encountering
today that will be important to address in
achieving overall aim/goals? What are your
biggest concerns?
• What’s worked well from prior quality
improvement efforts that would be valuable to
build on through the AAP asthma initiative?
• What other ideas do you have for overcoming
these challenges?