Transcript PowerPoint Presentation - Managing Service Operations Wrap

Choices and Outcomes:
The Effects of Improvement Project
Portfolio Choices on Clinical Outcomes
Anita L. Tucker
Assistant Professor, U. of Pennsylvania
Senior Fellow, Leonard Davis Institute of Health Economics
Ingrid M. Nembhard
Doctoral Candidate, Harvard Business School
Harvard Graduate School of Arts and Sciences
KLIC 4: INFORMS Conference
November 6, 2006
Financial Support from HBS DOR, Wharton’s Fishman Davidson Center
In Collaboration with Jeffrey Horbar, Richard Bohmer and Amy Edmondson
1
Background: Health Care in America
The Data:
1996: Dartmouth Atlas Project
shows inappropriately
geographic variations in care
1998: National Roundtable
reports “serious and
widespread problems” exist
in American medicine” . .
Problems of underuse, overuse,
misuse . . “Quality of care is
the problem.”
=
Publicity:
1999: Institute of Medicine
(IOM) reports 100,000
Americans die annually
from preventable
medical errors
2000: IOM finds quality
problems are a systemic
property, requiring
process improvement
+
2003: RAND Study reports
only 55% of patients
receive the
recommended care for
their condition
Need for improvement projects in
health care organizations
2
Improvement Projects
Improvement Projects “solve complex organizational problems
through the work of formal teams that use a structured
improvement method” (Christianson et al., 2005: p. 610)
 Healthcare Project Examples
Target Area
• Increase handwashing
(Reduce Infections)
• Reduce heelsticks in Neonates
(Pain management)
• Increase collaboration among MDs and RNs (Staff Retention)
Portfolio of Improvement Projects: An organization’s set of
improvement projects that are in progress at the same time
and draw on the same limited set of human, managerial
and financial resources
(Cooper, Edgett & Kleinschmidt, 1999; Wheelwright & Clark, 1992)
3
Research Question
How should health care organizations structure
their portfolios of improvement projects to
achieve better outcomes?
Hypotheses about Portfolio Choices
1) Number of projects (+)
2) Concentration within a target area (+)
3) Level of evidence for portfolio (+)
4) Novices should start with clinicallyoriented portfolio (+)
5) Extent of physician involvement (+)
4
Research Setting: Vermont Oxford Network

Collaborative of Neonatal Intensive Care Units (NICU)

44 NICUs working together for 2 ½ years (Apr 2002 - Oct
2004)
 Identified 7 target areas for improvement and 93 improvement
projects across those areas
 Met twice a year to learn QI methods (PDSA cycles), work on
developing best practice guidelines and share experiences
 In between meetings, implemented practices, conducted site
visits to other NICUs and phone conferences
Horbar, J. D. et al, 2001. Collaborative quality improvement for neonatal intensive
care. Pediatrics 107 (1) 14-22.
5
Portfolio Project Options
Target Area
Aim
# of
Sample practice project(s)
Operationally-oriented
Clinically-oriented
practices
Pain & Sedation
Decrease mean pain score by 50% during
10
NICU stay
Infection Control
Decrease hospital-acquired infections by
Intubation of medications
7
25-50% over 2 years
Respiratory Care
Decrease chronic lung disease by 10%,
Pain management during heelsticks
Promote hand washing education and
practices to prevent nosocomial infection
15
and decrease oxygen days, ventilator days,
Vitamin A supplementation
Ventilation by Tidal Volume Monitoring
steroid use
OB/NICU
Improve maternal & newborn caregiver
Relations
collaboration: periviability, delivery
6
Design process to increase collaboration and
communication during high-risk delivery
response, comfort care
Staff Retention
Decrease staff turnover by 50%
Family-Centered
Enhance ability to co-ordinate and deliver
Care
care so the infant and family needs are met
5
Improve nurse-physician collaboration
27
Provide resource materials that depict
newborn premature infants’ maturational and
postnatal environment.
Discharge
Embed discharge planning into all aspect
Planning
of patient care & communication
23
Develop "trigger point" checklist for discharge
teaching
93
Potential Projects in the Portfolio
**Each NICU indicated which projects they included in their project portfolio to the
collaborative sponsor.
6
Sum of + Have in place prior to NIC/Q 2002
Sum of o Implemented during NIC/Q 2002
Sum of X Working on
Sum of # plan to work on
Sum of not working on and don't plan to
o
o
x
x
1
4
2
2
1
+
6
0
0
0
4
Hospital 105
+
+
+
+
+
+
Hospital 104
Working on
+
x
+
Hospital 103
Implemented during
collaborative
#
#
o
+
o
Hospital 102
PBP
1 Intubation Medications
10 Pharmacological Factors affecting Opioid Tolerance
2 Reduce frequency of tracheal suctioning
3 Reduce frequency of heelsticks
4 Standardized Recommendations for Sucrose Analgesia
5 Pain Management During Heelsticks
6 Peripheral Vascular Procedures
7 Circumcision
8 Postoperative Pain Management
9 Guidelines for weaning from opioids
Hospital 101
Excerpt from the practices from Pain Management
Hospital 100
Unique Portfolios of Practices
#
+
o
o
o
o
o
o
o
+
+
+
#
+
+
x
x
+
#
#
o
#
x
x
+
x
#
#
3
0
1
0
6
1
7
0
1
1
6
0
2
1
1
1
1
3
5
0
Hospital 100 Implemented/Working on: Reducing frequency of tracheal
suctioning, standarized sucrose analgesia, peripheral vascular procedures,
circumcision, post op pain, weaning from opiods
VERSUS
Hospital 102: Reducing frequency of heelsticks
7
Evidence-base for the portfolio
 Level of evidence for all projects within each
target area assessed by target area team using
Muir-Gray (MG) score to rate articles
1 = strong evidence from at least one systematic review of multiple,
2=
3=
4=
5=
well-designed, randomized, controlled trials
properly designed randomized control trial of appropriate size
well-designed trials without randomization
well-designed non-experimental trials
opinions of respected authorities, based on clinical evidence,
descriptive studies or reports of expert committees
 Evidence base for NICU portfolio = the average
MG score of the portfolio
8
Outcome: Standardized Mortality Ratio
SMR level of analysis: Babies nested in NICU
1. LOGIT model (clustered by NICU) Outcome Death (0,1)
• Independent Variables: Established risk factors
(Zupanic et al. 2006)
2. Predict probability of death for each baby
3. By NiCU, sum up probability of death, actual deaths
4. Compute ratio:
Actual deaths

SMR Mortality 
 Expected deaths
• SMR < 1 Outcomes BETTER than expected
• SMR = 1 Outcomes equal to expected
• SMR > 1 Outcomes WORSE than expected
9
Means, SD, and correlations (N=27)
Variable
1 Staff/bed
Mean
SD
1
2
3
0.92
0.51
.02
0.78
13.04
0.75
5.91
.14
.12
.07
-.20
-.28
0.67
0.48
.03
.32
-.21
-.21
8.56
8.87
-.16
-.19
-.04
-.05
.05
3.44
3.75
5.29
0.43
-.08
-.04
-.19
.01
.01
.01
.03
.24
-.08 .92**
.10 .21 .39*
0.20
0.09
-.12
.19 .47** -.34^
.07
-0.02
0.19
-.17
.25
.19
-.07 -.48** -.42*
0.15
0.82
-.07
-.12
.31
3.34
2 Cardiac Surgery 0.44
3
4
5
6
7
8
9
10
11
# Prior
Collaboratives
QI Team Size
Teaching
Hospital
Total comp.
projects
Total comp.
LOS
Ave Evidence
% of MDs on
team
Improve. SMR
LOS 04-01
Improve. SMR
Mort. 04-01
4
.13
5
-.39* -.20
6
.22
.20
7
.21
.23
8
9
10
-.12
.05
-.06
-.11
.03
-.39
^=p<.1; *=p<.05; **=p<.01
10
OLS Regression results (H1, H3, H4)
Outcome measure
Improvement SMR
mortality (2004-2001)
Control Variables
Staff to beds ratio
-0.336* (0.145)
Cardiac surgery provider
-0.232 (0.240)
History of quality improvement
0.569* (0.208)
Team size
-0.020 (0.023)
Teaching Status
0.189 (0.279)
Independent Variables
Number of projects
-0.105* (0.044)
Number of projects squared
0.004** (0.001)
Evidence supporting portfolio
-0.526^ (0.293)
% of MDs on QI team
-3.429* (1.507)
Constant
4.139** (1.291)
Adj. R-squared
0.37
F
2.62
Sig
0.04
df
9, 16
H1: Supported u-shape
H3: Less evidence->Imp
H4: Supported
N = 26, (std error)
^ significant at 10%;
* significant at 5%;
** significant at 1%
11
R H2: Concentrating the within a target area helps
Outcome measure
Improvement SMR length of stay
(2004-2001)
Control Variables
Staff to beds ratio
-0.049 (0.036)
Cardiac surgery provider
0.118 (0.071)
History of quality improvement
0.082 (0.048)
Team size
0.008 (0.006)
Teaching Status
-0.047 (0.077)
Independent Variables
Number of LOS projects
-0.013^ (0.006)
Constant
-0.007 (0.161)
H2: Supported
Adj. R-squared
0.23
F
2.22
N = 26, (std error)
Sig
0.09
df
6, 19
^ significant at 10%;
* significant at 5%;
** significant at 1%
12
R H5: Initial portfolio orientation matters
Outcome measure
Improvement SMR mortality
(2004-2001)
Control Variables
Staff to beds ratio
-0.251 (0.144)
Cardiac surgery provider
History of quality improvement
Team size
Teaching status
Independent Variables
Number of projects
-0.213* (0.082)
Number of projects squared
0.005* (0.002)
Evidence supporting portfolio
% of MDs on QI team
Clinically oriented portfolio
-1.751* (0.697)
Constant
2.951* (1.008)
Adj. R-squared
F (df)
Sig
0.16
7.50 (4,6)
0.02
H5: Supported
N= 11
(Robust std error)
^ significant at 10%;
* significant at 5%;
** significant at 1%
13
Summary and Implications
An effective improvement project portfolio:
•
Includes neither too few or too many projects to manage
the tradeoff between synergy and distraction
•
Concentrates its efforts within a target area to maximize
inter-project learning
•
Focuses on operationally-oriented projects which build
performance-improvement capability
•
For novices is clinically-oriented where clearer benchmarks
are available
•
Is led by a team with significant physician membership.
14