Transcript Implications of Public and Private Reporting Quality Improvement

Implications of Public and Private Reporting
of Quality Data: Mechanisms for Driving
Quality Improvement
Deirdre E. Mylod, Ph.D.
[email protected]
Vice President, Public Policy
Press Ganey Associates, Inc.
June 27, 2006
Overview

Relationship between publicly benchmarked and private
benchmark reports for measures of quality
– Clinical, Safety, HCAHPS

Normative Rates of Change Year 1 to Year 2 for privately
reported patient evaluations

Relationship between hospital characteristics and
successful quality improvement using private reports

Relationship between hospital activities and quality
improvement related to private measures of patient
evaluations
– Public Reporting of Leapfrog Data
– Participation in IHI
– Access and Use of Data
Private vs. Public Reporting
Are the two related?
•
Private patient surveys are related to public clinical
measures at the hospital-level.
Gesell, S.B., Clark, P.A., Mylod, D.E., Wolosin, R.J., Drain, M., Lanser, P., &
Hall, M.F. (2005). Hospital-level correlation between clinical and service
quality performance for heart failure treatment. Journal for Healthcare
Quality, 27(6), 33-44.
Hospital-level correlation between heart failure patients’
overall satisfaction and percent of heart failure patients
given assessment of left ventricular function.
Overall Patient Satisfaction
90
85
80
75
r = .5
p < .01
70
70
75
80
85
90
95
Percent of Patients Given Assessment of Left
Ventricular Function
100
N=31
Hospital-level correlation between heart failure patients’
overall satisfaction and percent of heart failure patients
given discharge instructions.
Overall Patient Satisfaction
90
85
80
75
r = .6
p < .01
70
0
20
40
60
80
Percent of Patients Given Discharge
Instructions
100
N=26
Private vs. Public Reporting
Are the two related?
•
Private patient surveys are related to public safety
measures at the hospital-level.
Hospitals willing to publicly report to Leapfrog have
significantly higher patient satisfaction in private
patient reports.
84.0
82.7
Private vs. Public Reporting
Are the two related?
•
Private patient surveys are related to public patient
surveys at the patient- and hospital-level.
Mylod, D.E., & McCaffrey, K. (2006, March). Assessment of Convergent
Validity of HCAHPS® Using the Press Ganey Inpatient Survey. Poster
presented at the 10th National CAHPS User Group Meeting, Baltimore, MD.
Private vs. Public Reporting
Correlations Between HCAHPS Domains and Press Ganey Subscales
0.58 ** 0.56 **
0.30 ** 0.44 **
0.53 ** 0.36 **
Doctor Communication
0.29 ** 0.31 **
0.84 **
0.23
Hospital Environment
0.25 ** 0.69 **
0.55 ** 0.61 **
a
Responsiveness of Staff
0.24 ** 0.38 **
0.42 ** 0.37 **
Pain Management b
0.28 ** 0.36 **
0.33 ** 0.28
Communication re Meds
0.25 ** 0.39 **
0.29 *
0.19 *
Discharge Information
0.16 ** 0.21 **
* significant at .05 level ** significant at the .01 level
Nurse Communication
HCAHPS Domains
0.41
0.36
0.26
0.26
0.39
0.37
0.33
0.29
0.16
0.29
0.19
0.32
0.24
0.17
**
**
*
**
**
**
**
**
**
**
*
**
0.91
0.75
0.48
0.43
0.38
0.45
0.87
0.64
0.67
0.60
0.61
0.51
0.53
0.27
**
**
**
**
**
**
**
**
**
**
**
**
**
**
0.77
0.51
0.56
0.44
0.42
0.38
0.68
0.40
0.53
0.43
0.56
0.43
0.42
0.23
**
**
**
**
**
**
**
**
**
**
**
**
**
**
0.63
0.48
0.31
0.33
0.49
0.39
0.57
0.37
0.46
0.39
0.29
0.36
0.25
0.21
**
**
*
**
**
**
**
**
**
**
*
**
*
**
0.55
0.43
0.79
0.70
0.23
0.29
0.46
0.32
0.40
0.37
0.53
0.40
0.34
0.26
Personal Issues
Discharge
Physician
Visitors/Family
Tests/Treatments
Nurses
Meals
Room
Admission
Press Ganey Subscales
**
**
**
**
**
**
**
**
**
**
**
**
**
0.80
0.49
0.55
0.42
0.42
0.36
0.68
0.39
0.67
0.41
0.57
0.40
0.34
0.30
**
**
**
**
**
**
**
**
**
**
**
**
**
**
0.87
0.66
0.49
0.50
0.51
0.46
0.78
0.52
0.73
0.65
0.62
0.51
0.47
0.31
**
**
**
**
**
**
**
**
**
**
**
**
**
**
Facility-level correlations shown on the top of each row in blue. Largest row correlations are highlighted in bold.
Responsiveness of Staff was expected to relate to the Press Ganey Nurses subscale because the latter contains
measures of the full nursing experience including response time.
b Pain Management was expected to relate to the Press Ganey Personal Issues subscale because the latter includes
measures of pain control in addition to other aspects of supporting the unique needs of the individual patient.
a
Hospital-level correlation between private
patient surveys and public patient surveys
Overall Rating of Care
HCAHPS Average Rating (0-10)
10
9
8
r = .9
p < .01
7
75
80
85
90
95
Press Ganey Question Mean Score
100
N=67
Normative Rates of Change- Five Year Outlook
Proportion of Hospitals Getting Better/Worse
56
Decrease
Increase
54
Percent of Increase/Decrease
52
50
48
54.5
46
44
45.5
42
40
Proportion of Hospitals Significant Change
16
Significantly Wo rse
Significantly B etter
Percent of Significant Increase/Decrease
14
12
10
8
13.4
6
4
2
0
8.3
Proportion of Hospitals Significant Change
Significantly Wo rse
Similar
Significantly B etter
8.3
0%
78.3
10%
20%
30%
40%
50%
13.4
60%
70%
80%
90%
100%
How Much Change Do Hospitals See
Quartile with
biggest increase
Quartile with
greatest decline
Facility Characteristics Not Systematically
Related to Change

No consistent relationships between propensity to
change and:
– Bed size
– Teaching status
– Region of the country
Facility Characteristics Related to Change

Starting Performance
– Hospitals with lower patient satisfaction are more likely to
improve.

Patient Safety Journey
– Hospitals that are wiling to submit data to Leapfrog are more
likely to improve.
– Hospitals active in their quality improvement journey toward
safer care (IHI 100k).

Active Use of Patient Survey Data
– Hospitals that analyze access online data more frequently,
generate more reports and give more users permission to
use data improve more over a year.
With private patient surveys, the lower a hospital scored in
2004, the more likely it was to improve by 2005.
0.75
Mean Difference
0.50
0.25
0.00
-0.25
-0.50
1
2
3
2004 Quartile
Error bars: 95% CI
4
Willing to Submit Data to Leapfrog
Hospitals identified by Leapfrog as showing Good Early Stage
Effort achieve more positive change, and almost no negative
change, in private patient surveys (2004-2005).
1.0
Mean Difference
0.8
0.6
0.4
0.2
0.0
-0.2
Did Not Disclose
Willing to Report
Good Early Stage
Error bars: 95% CI
Good Progress
Fully Implemented
In the presence of not being the best in patient satisfaction,
willingness to report to Leapfrog is associated with more
positive change in patient satisfaction.
100k Hospitals More likely to have significant
improvement and less likely to have significant decline
16
Not 100K Hospital
100K Hospital
14
12
10
8
6
4
2
0
Significantly Worse
Significantly Better
Hospitals that access their private patient survey data via
an online analysis tool more, improve more over one year.
0.6
Mean Difference
0.4
0.2
0.0
-0.2
0-200
201-550
551-1150
Total Number of Logins
1151+
Hospitals that generate more custom reports of their
private patient survey data, improve more over one year.
0.4
Mean Difference
0.3
0.2
0.1
0.0
0-200
201-450
451-1000
Number of Custom Reports
1001+
Hospitals that give permission to more staff members
to access to the private patient survey data, improve
more over one year.
0.6
Mean Difference
0.4
0.2
0.0
-0.2
0-20
21-50
51-75
Number of Users
76+
Next Steps: Prospective Study of Characteristics of
Culture and Activities Linked To Change

Overarching Framework
– Culture-wide concern/commitment about performance
– Culture of transparency
– Ongoing, in-depth analysis of data (not just collection)
– Behaviorist assessment of uses of feedback
– Infrastructure for quality improvement
– Activities of quality improvement