Transcript Aligning Measurement-Based QI with Evidence-Based

Aligning
Measurement-Based QI with
Evidence-Based Practice Implementation
Richard Hermann, MD, MS
Associate Professor of Medicine and Psychiatry
Tufts University School of Medicine
Center for Quality Assessment & Improvement in Mental Health
at Tufts-New England Medical Center
www.cqaimh.org
Overview

How does evidence-based practice implementation (EBPI)
relate to measurement-based quality improvement (MBQI)?
– different paradigms
– similarities and differences

Potential for convergence & synergy

What obstacles need to be addressed?

Current research study on QI
2
Evidence-Based Practices
EBP
Rating
ACT / ICM
Evidence-based Psychotherapies
Family Psychoeducation
Supported Employment
Integrated Dual Diagnosis Treatment
Medication Management
Multi-Systemic Therapy
A = RCTs
B = less rigorous studies
C = consensus or opinion
A
A
A
A
A
A
A
3
EBP Implementation: a Top-Down Model
Research:
Controlled trial of clinical intervention
Development:
Codification of EBP by experts
↓
↓
Commercialization: Packaging: tools, scales, materials
↓
Diffusion:
Social marketing, training, support
Adoption:
Local provider organizations
Consequences:
Change to practice & outcomes
↓
↓
4
Rogers, Diffusion of Innovations, 2003
Measurement-Based QI

A “bottom-up” model
– Activities conducted by local provider organizations
– Influenced by external groups

MBQI is in wide use:
– 90-98% of hospitals report formal programs

MBQI is costly:
– estimated cost ~$200,000 per hospital per year
5
Principles of Measurement-Based QI

Quality as problems in “processes”

Measurement & analysis

Broad participation

Inductive reasoning

Trial and error
6
Model for Measurement-based QI
Aim
Intervene
Measure
Plan
Diagnose
Commonalities between MBQI and EBPI

Both address important problems—some overlap

Both employ measurement
– MBQI: rates of EBP use, appropriateness
– EBPI: fidelity to evidence-based model

Both start with an understanding of underlying processes
– MBQI: determined locally, informed externally
– EBPI: studied externally, expanded locally

Both involve systematic intervention to change practice
– MBQI: determined locally, informed by research & experience
– EBPI: developed by experts, customized to local circumstances
8
Potential for MBQI to Enhance
Evidence-Based Practice Implementation

Promotes local organizational development
–
–
–
–
system perspective
team work
analytic skills
experience implementing change
Increases awareness of gaps
 Prompts investigation
 Motivates exploration of available interventions
→ Potential for uptake of EBPs

9
Integrating MBQI with EBPI Requires
Alignment Across Healthcare System
Environment
(eg, payers, accreditors)
Local Organization
(eg, hospital)
Micro-system
(eg, hospital inpatient unit)
Conditions for Successful Alignment
1.
Local organizations need to select QI objectives that
address gaps between actual & evidence-based practice
2.
External organizations mandating measures also need to
emphasize measures of EBPs
3.
Microsystems within local organizations need to execute
these QI activities effectively
11
1. Do Quality Measures Used for Local MBQI
Address Evidence-Based Practices?

Reviewed measures developed for mental health QI

308 measures identified & evaluated:
– 9% supported by RCTs
– 30% supported by less rigorous evidence
– 61% not supported by evidence

Evidence-based measures less likely to be adopted

Pilot study of QI objectives adopted by MA hospitals:
< 10% of hospital objectives address EBPs
National Inventory of Mental Health Quality Measures (www.cqaimh.org)
12
2. Do Mandated Quality Measures Address
Evidence-based Processes of Care?

Measures established by:
– Accreditor requirements
– Government reporting requirements
– Benchmarking collaboratives

Results increasingly linked to:
– Pay for performance incentives
– Public disclosure
– Employer purchasing decisions
13
2. Do Mandated Quality Measures Address
Evidence-based Processes of Care?
Illustrative Measures
Restraint / seclusion rates
Elopement rate
Injury rate
Number of medications
Readmission rate
Medication errors
Antipsychotic dose
Antidepressant Adherence
Rating
C
C
C
C
C
B
A
A
14
A = RCTs
B = less rigorous studies
C = consensus or opinion
Evidence-Based Practices
EBP
Rating
ACT / ICM
Evidence-based Psychotherapies
Family Psychoeducation
Supported Employment
Integrated Dual Diagnosis Treatment
Medication Management
Multi-Systemic Therapy
A = RCTs
B = less rigorous studies
C = consensus or opinion
A
A
A
A
A
A
A
15
Attributes Informing Quality-Measure Selection
Meaningful
Maximize
Measure
Attributes
Feasible
Actionable

stakeholder needs

precisely specified

quality problem

clinically important

data available

under user’s control

evidence-based

affordable

interpretable

valid

accurate

results

comprehensible

reliable

norms

case mix adjustment

benchmarks

pt. confidentiality

standards
Domains of Process
(prevention, detection, access, assessment, treatment, continuity, coordination,
safety/errors)
Clinical Population
(diagnostic groups, comorbidities, prevalence, morbidity)
Vulnerable Groups
(children, elderly, racial/ethnic minorities)
Modalities
(medication, psychotherapy, other somatic, other psychosocial)
Clinical Setting
(inpatient, ambulatory, residential, partial, emergency service)
Purpose of Measurement
(internal QI, external QI, consumer selection, purchasing, research)
Level of Health Care System
(population, plan, delivery system, facility, provider, patient)
16
Represent Mental Health System Broadly
Evidence-Based Objectives for Inpatient QI: Schizophrenia

↑ use of antipsychotic drugs w/in recommended dose range

↓ use of multiple antipsychotics without adequate rationale

↑ % receiving adequate drug trials for refractory sx

↑ assessment/detection for EPS, akathisia or TD; ↑ rate of
evidence-based treatment

↑ enrolled/referred to ACT among inpatients at high risk for relapse

↑ family members provided/referred to psychoeducation

↑ fidelity of inpatient psychoeducation program.
17
Evidence-Based Objectives for Inpatient QI: Depression

↑ use of antidepressant drugs w/in recommended dosage range

↑ assessment/detection of psychosis among depressed
inpatients; ↑ use of adequate pharmacotherapy or ECT for
psychotic depression

↓ use of anticholinergic antidepressants among depressed
elderly inpatients

↑ % of inpatients w/ major depression referred to OP clinicians
providing evidence-based psychotherapy
18
Other Evidence-Based Objectives for Inpatient QI

↑ assessment & detection of medical conditions

↑ % receiving appropriate inpatient medical care, outpatient
referral & communication between IP & OP clinicians

↑ assessment/detection of SUD; ↑ % receiving inpatient
treatment & OP referral
19
3. Do Local Healthcare Organizations
Execute QI Activities Effectively?
Effectiveness in controlled trials
 Shortell (1998) reviewed 55 studies finding “pockets of
improvement” rather than evidence of widespread change
Effectiveness of routine QI
 Not well studied
 Case reports of successful initiatives
 Anecdotal evidence suggests much of local QI is ineffective
20
Macro Model of Organizational Predictors of QI
Environment
Hospital
QI Implementation
QI Outcomes
Culture
Organizational Factors
Structure
Stategic
Technical
Shortell, 1995
21
Predictors of QI Implementation
Cultural: beliefs, values & behaviors relative to QI
+ organizational culture emphasizing teamwork & innovation
+ commitment of senior managers & physicians
Structural: individual & group responsibilities
+ Decentralized decision-making
+ Longer experience
+ Greater number of teams & projects
Strategic: approach to QI
+ “prospector” approach
Technical: resources
+ presence of organization-wide information systems
22
Study of MBQI in Inpatient Psychiatric Units


NIMH-funded study of 32 hospitals in MA & CA
What are inpatient psychiatry units trying to improve?
– effectiveness
– access
– equity

-- patient-centered care
-- safety
-- efficiency
To what extent do these objectives address EBPs?
– Facilitators & barriers to adoption

To what extent do hospitals achieve measurable change?

Hypothesis
– Fit between organization & predict QI effectiveness
23
Micro Model of Organizational Predictors of QI
QI Progress
Environment
Diagnose
Measure
Plan
Intervene
Selected Aims
& Measures
QI Outcomes
Culture
Organizational Factors
Structure
Leadership
Resources
Hermann, 2005
Culture

Inpatient clinicians’ knowledge & beliefs about
evidence basis for QI objective

Inpatient clinicians’ beliefs about the value of the
QI objective to their patients’ care & outcomes
25
Structure
Course of QI objective as tracer of organizational
structure:
– serial reports of results disseminated to inpatient clinicians?
– are interventions attempted?
– reports of progress (or barriers) to appropriate committees?
– participation / coordination among necessary departments?
26
Leadership

Selecting objectives that are priority of hospital
leaders?

Responsive to external pressures?

Leaders actively involved or monitoring progress?
27
Resources
Availability of resources for achieving QI objective
– training
– tools
– time
– support (eg, data collection & analysis)
28
Conclusion

Greater progress toward implementing EBPs may be
achieved by aligning organizations’ QI activities with EBP
goals

Components of alignment:
– Provider organizations need to select evidence-based QI objectives
– External groups need to reinforce emphasis on EBPs
– Local MBQI needs to be more effective

Ongoing research aimed at:
– understanding barriers to adopting evidence-based QI objectives
– understanding organizational factors influencing QI progress
– developing interventions to improve effectiveness of local QI
29