Transcript Achieving Our Mission The Role of CQI in Public Safety

Achieving Our Mission
The Role of CQI in Public Safety
Kimberly Gentry Sperber, Ph.D.
Efforts To Date
• “What Works” Literature
– Principles of Effective Interventions
– Growing evidence based on individual
program evaluations and meta-analyses
• Continuing Gap Between Science and
Practice
– Few programs score as satisfactory on
CPAI/CPC
Importance of CPAI/CPC Criteria
• Differences in recidivism rates based on
CPAI scores:
– Scores of 0-49% demonstrated 1.7% reduction
compared to comparison group.
– Scores of 50-59% demonstrated 8.1%
reduction.
– Scores of 60-69% demonstrated 22%
reduction.
What are implications for ODRC standards
for correctional programs?
Why Isn’t “It” Working?
Latessa, Cullen, and Gendreau (2002)
• 4 common failures of correctional
programs:
– Failure to use research in designing programs
– Failure to follow appropriate assessment and
classification practices
– Failure to use effective treatment models
– Failure to evaluate what we do
What Happens When We
Don’t Measure Ourselves?
The Role of Fidelity
Washington State Example
(Barnoski, 2004)
• For each program (FFT and ART), an
equivalent comparison/control group was
created
• Felony recidivism rates were calculated for
each of three groups, for each of the
programs
• Youth who received services from therapists
deemed ‘competent’
• Youth who received services from therapists
deemed ‘not competent’
• Youth who did not receive any services (control
group)
Functional Family Therapy Results:
% New Felony
70
60
50
40
32
10
27
25
30
20
FFT Not Competent
Control group
19
13
9 6
FFT Competent
17
11
0
6 Months
12 Months
18 Months
Results calculated using multivariate models in order to control for potential differences between groups
Project Greenlight
• Short-term prison-based reentry
program in New York
–
–
–
–
–
–
–
–
–
CBT Skills Training
Employment Services
Housing Services
Drug Education and Awareness
Family Counseling
Practical Skills Training
Community-Based Networks
Familiarity With Parole
Individualized Release Plans
Project Greenlight Benefits
• Participants received more service
referrals
• Participants reported more contacts
with community services after release
• Participants demonstrated significantly
more familiarity with parole conditions
• Participants were more positive about
parole
But Did It Work?
34.1
35
30
26.8
24.2
25
20
15
17.2
14.4 13
Greenlight
No Services
10
Existing Trans
Services
5
0
Arrests at 6 Months
Arrests at 12
Months
What Went Wrong?
• Violation of the risk principle
– Ceased use of risk assessment instrument when staff
deemed process too cumbersome
• Violation of the need principle
– All offenders received same services whether needed
or not
• Violation of the fidelity principle
– Staff modified delivery of the CBT curriculum
(shortened the duration, increased frequency,
increased class size)
• Differential staff competence
– Certain case managers produced worse outcomes
UC Halfway House/CBCF Study in Ohio: A
Look at Fidelity Statewide (2002)
• Average Treatment Effect was 4%
reduction in recidivism
• Lowest was a 41% Increase in recidivism
• Highest was a 43% reduction in recidivism
•
Programs that had acceptable termination rates, had been in
operation for 3 years or more, had a cognitive behavioral
program, targeted criminogenic needs, used role playing in
almost every session, and varied treatment and length of
supervision by risk had a 39% reduction in recidivism
2010 UC Halfway House/CBCF Study in Ohio:
Adherence to CBT in Groups and Changes in Recidivism
7
6
5
4
3
2
1
0
-1
-2
-1
0
1
What Do We Know About
Fidelity?
• Fidelity is related to successful
outcomes (i.e., reductions in recidivism,
relapse, and MH instability).
• Poor fidelity can lead to null effects or
even iatrogenic effects.
• Fidelity cannot be assumed
• Fidelity can be measured and
monitored.
Bringing Mission to the
Forefront
Becoming a Mission-Driven
Organization
• Knowing the mission statement
• What does it mean to you?
• What does it mean for your role?
• Who is responsible for fulfilling the
agency’s mission statement?
• How can QA/QI help us accomplish
our mission statement?
Providing Managed Change
• Achieving and maintaining quality
is not accidental.
• It requires a deliberate effort on
the part of everyone.
Achieving Quality
• Responsibility for quality falls on
both the organization and the
individual.
• The individual and the organization
should be linked in a formal
framework designed to continually
improve quality.
Quality Assurance (QA)
• Retrospective review process
• Emphasis on regulatory and
contract compliance
• Catching people being bad leads to
hide and seek behavior
Continuous Quality
Improvement (CQI)
• CQI is a prospective process
• Holds quality as a central priority within
the organization
• Focus on customer needs; relies on
feedback from internal and external
customers
• Emphasizes systematic use of data
• Not blame-seeking
• Trust, respect, and communication
• Move toward staff responsibility for quality,
problem solving and ownership of
services
Focus on Causes not
Symptoms
• Focus on processes/systems rather
than individuals or specific errors
• Identification of risk points and
their contribution to the problem
• Identify changes in these
processes that reduce risk of reoccurrence
Objectives of CQI
•
•
•
•
To facilitate the Agency’s mission
To ensure appropriateness of services
To improve efficiency of services/processes
To improve effectiveness of directing
services to client needs
• To foster a culture of learning
• To ensure compliance with funding and
regulatory standards
Common Elements
• Documentation Review
• Performance Indicators
– Process vs. Outcomes
• Satisfaction – Client, Employee,
Referral Source/Stakeholders
• Protocols for Data Use
• Infrastructure
Focused Discussion:
Why Examine Documentation?
• Clinical/Service Implications
– Documentation is not separate from service delivery.
– Did the client receive the services he/she needed?
• Operational Implications
– Good documentation should drive decision-making.
– Means of communication
• Risk Management Implications
– If it isn’t documented, it didn’t happen.
– Permanent record of what occurred in the
program/unit.
• Source of Staff Training
• Reflection of the provider and organization’s
competency:
– EBP
– Outcome of care
Action Plans
• Plan of correction
• Proactive approach to problemsolving
• Empowers staff
• Using objective data to inform
decision making
Who Creates Action Plans?
• Anyone and everyone can create
action plans
• Focus should be on who has
knowledge or expertise to
contribute
• Focus should not be on the
person’s title
Minimum Requirements
• Buy-in from staff at
organization
• Sufficient resources
training
• Sufficient resources
to participate in the
all levels of the
allocated for staff
allocated for staff
process
– Meetings
– Data collection, reporting, use
• Sufficient information systems
Why Invest the Resources?
What makes this effort too compelling to ignore?
A CEO’s Perspective
• Why invest in CQI?
– Because it’s the right thing to do!
– Better for clients (i.e., better outcomes)
–
–
–
–
–
• Mission-driven
Increased staff satisfaction
Increased staff retention
Improved referral source satisfaction
More business for related projects
Outcomes to sell to business community
and other payers
– Demonstrates fiscal responsibility (i.e.,
effective use of dollars)
Strategic Use of CQI Data
• CQI data used to provide testimony
before legislature
• CQI data and infrastructure used to
secure new contracts and grants
• CQI data used in newsletters, media
relations, levy campaigns, etc.
• CQI data used to negotiate
programmatic changes with
stakeholders
Performance Examples
• Men’s halfway houses employment with
benefits
– 33% to 67% over 5 years
• Changes in HIT over time per strategic
plan
– 57.4% to 72% over 5 years
• CCC’s employment indicator
performance
– <10% to 75% over 5 years
How do these changes in performance
link to mission?
Top 10 Practices for Reducing
Recidivism in Drug Courts
(Carey & Finigan, 2013)
1. Program caseload less than 25
2. Participants have greater than 90 days
clean before graduation
3. Judge spends average of 3 minutes or
more per person during status review
hearings
4. Treatment communicates with court via
email
5. Representative from treatment attends
drug court team meetings
Top 10 Practices for Reducing
Recidivism in Drug Courts
(Carey & Finigan, 2013)
6. Review of the data/program stats has led
to modification in operations
7. Representative from treatment attends
court sessions
8. Drug court allows non-drug charges
9. Law enforcement is member of drug court
team
10.Results of program evaluations have led
to modification in operations
Top 10 Practices for Increasing Cost
Savings in Drug Courts
(Carey & Finigan, 2013)
1. Review of the data/stats has led to
modifications in operations
2. Results of program evaluations have led
to modification in operations
3. Sanctions are immediate
4. Defense attorney attends drug court
team meetings
5. Must have a job or be in school to
graduate
Top 10 Practices for Increasing Cost
Savings in Drug Courts
(Carey & Finigan, 2013)
6. Representative from treatment attends
court sessions
7. Team members are given copy of
sanctioning guidelines
8. Drug test results back in 48 hours or less
9. Drug tests collected at least 2x/week
during first phase
10.Law enforcement attends court sessions
NPC Research on Drug Courts
Percent Improvement in Outcome Costs*
Drug Court Uses Evaluation Feedback to Make Modifications
40%
37%
30%
20%
16%
10%
0%
Yes
N=20
No
N=15
* "Percent improvement in outcome costs" refers to the percent savings for
drug court compared to business-as-usual
The Role of QA/QI in Community Corrections
(based on UC Halfway House and CBCF study)
% Change in Recidivism
8
6
6
4
2
1
0
Internal QA
No Internal QA
Bottom Line
• Many programs are implementing evidencebased practices with little evidence of strong
fidelity.
• Result is an ongoing gap between science and
practice.
• This gap often results in null or even iatrogenic
effects.
• Correctional organizations have a responsibility
to ensure effective services.
• Responsibility for EBP needs to be aligned at all
levels – administration, management, line staff.
• Need to focus on creating formal infrastructure
to support EBP and mission achievement
Continuous Quality
Improvement
Questions & Answers