Transcript Title

Annual Site Reports
2006-2007 FY
A Training and Overview
Los Angeles County Evaluation System:
An Outcomes Reporting Program
2008
Purpose of this Presentation
• Overview of the Los Angeles County
Evaluation System (LACES).
• Review the content and purpose of the
Site Reports.
• Discuss those content areas that are the
focus of the evaluation.
• Describe future evaluation activities
– Discharge Outcome Report (DOR)
Overview of LACES
• LACES is an eight year collaborative partnership
between ADPA, contracted providers,
UCLA/ISAP and other county and state
representatives.
• The purpose of LACES is to design, implement,
and conduct an evaluation of the adult alcohol
and other drug treatment/recovery system in Los
Angeles County.
• On-going evaluation, not a temporary study.
Accomplishments of LACES
To date LACES has:
1. Fundamentals of the ongoing evaluation
system.
2. Designed and implemented evidence-based,
standardized evaluation procedures.
3. Determined that alcohol and other drug
treatment/recovery is cost effective.
4. Review & improve existing treatment forms
and instrument.
5. Designed reports to benefit treatment.
First… Some Questions…
• Have you heard of a site report?
• Have you seen a site report?
• Have you had the opportunity to
review the contents and findings of
the site report with the director or
program staff?
A Site Report Is…
• Brief: six pages, double-sided.
• Info from admission & discharge
LACPRS.
• Demographics, outcomes &
performance measures.
• Information is included on your site and
for all sites of the same program type.
• Annual and some quarterly reports
What is the Purpose of the
Site Report?
• How participants are benefiting from
treatment.
• Provide information on program
functioning and short-term treatment
outcomes.
• Identify provider performance strengths
and areas needing improvement.
What the Site Report Does Not Do?
• Provides “site specific” information – a
single program type (e.g., outpatient) for
a single address.
– The site report does not consider funding
source.
• Provides feedback concerning
performance in the focus areas of the
evaluation.
Contents of the Site Report
• Executive Summary
– Overall participant admission and discharge numbers.
– Information on key outcomes and performance.
• Full Report
– Demographic information.
– More details on treatment outcomes and performance.
– Charts illustrating admission to discharge changes.
Executive Summary
• Single page, stand alone document.
• Admission to discharge changes.
• Key Areas include
– “Participant Flow” chart.
– Treatment Outcomes
– Program Functioning
Executive Summary Tables
• Table A: Number of admissions and discharges
for the time period covered by the report.
• Table B: Information on those discharged.
– Determine areas of improvement
• Table C: Admission to discharge changes
outcome areas.
– Includes participants who were in treatment 30+ days
and completed an exit interview.
Program Functioning and
Treatment Outcomes
Areas included in the Site Reports include:
• Program Functioning
– Number of admissions and discharges.
– Percent of exit interviews.
– Participants with “positive compliance.”
• Treatment Outcomes
– Retention in treatment (average days).
– Changes in alcohol and other drug use.
– Changes in other areas including health problems,
employment, and social support.
Program Functioning
(Performance)
• Admission and discharge numbers
– Review to ensure ADPA received all submitted
records.
• Positive Compliance
– Discharge status = completed treatment or left
with satisfactory progress.
– Important to determine the effectiveness of the
program.
Performance cont’d.
• Exit Interviews
– Responses to all discharge LACPRS
questions constitutes an “Exit Interview.”
– Information from both admission and
discharge is required to measure treatment
effectiveness.
– No exit interviews = No measurement of
the participant changes that occurred
during AOD treatment.
Table B: Participant Flow by Fiscal Quarter and
Year to Date (YTD)
Type of Transaction
First
Second
Third
Fourth
YTD
Total
Participants Discharged
50
25
25
30
20
125
125
Participants with Positive Compliance
25
30
20
15
90
50%
60%
67%
75%
72%
Participants in Treatment at least 30
days
18
22
10
12
62
Percent in Treatment at least 30 days
36%
44%
50%
80%
50%
50%
Participants in Treatment at least 90
days
12
15
12
11
50
Percent in Treatment at least 90 days
24%
30%
40%
55%
40%
40%
20
25
15
15
45
40%
50%
50%
75%
36%
Percent with Positive Compliance
Participants with Exit Interviews
Percent with Exit Interviews
Total by Fiscal Quarter
Treatment Outcomes
• Measures are similar to state and federal
government.
• Retention in treatment for 30/90 days.
– In LA County, some participants leave
treatment during the first two weeks.
– Research shows that without an adequate
amount of time in treatment, few improvements
are found.
Treatment Outcomes, cont’d.
• Performance Measures cover changes in:
– Substance use (alcohol and other drugs).
– Criminal justice involvement (days in
jail/prison).
– Homeless at admission versus discharge.
– Positive employment activities (full- & part-time
work, job training, and enrollment in school).
– Social Support Activities (AA, NA, etc.).
Table C: Quarterly Performance Measures
Performance Measure
DaysDays
of Drug/
Average
of Primary
Alcohol Use*
Use
Drug/Alcohol
Average Days of Criminal
Justice Involvement
Average Change from Admission to Discharge
Residential Services
ThisFacility
Facility Site
All Residential Service Sites
This
Site
All Sites
Admission Discharge Change
Change
Admission Discharge
Discharge
Change
Admission
12.3
12.3
2.4
Average Days in Treatment
Percent Reported
Homeless
Reported
Homelessness
Percent Reported Positive
Employment Activities
Percent Reported Social
Support Activities
0
0
-12.3
-12.3
15.9
0.3
-15.5
0
-2.4
4.9
0.2
-4.7
201
49.6%
49.6%
117.4
4.6% -45%
-45%
4.6%
47.2%
23.9%
-23.3%
3.5%
8.0%
4.5%
6.9%
22.4%
15.4%
57.0%
93.0%
36.0%
42.6%
92.9%
50.2%
Full Report
• Provides an overview of LACPRS
admission and discharge information.
• Section includes:
– General demographics.
– Primary and secondary alcohol and other drug
use information.
– Program performance.
Demographics
• Table provides demographic information on
participants discharged .
• It includes demographic information on other
sites for the benefit of the reader.
• Provides information on populations of
special interest such as General Relief and
homeless.
Primary & Secondary Substances
• Percentage of treatment participants by
primary or secondary drug problem.
• Primary or secondary drug information from
other sites is also included.
• These tables may be used to track changes
in drug use patterns for the site.
Program Outcomes Section
Refers to the four major sections.
1.
2.
3.
4.
Discharge status.
Changes in alcohol and other drug use.
Changes in employment and social support.
Changes in health problems and other areas
of social functioning.
Discharge Status
• Positive compliance examined separately
from negative compliance
– Left with unsatisfactory progress
• Average length of stay demonstrates
differences in stay by discharge status
– Useful to providers attempting to increase
overall retention.
Alcohol and Other Drug Use
• Bar chart shows change in average of
days of primary and secondary drug use.
• Complete abstinence is not expected.
• Changes may also be found for injected
drug use – especially for those sites that
have a high percentage of needle users.
Chart 1: Admission to Discharge Changes
in Drug and Needle Use
Avg Days in Last 30
30
18
20
7
10
4
1.2
0
0
0
Days Injected Drug
Use
Primary Drug
Admission
Discharge
Secondary Drug
Health Problems &
Social Functioning
• Changes in jail days, family conflict, &
physical health problems.
• In some instances the number of jail days
will be very low.
• May show an increase in physical health
problems due to increased access to
health care.
Chart 2: Admission to Discharge Changes
in Health Problems and Social Functioning
Avg Days in Last 30
30
20
7
10
2
2
0
4
1
0
Days in Jail
Family Conflict
Admission
Discharge
Physical Health
Problems
Employment and
Social Support
• Changes in days spent employed and
engaged in social support activities.
• Not all sites focus on employment or
provide social support activities.
• Useful to examine the utilization of these
activities by the participant.
Chart 3: Admission to Discharge Changes in
Employment & Social Support
Avg Days in Last 30
30
22
20
10
7.2
3.7
0
0
Paid for Working
Admission
Social Support Activities
Discharge
Important Performance and
Treatment Outcome Measures
•
•
•
•
Completion of treatment/Positive Compliance.
Abstinence from/Reduction in drug/alcohol use.
Exit Interviews
Length of Stay/Enrollment in Treatment
(2 benchmarks)
– 30 Days = Engagement.
– 90 Days = Retention.
• All important aspects of the treatment plan.
Why Completion of Treatment?
• Most outside of this field are interested
in completion figures.
• Positive Compliance
– Completed treatment.
– Left treatment prior to completion with
satisfactory progress.
• Others have adopted this mindset, but
many have not.
Why Reduction/Abstinence?
• Silly picture, for a silly question.
• That is our business.
• If we cannot demonstrate abstinence or
reductions in use, funders will move the money
to those who can.
Interviews
• Responses to all discharge LACPRS
questions.
• One of the most important of the evaluation
foci.
• Information from both admission and
discharge is required.
• No exit interviews = No measurement of
changes.
Why Engagement & Retention?
• First 30 days are critical to the outcome of
treatment.
• Without adequate treatment, few
improvements are found.
• Funding organizations are beginning to take
note of engagement and retention rates.
Note on Documentation
• Accuracy is extremely important.
– Length of stay/enrollment very important.
– Example – length of stay/enrollment.
• In 2004-2005, average length of stay was 105 days.
• But only 40% were in treatment for 30+ days.
• If this information is inaccurate, others may
become suspicious of all information reported.
Max LOS/LOE by Funding Source
• All funding sources support 30- and 90-day
length of stay/enrollment benchmarks.
FUNDING
Outpatient LOE Residential LOS Day Care LOE
CalWORKS
365 Days
365 Days
365 Days
Drug Court
730 Days
90 Days
N/A
Female Offender N/A
180 Days
N/A
General Relief
180 Days
180 Days
180 Days
PPN
180 Days
180 Days
N/A
Prop 36
300 Days
90 Days
204 Days
Drug Medi-Cal
No max length - approval is need for 180+ days.
Average Provider Performance
• During the last 3 fiscal years, average
performance in these areas was…
PERFORMANCE AREA
Outpatient Residential Day
Care
Reduction in Primary Drug Use 75.7%
97.7%
77.9%
30-Day Retention
71.8%
64.1%
68.9%
90-Day Retention
48%
38.1%
40.6%
Exit Interviews
48.6%
73.7%
47%
Strategies & Suggestions for
Improving Outcomes
Improving Exit Interview Rates
• Some ways to increase rates of exit
interviews include
– Informing the participant of the exit
interview when admitted.
– Include completion of the exit interview as
part of the individualized treatment plan.
– Remind the participant when they are
nearing completion of their plan of the exit
interview.
Improving Engagement & Retention?
• Engage them from first contact
• Make same day appointments if possible.
• Appointment cards
• To encourage forgetful clients.
• Phone calls
• To remind clients of appointments.
• To follow-up with clients who may have
dropped out.
Other Suggestions?
Challenges
• LACPRS
– Provides a lot of good information.
– Revisions and monitoring of information will be
needed to ensure quality & validity.
• Site Reports
– Also requires revisions to format & content.
– Not getting past Executive Directors’ eyes.
Your Views on the Site Reports
• What do you think about the
site reports?
• What have you heard from your
directors?
• Do you have suggestions or
recommended changed to the
reports?
Future of the Site Reports
• Assess individual sites in the key areas
described in this presentation.
• Develop peer performance standards.
• Programs that fall short will be offered
– Information,
– Training,
– and other forms of assistance.
LACES Phase III
• Focus on engagement and retention.
• Continue with community contact.
• Continue to expand and refine the site
reports.
– Include Narcotic Treatment Programs.
– Develop web-based training and educational
modules.
• Continue the expansion of LACES activities
– Discharge Outcome Report
Discharge Outcome Reports (DOR)
• Provides admission and discharge
information on a single individual.
– A site report for one person.
• Useful when you need to report
treatment outcomes to others.
• May also be useful when transferring
someone to a new level of care outside
your agency.
The End