Transcript Robert Landry, Ph.D. Doing Effectiveness Research at the County

Doing Effectiveness Research at
the County Level
Robert Landry, Ph.D.
Yolo County Department of Alcohol, Drug and Mental Health
Contact: [email protected]
Problem
• We are charged with helping the most difficult welfare
recipients move from a culture of welfare dependency to
independence.
• We must not only address traditional substance abuse
and mental health issues. We need to find new ways to
facilitate what is essentially a cultural transition.
• There has been little research on how to do this, let alone
whether it is even possible.
• This presentation covers Yolo county’s attempt to come
to grips with this problem.
Questions
• What percentage of CalWORKs participants are referred
for Alcohol Drug and Mental Health services?
• How many of these participants enter treatment?
• What barriers do they face?
• How are the barriers related to engagement in ADMH
treatment and to success in employment services?
• Do clients get better in treatment?
• Does ADMH treatment help clients achieve greater
financial independence?
What percentage of CalWORKs participants are
referred for Alcohol Drug and Mental Health
services?
26%
How many of these participants enter treatment?
Attendance of MH Appointments
Referred for Service
100%
80%
Attend Follow Up
55.5%
Attend At Least 3 Sessions
41.5%
26%
0% 20% 40% 60% 80% 100%
What barriers do clients face?
• In a program with a broader mandate than simply
treating mental health problems, we found it
necessary to summarize the clients’ employment
barriers.
• We developed an Employment Barriers Checklist.
• We also measured the clients’ symptom distress with
the BASIS 32 and their social functioning with the
CA-QOL.
Percentage of Clients with Barriers
 Mental Health
 Alcohol/Drug
 Domestic Conflict
 Attitude
 Education
 Children
 Work
 Medical Problems
 Housing
 Legal
 Transportation
85.3
60.6
55.8
35.5
32.9
30.4
25.6
22.4
19.6
16.8
15.9
Client Reported Symptomatology and
Quality of Life
• The Basis32 and the CA-QOL were administered during
assessment (n = 288).
• The BASIS32 is a symptom inventory completed by the
client.
• We compared the means of our sample to that of the
normative female in-patient group at discharge .
• The CalWORKs group is slightly worse than the inpatient group at discharge, indicating CalWORKs
ADMH clients are reporting serious pathology.
How are the barriers related to engagement in
ADMH treatment and success in employment
services?
Method Tasks:
• Collapse the ADMH termination status into a treatment and
no treatment group.
• Collapse the CWES disposition (what the client was does
after termination) into a participation and no participation
group.
• Decide which barriers, or independent variables, we will
measure.
57%
43%
69%
31%
Frequency of Cancellation/No Shows
For Clients Who Start Treatment
MISSED
PERCENT
APPOINTMENTS
0
51.3
1 or 2
37.3
3+
27.4
There was a significant relationship between the number of
no shows and starting treatment.
(Pearson’s Chi Squared 17.599 df = 2 p < .001)
Frequency of Cancellation/No Shows
For Clients Who Participate in CWES
MISSED
PERCENT
APPOINTMENTS
0
46.1
1 or 2
47.3
3+
24.6
There was no difference between the 0 missed session
group and the 1or 2 missed session group. However, there
was a large difference starting with 3 missed sessions
(Pearson’s Chi Squared 11.147 p = .004)
Client Reported Symptomatology and
Quality of Life
• The Basis32 and the CA-QUAL were
administered during assessment (n = 288).
• There was no relation found between client
reported symptomatology or quality of life and
participation in either treatment or employment
services.
Percentage of Clients Who Participate
in CWES By Number of Barriers
NUMBER OF PERCENT
BARRIERS
2-4
50%
5-6
40%
7+
14%
Barriers associated with starting
treatment:
• Anxiety
• Depression
• Interpersonal/Axis II Problems
• No High School Diploma
• Diagnosed Chronic Physical Disability
Barriers associated with a lack of
participating in CWES:
•AD abuse
•Poor Attendance
•Past Domestic Violence (inverse relationship)
•Unstable Housing
• Never Worked.
Conclusions about Barriers and Starting
Treatment or Participating in CWES
• The presence of barriers increases the chances
of starting treatment and decreases the
chances of participating in CWES.
• Assessment no shows were associated with
poor participation in both treatment and CWES.
Do clients get better in treatment?
• Tracking therapeutic and quality of life changes is
essential for determining the impact of treatment
on employment.
• The BASIS 32 and CA-QOL are readministered
every 15 sessions and at termination.
• Significant symptom reduction was indicated by the
BASIS 32 and significant improvements in quality
of life were indicated on 3 of the 9 CA-QOL scales.
How Does Starting Treatment Affect the Chances
That a Client Will Participate in CWES?
Started
Treatment
Probability of
CWES
Participation
NO
36%
YES
58%
(Pearson’s Chi Squared p < .001).
Is there a causal relationship between treatment
and success in employment?
•Experimental designs provide the most powerful proof of causality,
but can be ethically and legally problematic.
•We recommend A B designs and other time study designs.
•We tracked quarterly change in treatment completion and found the
percentage of clients who completed treatment increased by 38%
over 2 years.
•The percentage of treatment completers who participated in
CalWORKs and found employment stayed constant at 56-58%.
•This indicates there is a causal relationship between treatment and
employment.
Conclusions: Outcome Research
• The clients self report high levels of symptomatology.
• The degree of symptomatology within the group referred for treatment
is not a good predictor of participation in treatment or CWES.
• Therapy drop out rates are very high.
• Most clients report they improve in therapy.
• There is a relationship between starting treatment and participation in
employment services.
Conclusions: Program Implications
• More than 2 missed assessment appointments warrants flagging for
alternate interventions.
• Never employed participants need alternate interventions.
• There needs to be a high degree of integration of therapy and other
components in order to address interrelated issues.
• Therapy should have a significant focus on motivating clients to
overcome their life barriers.
• Intensive case management may be needed to engage the most
intractable clients.
• Once the most intractable clients are engaged there needs to be
graduated pathways towards increasing responsibility, or an effective
SSI advocacy program if indicated.
The Prerequisites For Outcome
Assessment
• A manager committed to outcome assessment.
• A user friendly database that makes life easier for staff by:
– Simplifying standard case management paperwork tasks.
– Collects most data as a bi-product of what they are doing anyway.
– Provides the staff with quick access to useful information.
• A commitment of resources to the start up process.
• Someone familiar with research to help with the design and number
crunching.