Transcript Juvenile Treatment Drug Court

Using Data to Inform Practice
Michael L. Dennis, Ph.D.
Chestnut Health Systems, Normal IL
Presentation at SAMHSA/CSAT Satellite Session, “ Implementing
Evidenced Based Treatment for Adolescent Based Treatment,”
College of Problems on Drug Dependence, Reno, NV, June 20, 2009.
The opinions in this presentation are those of the author and do not
reflect positions of the government. This presentation was supported
by SAMHSA/CSAT contract no. 270-07-0191. Available from
www.chestnut.org/li/posters or by contacting author at
[email protected] or 309-451-7806.
Common Questions in
Local Program Evaluation and Development
1. Who is being served?
2. What services are they receiving?
3. To what extent are services being targeted at
those in need?
4. To what extent are services being delivered as
expected?
5. Which is the most effective of several services
delivered?
6. What does it cost, cost effectiveness?
Source: Dennis, Fetterman & Sechrest (1994)
Exploring Need, Targeting & Unmet Need
At Intake . No/Low
After 3 mon
Need
Any Treatment
21
No Treatment
1068
Total
1089
Mod/High
Need
Total
7
28
7/28=25% to targeted
139
1207
139/146=95%
unmet need
146
1235
146/1222=40% in need
Size of the Problem
Extent to which services are currently being targeted
Extent to which services are not reaching those in most need
Mental Health Problem (at intake) vs.
Any MH Treatment by 3 months
100%
89%
90%
80%
70%
78%
66%
60%
50%
40%
30%
20%
10%
0%
% of Clients With
Mod/High Need
(n=806/1214)*
% of Services Going to % w Need but No Service
Those in Need
After 3 months
(n=176/197)
(n=630/806)
*3+ on ASAM dimension B3 criteria
Source: Ives & Moritz 2009 CSAT Juvenile and Family Drug Court Presentation
71%
55%
60%
61%
79%
57%
70%
66%
80%
78%
90%
82%
89%
100%
78%
MH Issues at Intake vs.
MH Treatment at 3 months
30%
40%
31%
50%
30%
20%
10%
0%
3+ ASAM B3
In need of treatment
Any prior MH Tx
Any Co-Occ. Diag.
Services to those in need
Multiple Co-Occ. Diag.
Need but no treatment
Source: Ives & Moritz 2009 CSAT Juvenile and Family Drug Court Presentation
Other MH Issues at Intake vs.
MH Treatment at 3 months
79%
82%
23%
17%
8%
10%
17%
30%
17%
40%
23%
50%
20%
77%
39%
60%
57%
70%
58%
64%
80%
74%
90%
79%
100%
0%
Any Suicide
Prob.
In need of treatment
Ever
Victimized/
Worried
High level
Past Year
Victimization Victimization
Services to those in need
Current
Victimization
Need but no treatment
Source: Ives & Moritz 2009 CSAT Juvenile and Family Drug Court Presentation
GRRS Treatment Planning Needs:
Mental Health
Behavior control problems
Anger manangement
Major psycho-social stressors
Homocide/suicide risk
MH Medication compliance
Self-mutilation
Eating disorder
Juvenile Treatment Drug Court
Monitor Self-mutilation
Family Treatment Drug Court
Cognitive impairment
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Source: Ives & Moritz 2009 CSAT Juvenile and Family Drug Court Presentation
Why Do We Care About Unmet Need?
• If we subset to those in need, getting mental
health services predicts reduced mental health
problems
• Both psychosocial and medication interventions
are associated with reduced problems
• If we subset to those NOT in need, getting
mental health services does NOT predict change
in mental health problems
Residential Treatment need (at intake) vs.
7+ Residential days at 3 months
100%
90%
80%
70%
60%
Opportunity to
redirect existing
funds through
better targeting
84%
51%
50%
40%
30%
29%
20%
10%
0%
% of Clients With
Mod/High Need
(n=343/1203)*
% of Services Going to % w Need but No Service
Those in Need
After 3 months
(n=54/107)
(n=289/343)
Source: Ives & Moritz 2009 CSAT Juvenile and Family Drug Court Presentation
GRRS Treatment Planning Needs:
Substance Use and Treatment
Not close to one in recovery
Tobacco use
Detox or w/drawal services
Cont. care after controlled env.
Non-opioid w/drawal & relapse svs
Tx dissatisfaction
Realistic Treatment Goals and CC
Relapse prevention
Treatment Mentor
Juvenile Treatment Drug Court
Opiate w/drawal & relapse svs
Family Treatment Drug Court
Monitor withdrawal & compliance
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Source: Ives & Moritz 2009 CSAT Juvenile and Family Drug Court Presentation
Impact of Intake Severity on Outcome
10
SPSM groupings
8
OVERALL
No problems (0-25%ile)
6
1-3 problems (25-50%ile)
4-8 problems (50-75%ile)
4
9+ problems (75-100%ile)
Dot/Lines show Means
2
0
0
6
Wave
Source: ATM Main Findings data set
Intake Severity
Correlated -.66 with
amount of change
• Programs with low severity look
better with absolute outcomes
(e.g. abstinence)
• Programs with high severity look
better with amount of change
Example of Multi-dimensional
HIV Subgroups
0.01
0.00
0.00
0.20
-0.40
Unprotected Sex Acts (f=.14)
-0.60
-0.39
-0.29
-0.08
-0.20
-0.03
-0.10
-0.02
0.00
-0.04
Cohen's Effect Size d
0.15
0.10
0.20
0.27
0.40
Days of Victimization (f=.22)
-0.80
-0.69
Days of Needle Use (f=1.19)
A.
Low Risk
Source: Lloyd et al 2007
B.
C.
Mod. Risk
Mod. Risk
W/O Trauma With Trauma
D.
High Risk
Total
Possible Comparison Groups
•
•
•
•
•
•
•
•
•
published data
site over time
subsites, staff, or clinics
compare site to larger program (all sites)
compare site to similar level of care, geography,
demographic subgroup, or clinical subgroup
match clinical subgroups from GAIN related
presentations or papers
formal matching or propensity scoring to make groups
more statistically comparable
formal randomized experiments
path or mediation models to test whether it is actually
the dosage or key ingredient driving the change
Evaluation of an OTI Waiting List Reduction
Grant from Appointment and Admission Log
Used up slots in 2
months and
(unexpectedly)
had 200 person
waiting list
Grant allowed
program to add
100 slots and
reduced time to
readmission
Source: Dennis, Ingram, Burks & Rachal, 1994
Implementation is Essential
(Reduction in Recidivism from .50 Control Group Rate)
The best is to
have a strong
program
implemented
well
Thus one should optimally pick the
strongest intervention that one can
implement well
Source: Adapted from Lipsey, 1997, 2005
The effect of a well
implemented weak program is
as big as a strong program
implemented poorly
Major Predictors of Effective Programs
that we have to be cognizant of…
1. Triage to focus on the higher severity
subgroups of individuals
2. An explicit intervention protocol (typically
manualized) with a priori evidence that it
works when followed with targeted population
3. Use of monitoring, feedback, supervision and
quality assurance to ensure protocol
adherence and project implementation
4. Use proactive case supervision at the
individual level to ensure quality of care
Source: Adapted from Lipsey, 1997, 2005