Transcript WHAT IS THE SENSITIVITY OF THE 14

Are Washington Circle Performance
Measures Associated With Decreased
Criminal Activity Following Treatment?
Deborah W. Garnick
Constance M. Horgan
Margaret T. Lee
Lee Panas
Grant A. Ritter
DRAFT
Institute for Behavioral Health,
Heller School for Social Policy
and Management,
Brandeis University
Tracy Leeper
Steve Davis
Becki Moore
Mark Reynolds
Oklahoma Department of
Mental Health and Substance
Abuse Services (ODMHSAS)
First Annual NIATx Summit -- San Antonio Texas – April 24, 2007
Supported by: NIAAA (Grant #R21 AA14229), NIDA (Grant #R21 DA15704), and SAMHSA
through a supplement to the Brandeis/Harvard NIDA Center on Managed Care and Drug Abuse
Treatment (Grant #3 P50 DA010233)
KEY RESEARCH QUESTIONS
•
•
Is there an association between clients
meeting the Washington Circle
specifications for amount and timing of
substance abuse treatment services and
decreased criminal justice activity?
How can information on this question be
useful for state decision making?
04/24/07
2
OVERVIEW OF PRESENTATION
•
Background
–
–
–
Washington Circle Performance Measures
Oklahoma use of measures in regional
performance reports
Importance of the research question
•
Data and analytic methods
•
Results
•
Using the results to influence state policy
04/24/07
3
WASHINGTON CIRCLE: HISTORY
• Convened in 1998 by SAMHSA’s Center for Substance
Abuse Treatment
• Goals:
– Develop and pilot test performance measures for substance
abuse treatment
– Promote adoption of these measures by public and private
stakeholders (McCorry et al 2000)
• Brandeis works with WC to develop and test
performance measures -- beginning with commercial
managed care plans (Garnick et al 2002)
• NCQA adopts measures in 2003
• Oklahoma adapts two measures for regional reports in
2003
• Formation of the WC Public Sector Workgroup in 2004
04/24/07
4
WC PUBLIC SECTOR WORKGROUP
MEMBERS
Teresa Anderson, Ph.D.
Andrew Hanchett, M.P.H.
Massachusetts Department of Public Health
Maria Canfield
Brad Towle, MA, MPA
Nevada State Health Division
Charles Bartlett, MSW
Kansas Social and Rehabilitation Services
Doreen A. Cavanaugh, Ph.D.
Georgetown University Public Policy Institute
Astrid Beigel, Ph.D.
County of Los Angeles, Department of Mental
Health
Mady Chalk, Ph.D.
A. Thomas McLellan, Ph.D.
Treatment Research Institute
Minakshi Tikoo, Ph.D.
Alfred Bidorini
Michael J. Hettinger
Connecticut Dept. of Mental Health and
Addiction Services
Barbara A. Cimaglio
Vermont Department of Health
Kevin Campbell, Ph.D.
Washington State Division of Alcohol and
Substance Abuse
04/24/07
Spencer Clark, ACSW
Adam Holtzman
North Carolina Department of Health and Human
Services
Kay Miller
Thomson Healthcare
5
WC PUBLIC SECTOR WORKGROUP
MEMBERS
Sarah A. Wattenberg, LCSW-C
Frances Cotter, MA, MPH
Hal Krause
Anne Herron, M.S.
CSAT/SAMHSA
Keith Humphreys, Ph.D.
VA Palo Alto Health Care System
Ann Doucette, Ph.D.
The George Washington University
Steve Davis, Ph.D
Mark Reynolds, Ed.D.
Tracy Leeper, M.A.
Oklahoma Department of Mental Health and
Substance Abuse Services
Jay Ford, Ph.D
Network for the Improvement of Addiction
Treatment
Frank McCorry, Ph.D
Robert J. Gallati, M.A.
Dawn Lambert-Wacey, M.A.
New York State Office of Alcoholism and
Substance Abuse Services
Craig Anne Heflinger, Ph.D.
Robert Saunders, M.P.P.
Vanderbilt University
04/24/07
Jack Kemp, M.S.
Delaware Health and Social Services
Kathleen Nardini
National Association of Alcohol and Drug Abuse
Directors
Melissa Lahr Thomas, M.Ed.
Meena Shahi
Arizona Department of Health Services
Constance Weisner, Dr. P.H., MSW
University of California, San Francisco
Kaiser Permanente
6
FOUR DOMAINS – CONTINUUM OF
CARE
1. Prevention/Education -- Activities to raise the
awareness of substance abuse as a major debilitating
disorder affecting individuals, families, and society
2. Recognition -- Efforts at case-finding, including:
screening, assessment, and referral
3. Treatment – Activities associated with rehabilitation of
individuals who have an alcohol or other drug disorder
diagnosis
4. Maintenance -- Activities related to sustaining long-term
positive outcomes
04/24/07
7
THREE COMMERCIAL MANAGED CARE
MEASURES
• Identification -- Percent of adults in the
enrolled population with a SA service on
an annual basis.
• Initiation -- Percent of adults with a new
episode of a SA outpatient service and
any additional SA services within 14 days.
• Engagement -- Percent of adults that
become initiated and receive two
additional AOD services within 30 days of
the initiation of care.
Washington Circle Public Sector Workgroup Meeting #4, November 16-17, 2006, Hyatt Regency Bethesda
8
Example Outpatient Initiation and Engagement
Days
04/24/07
9
Example Initiation Only
Days
04/24/07
10
WC PUBLIC SECTOR WORKGROUP
PARTICIPANT STATES
VT
WA
MA
NY
CT
DE
NV
KS
AZ
04/24/07
OK
TN
NC
11
ANY ILLICIT DRUG USE IN PAST MONTH AMONG PERSONS
AGED 12 OR OLDER, BY STATE: PERCENTAGES, ANNUAL
AVERAGES BASED ON 2003 AND 2004 NSDUHS
04/24/07
12
GOALS OF WC PUBLIC SECTOR
WORKGROUP
• Improve delivery of substance abuse
treatment services in public sector at state
level
• Adapt WC performance measures for use
in states for continuous quality monitoring
• Common approach among states
04/24/07
13
TRANSLATING THESE MEASURES FOR PUBLIC SECTOR
APPLICATIONS –OPPORTUNITIES AND CHALLENGES
• Opportunities
 Importance of public sector in treatment of substance use
disorders
 Current interest in performance measurement
 More detail about types of services
• Challenges
 No enrolled population
 Variability in states’ data reporting capabilities
 Data completeness influenced by some clients’ multiple
funding sources, e.g., state agency & Medicaid
 More detail about types of services
04/24/07
14
REVISIONS TO SPECIFICTIONS –
EXPANDED TO TEN MEASURES
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
04/24/07
Identification
Initiation after Outpatient
Engagement after Outpatient
Initiation after Intensive Outpatient
Engagement after Intensive Outpatient
Continuity of care after Assessment Service
Continuity of care after Detoxification
Continuity of care after Short-term Residential
Continuity of care after Long-term Residential
Continuity of care after Inpatient
15
WC/NIATx Comparison
•
•
•
WC Initiation = percent of clients,
who have an index* outpatient
service with no other SA services
in the previous 60 days and
received a second SA service
(other than detox or crisis care)
within 14 days after the index
service
WC Engagement = percent of
clients who initiated outpatient
substance abuse treatment and
received two additional services
within 30 days after initiation
WC Continuity of care = percent
of clients with an additional
services within 14 days -calculated separately for
assessment, detox, residential
and inpatient
04/24/07
•
•
NIATx Time from 1st request =
time from first request for service
to 1st treatment unit of service
post-admission.
NIATx Retention = Percent
clients receiving 4 units of
services within 30 days of 1st day
of tx.
16
DEFINITION: OUTPATIENT INITIATION
Initiation = Individuals with an OP index* service
who received a second service** within
14 days after the index service
Individuals with an OP index service
*Index service defined as first service after a 60-day
“service-free period.” Can have assessment or detox
during service-free period.
**Not detox or crisis care
04/24/07
17
PILOT RESULTS: OUTPATIENT
INITIATION
100%
9
73%
Percent
63%
15,470
0%
State
CT
61%
4,176
42%
42%
22,822
33,031
MA
NC
10,115
OK
WA
Note: Numbers in bars are denominators for measures.
04/24/07
18
DEFINITION: OUTPATIENT ENGAGEMENT
Engagement = Individuals who initiated OP SA
treatment and received two additional
services** within 30 days after initiation
Individuals with an OP index* service
*Index service defined as first service after a 60-day
“service-free period.” Can have assessment or detox
during service-free period.
**Not detox or crisis care
04/24/07
19
PILOT RESULTS: OUTPATIENT
ENGAGEMENT
100%
9
Percent
64%
56%
91,875
46%
10,115
15,470
27%
22,822
0%
State
CT
4,176
25%
33,031
MA
NC
OK
WA
Note: Numbers in bars are denominators for measures.
04/24/07
20
Percent
OKLAHOMA – INITIATION OF S.A. TREATMENT
FOLLOWING A FIRST OUTPATIENT SERVICE
FY04-3rdQtr
100
90
80
70
60
50
40
30
20
10
0
FY04-4thQtr
FY05-1stQtr
FY05-2ndQtr
FY05-3rdQtr
FY05-4thQtr
FY06-1stQtr
FY06-2ndQtr
CN
EC
NE
NW
OK
SE
SW
TU
State
Region
04/24/07
Source: Oklahoma Department of Mental Health and Substance Abuse Services, Regional
Performance Management Report, Second Quarter FY 2006.
21
Percent
OKLAHOMA– INITIATION OF S.A. TREATMENT
FOLLOWING A FIRST OUTPATIENT SERVICE
100
90
80
70
60
50
40
30
20
10
0
FY04-3rdQtr
FY04-4thQtr
FY05-1stQtr
FY05-2ndQtr
FY05-3rdQtr
FY05-4thQtr
FY06-1stQtr
FY06-2ndQtr
CN
EC
NE
NW
OK
SE
SW
TU
State
Region
04/24/07
Source: Oklahoma Department of Mental Health and Substance Abuse Services, Regional
Performance Management Report, Second Quarter FY 2006.
22
• Another slide to be added regarding
Oklahoma’s rationale for regional
performance reports, use of the
information, experience in reporting etc.
04/24/07
23
RELATIONSHIP BETWEEN PROCESS
AND OUTCOME MEASURES
Washington Circle
Measures of Process
of Care
 Substance Use
 Healthcare Utilization
 Employment
 Criminal Activity
 Family/Social
 Initiation
 Engagement
Patient
Factors
04/24/07
Clinical Outcomes
Treatment System
Factors
24
OKLAHOMA DATA
•
Oklahoma Department of Mental Health and
Substance Abuse Services
-
-
•
Linked to state criminal justice data
-
04/24/07
Admission information on sociodemographics,
drug use, prior employment
Substance abuse service type and date
Arrest date
Incarceration date
Driving under the influence date
25
SELECTION OF CLIENTS FOR ANALYSIS
Collect pretreatment data
on criminal
justice
involvement
2000
04/24/07
Select clients
with new
outpatient
episodes
2001
Track treatment
and criminal
justice
involvement
2002
26
ANALYTIC METHODS
•
Survival analyses studying time to arrest or
incarceration for:

•
Controlled for:




04/24/07
5,163 adult clients beginning a new episode with an
outpatient service
Sociodemographics
Prior year arrest or incarceration, employment
Specific treatment program
Self-reported drug use at admission
27
DESCRIPTION OF ANALYSIS SAMPLE:
WASHINGTON CIRCLE
Washington Circle Measures
No initiation
Initiation only
Initiation and engagement
04/24/07
32 %
17 %
51 %
28
DESCRIPTION OF ANALYSIS SAMPLE:
SOCIODEMOGRAPHICS
Age
18-20
21-30
31-44
>45
11%
36%
40%
13%
Male
60%
Race
White
Black
Native American
68%
13%
16%
04/24/07
Education
<High school
≥High school
35%
65%
Marital Status
Never married
Married
Divorced/widowed
Homeless
36%
28%
35%
2%
29
DESCRIPTION OF ANALYSIS SAMPLE:
CRIMINAL JUSTICE AND EMPLOYMENT
Prior Year
Arrest or Incarceration
27 %
DUI
14 %
Employed
50 %
Year After Treatment
Arrest or Incarceration
04/24/07
15 %
30
RESULTS – OVERVIEW
•
04/24/07
For clients with a new episode of
outpatient treatment settings, initiation
and engagement in treatment was
significantly associated with decreased
arrests or incarcerations
31
RESULTS – WC VARIABLES RELATED
TO ARRESTS/INCARCERATION
Initiation only
0.9
Initiation and
Engagement
0.7**
0.4
**p<.01
04/24/07
*
*
0.5
0.6
0.7
0.8
0.9
1
1.1
1.2
Hazard Ratio
1.3
1.4
1.5
1.6
1.7
1.8
32
RESULTS – SIGNIFICANT SOCIODEMOGRAPHIC
VARIABLES RELATED TO ARRESTS/INCARCERATION
Age 45+
0.7*
Female
0.8**
Black
1.3*
≥ High School
*p<.05
**p<.01
04/24/07
0.9*
0.4
0.5
0.6
0.7
0.8
0.9
1
1.1
Hazard Ratio
1.2
1.3
1.4
33
RESULTS – SIGNIFICANT PRIOR CRIMINAL JUSTICE
RELATED TO ARRESTS/INCARCERATION
Any prior DUI
1.5***
Prior arrest/incarcer. Q1
1.8***
Prior arrest/incarcer. Q2
1.8***
Prior arrest/incarcer. Q3
1.8***
Prior arrest/incarcer. Q4
2.0***
0.8 0.9
*p<.05, **p<.01, ***p<.001
04/24/07
1
1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9
2
2.1 2.2 2.3 2.4 2.5 2.6 2.7
Hazard Ratio
34
POLICY IMPLICATIONS
•
Initial evidence that adherence to WC
performance measure for substance abuse
services is associated with lower likelihood of
negative criminal justice outcomes
•
Support for initiatives to increase engagement
in treatment
•
Benefit of linking data across state agencies
04/24/07
35
STATE-LEVEL APPLICATION OF
RESEARCH
• Importance of relationship between
Washington Circle measures and client
outcomes at the state level
• State leverage of this knowledge
– Legislators and staff
– Providers
– Academic partners
04/24/07
36