Transcript 2017-02-02 JJDPC Pen.. - EPISCenter

The Evidence-Based Prevention and
Intervention Support Center:
PA Infrastructure for EBPs
Standardized. Localized. Award-Winning.
Overview for Juvenile Justice and Delinquency
Prevention Committee
Stephanie A. Bradley, Ph.D.
February 2, 2016
Results of the PCCD & Penn State Partnership?
Over $50 million estimated net benefit to the
Commonwealth for services provided across
four fiscal years to 14,824 youth and parents
Results of the PCCD & Penn State Partnership?
Recognition:
• National Criminal Justice Association, “Outstanding Criminal Justice
Program” for the NE region
• Annie E. Casey Foundation “Center of Excellence”
Reports:
• Pew-MacArthur Foundation, Results First initiative, Implementation
Oversight for Evidence-Based Programs: A policymaker's guide to effective
program delivery
• Trust for America’s Health, Reducing Teen Substance Misuse: What Really
Works
• Annie E. Casey Foundation, Building Cross-System Implementation Centers
• Results for America & Bridgespan Group, The What Works Marketplace:
Helping Leaders Use Evidence to Make Smarter Choices
• PA General Assembly, Joint State Government Commission, Violence
Prevention in Pennsylvania: Report of the Advisory Committee on Violence
Prevention
Goals for Today
• Describe the foundation of OJJDP’s
progressive prevention strategy
• Illuminate the inner workings of EPISCenter
technical assistance
• Share knowledge and inspire
4
PCCD’S STRATEGY FOR
PREVENTION:
USING THE PUBLIC HEALTH
APPROACH
Adolescent Problem
Behaviors (Outcomes)
Equifinality- Key Point:
There are multiple routes
to address a targeted
Risk Factors (Causes)
behavior problem.
6
Adolescent Problem
Behaviors (Outcomes)
Multifinality- Key Point:
When we target risk
factors (the underlying
address more than one
problem behavior.
Risk Factors (Causes)
causes of behavior), we
7
Adolescent Problem
Behaviors (Outcomes)
Implications:
State and local agencies
interested in efficiently
addressing youth
problem behaviors
should collectively focus
on underlying risk and
protective factors that
Risk Factors (Causes)
and effectively
drive common and
shared problems.
8
Communities That Care:
The Public Health Approach in Action
Form local coalition
of key stakeholders
Re-assess risk and
protective factors
Collect local data on
risk and protective
factors
Leads to community
synergy and
focused resource allocation
Use data to
identify priorities
Select and implement evidencebased program that targets
those factors
9
5 year Longitudinal Study of PA Youth
% Change of CTC/EBP Youth Over
Comparison Group
40
• 419 age-grade cohorts over
a 5-year period
35
Youth in CTC communities
with EBPs:
25
Academic
Performance
33.2
30
20
• Lower rates of delinquency
School
Engagement
15
• Greater resistance to
16.4
10
negative peer influence
5
• Stronger school
0
engagement
• Better academic
achievement
-5
-10
-15
-10.8
-10.8
Delinquency
Negative Peer
Influence
Feinberg, M.E., Jones, D., Greenberg, M. T., Osgood, W. D., & Bontempo, D. (2010). Effects of the Communities that Care
model in Pennsylvania on change in adolescent risk and problem behaviors. Prevention Science, 11, 163-171.
10
Pennsylvania’s EBP Dissemination Model
• PA Youth Survey +
1989
• CTC prev. infrastructure & prioritize RPFs +
1994
• Targeted support for selected EBPs +
1998/2001
• Technical assistance to promote quality,
2001/2008
dissemination, and sustainability =
• Population level impact
2003/2005
• To “move the needle” on key indicators of (behavioral)
health at the POPULATION level
11
COMPREHENSIVE TECHNICAL
ASSISTANCE FOR DELINQUENCY
PREVENTION IN PA
Delinquency Prevention Initiatives
Support to
Community Prevention
Coalitions
Support to
Evidence-based
Prevention & Intervention
Programs
Improve Quality of
Local Innovative Programs
and Practices
• Communities That Care (CTC)
• Drug-Free Communities (DFC)
• Strategic Prevention Framework (SPF)
• Integrated Services Plan
• Hybrid models
13
Delinquency Prevention Initiatives
Support to
Community Prevention
Coalitions
Support to
Evidence-based
Prevention & Intervention
Programs
Improve Quality of
Local Innovative Programs
and Practices
1. Big Brothers Big Sisters (BBBS)
2. Incredible Years (IYS Parenting; Basic & Advanced)
3. Incredible Years (IYS Youth; Classroom & Small Group)
4. Familias Fuertes (FF)
5. LifeSkills Training (LST)
6. Olweus Bullying Prevention Program (OBPP)
7. Positive Action (PosA)
8. Positive Parenting Program (Triple P)
9. Promoting Alternative Thinking Strategies (PATHS)
10. Project Toward No Drug Abuse (PTNDA)
11. Strengthening Families Program 10-14 (SFP 10-14)
12. Strong African American Families (SAAF)
14
Delinquency Prevention Initiatives
Support to
Community Prevention
Coalitions
•
•
•
•
•
Support to
Evidence-based
Prevention & Intervention
Programs
Improve Quality of
Local Innovative Programs
and Practices
Family Bereavement Program (FBP)
Functional Family Therapy (FFT)
Multisystemic Therapy (MST)
Trauma-Focused Cognitive Behavioral Therapy
Treatment Foster Care Oregon (TFCO)
15
Delinquency Prevention Initiatives
Support to
Community Prevention
Coalitions
Support to
Evidence-based
Prevention & Intervention
Programs
Improve Quality of
Local Innovative Programs
and Practices
Standardized Program Evaluation Protocol (SPEP)
• Restorative Services
1. Restitution/Community Service
2. Mediation
• Counseling
3. Individual
4. Family
5. Family Crisis
6. Group
7. Mixed
8. Mentoring
• Skill-Building
9. Behavior Management
10. Cognitive-behavioral Therapy
11. Social Skills Training
12. Challenge Programs
13. Remedial Academic Programs
14. Job-Related Training
What are the Goals of EPISCenter?
Our Work Promotes These Intermediate (Mid Term) Outcomes:
How does EPISCenter achieve those goals?
Translation,
Synthesis,
and
Analysis
Strategic
Planning
and
Outreach
Training
Resource
Development
Tailored or
Direct TA
Facilitating
Networking
HARMFUL
Ineffective
unknown
Promising
EFFECTIVE
TRANSLATION, SYNTHESIS,
ANALYSIS FOR PROMOTING EBPS
Communicating About “Evidence”
Very
Confident
HARMFUL
Ineffective
unknown
Promising
EFFECTIVE
Very
Confident
Factors that influence where a program will be on the continuum:
1.
Clarity of the theory of change
2. Scientific soundness of program theory
3. Theoretical and/or structural similarity to other evaluated programs
4. Presence of an evaluation or research trial(s)
5. Quality of evaluation or research trial(s)
6. Fidelity of program implementation in evaluation/research trial(s)
7. Evidence of effectiveness/harm
8. Sustained effects over time
9. Replications, independent and varied populations
Common Pitfalls
Very
Confident
HARMFUL
Ineffective
unknown
Promising
EFFECTIVE
Very
Confident
1. Believing effective without adequate data to support the belief
2. Mistaking liking with improving
3. Doing “something” to fill a need
4. Using community programs without evaluating them
5. Adopting/running an EBP without quality and fidelity
*Bumbarger & Rhoades, 2012
Control-Oriented Programs
In a review of over 500 studies on delinquency prevention programs, discipline and
deterrence programs increased youth recidivism.
Increased Recidivism
Howell, J.C., & Lipsey, M. W. (2012) Research-based guidelines for juvenile justice programs. Justice
Research and Policy, (14) 1, p.1-18.
22
Juvenile Awareness Programs
Control Group Difference Notes
Tour-Con
Recidivism
Study
Year
Tour Group
Recidivism
Michigan DOC
1967
43%
17%
+26%
Illinois Scared
Straight
1979
17%
12%
+5%
Michigan JOLT
1979
.69
.47
+.22
Virginia Insiders
1981
41%
39%
+.02%
Texas Face-to-Face
1981
36-39%
28%
+8-11%
New Jersey Scared
Straight
1982
41%
11%
+30%
California SQUIRES
1983
81%
67%
+14%
Mississippi Project
Aware
1992
1.32
1.25
+.07
Avg offense rate
Avg offense rate
Youth in awareness programs were 72% more likely to have delinquency outcomes
compared to youth who did not participate in these programs.
Petrosino, A., Turpin-Petrosino, C., and Buehler, J. (2003). Scared Straight and other juvenile awareness
programs for preventing juvenile delinquency: A systematic review of the randomized experimental
evidence. The ANNALS of the American Academy of Political and Social Science, 589, 41-62.
23
Adolescent Brain Development
Image from: Steinberg, L., (2013). The influence of neuroscience on US Supreme Court decisions about
adolescents’ criminal culpability. Nature Reviews Neuroscience, 14, 513-18.
24
IMPLEMENTATION SUPPORT FOR
SOCIAL AND ECONOMIC BENEFIT
Strengthening Families 10-14 Program (SFP 10-14, or ISFP)
% of Youth
Lifetime Prescription Drug Misuse (PDM) SFP 10-14
18
16
14
12
10
8
6
4
2
0
15.5
13.5
9.3
8.7
5.4**
1.2**
Age 21 General
4.7**
0.6**
Age 21
Narcotics
ISFP
Age 25 General
Age 25
Narcotics
Both
outcomes
(general,
narcotics)
at both time
points
(age 21, 25)
are
statistically
and
practically
significant.
Control
Sources: Spoth, Trudeau, Shin, et al. (2013). Longitudinal effects of universal preventive intervention on prescription drug misuse: Three RCTs with late
adolescents and young adults. American Journal of Public Health, 103, 665-672.
Also see Spoth, Trudeau, Shin & Redmond (2008). Long-term effects of universal preventive interventions on prescription drug misuse. Addiction, 103(7),
1160-1168.
Notes: General=Misuse of narcotics or CNS depressants or stimulants., **p <.01.
Longitudinal Study of Rx Drug Misuse, SFP 10-14 + LST
Young Adult Outcomes – Higher-Risk Subsample
Rx Opioid Misuse (POM), Total Rx Drug Misuse (PDMT)
40
36
37.9
35
35.1
37.9
34.8
37.6
% of Youth
30
25
19.7
20
13.3*
15
10
7.5**
20*
13.8**
7.8**
5
0
Age 21 POM
Age 21
PDMT
Age 22 POM
SFP 10-14 + LST
Age 22
PDMT
Age 25 POM
Age 25
PDMT
Control
Source: Spoth, Trudeau, Shin, Ralston, Redmond, Greenberg, & Feinberg (2013). Longitudinal effects of universal preventive intervention
on prescription drug misuse: Three RCTs with late adolescents and young adults. American Journal of Public Health, 103(4), 665-672.
Note. POM = prescription opioid misuse, PDMT = prescription drug misuse overall;*p < .05; **p < .01.
Did you know?
• The Surgeon General’s recent report on
addiction featured these two programs as
effective prevention programs?
• PCCD-OJJDP has administered 55 grants for
these two programs since 2008?
How do we ensure PA gets the
researched outcomes?
The Connection Between Implementation &
Outcomes
Berkel, C., Mauricio, A. M., Schoenfelder, E., & Sandler, I. N. (2011). Putting the pieces together: An integrated model of
29
program implementation. Prevention Science, 12(1), 23-33
Supporting All Phases of EBP Implementation
Program
Selection
Training &
Start Up
Implementa
tion
Impact
Assessment
Sustainment
Coordination with program developers, funders, practitioners.
30
High Quality Implementation & Program Monitoring
Create
Implemen
tation and
Data Tools
PATHS
LifeSkills Training
ART
BBBS
Develop
Formulas
PATHS: Data collection spreadsheets for tracking, correct
calculations of delivery, attendance, dosage, constructs. Addresses
issues of minimum data to calculate. Summary tabs.
Identify
Program
Targets &
Surveys
PATHS: Tracking lesson delivery, student attendance, ageappropriate measurement tools that capture relevant short-term
outcomes
Understand
Structure
PATHS: School-based, daily lessons, integrated into curriculum, yearlong
Read
Evidence
Base
PATHS: Emotion knowledge, Self-control, Social problem solving
High Quality Implementation & Program Monitoring
Standardized
Tools
Individual
Grantees
Create
Implemen
tation and
Data Tools
PATHS
# 1343
PATHS
# 1389
PATHS
LST
# 1352
LST
# 1326
Tailored
Approaches
LST
# 1399
LifeSkills Training
ART
#1411
ART
#1417
ART
BBBS
#1423
BBBS
#1444
BBBS
BBBS
#1456
Implementation and Data Tools: Readiness Guides
Implementation and Data Tools:
Implementation Plan
High Quality Implementation & Program Monitoring
Process
Program Fidelity, Participant Dosage
Process
Number of Youth Served
Programlevel Data
Individual
Grantees
PATHS
PATHS
# 1343
PATHS
# 1389
LifeSkills Training
LST
# 1352
LST
# 1326
LST
# 1399
Aggression
Replacement
Training
ART
#1411
ART
#1417
Big Brothers
Big Sisters
BBBS
#1423
BBBS
#1444
BBBS
#1456
High Quality Implementation & Program Monitoring
On average, 95% of grantees achieve minimum levels of fidelity.
100%
98%
96%
94%
92%
90%
88%
86%
84%
82%
100% 100% 100% 100%
98%
100%
98%
95%
95%
100% 100%
Chart 3
93%
91%
89%
Percent met Minimum Fidelity
Average 95%
Minimum fidelity typically defined as at least 75% of the program delivered as
intended.
High Quality Implementation & Program Monitoring
Benefits
Current WSIPP Benefits Estimates
Costs
PCCD Cost per Youth Served
Decreased Antisocial Attitudes/Behaviors
Outcome
Process
Program Fidelity, Participant Dosage
Process
Number of Youth Served
Programlevel Data
Individual
Grantees
PATHS
PATHS
# 1343
PATHS
# 1389
LifeSkills Training
LST
# 1352
LST
# 1326
LST
# 1399
Aggression
Replacement
Training
ART
#1411
ART
#1417
Big Brothers
Big Sisters
BBBS
#1423
BBBS
#1444
BBBS
#1456
Aggression
Replacement Training
(probation)
Big Brothers/Big
Sisters Mentoring
Program
Incredible Years
(Parent Only)
Life Skills Training
Strengthening
Families Program1014
Promoting Alternative
Thinking Strategies
Project Toward No
Drug Abuse
Total
PCCD Mean Cost
Per Participant
(+/- range)
$1,223
Benefit Per
Participant
(WSIPP as of
July 2016)
$12,327
Actual
Numbers
Served
Increase
Program
Reduce
PCCD Cost-Benefit Analysis: FYs 2010-13
665
(+/- $549)
$3,394
$13,190
1649
$2,153
1244
$1,706
4225
$4,126
1423
$2,894,887
(+/- $586,745)
$7,560
2837
(+/- $59)
$89
$5,820,712
(+/- $913,702)
(+/- $412)
$229
-$3,382,168
(+/- $3,697,743)
(+/- $216)
$2,092
$16,153,566
(+/- $3,046,265)
(+/- $2,973)
$328
$7,384,087
(+/- $365,187)
(+/- $1,847)
$4,872
Mean
Estimated Total
Return on Investment
(+/- range)
$20,797,463
(+/- $166,215)
$427
2781
(+/- $31)
$939,015
(+/- $87,104)
14,824
$50,607,562
39
(+/- $8,862,962)
PA Cost-Benefit Analysis: FY2015/16
Program
(WSIPP Benefit) –
(PA Cost per Youth*)
Clinical Discharges
Juvenile Justice
Referred Youth
FY 2015-2016
Estimated Economic
Benefit
(crime reduction)
FFT
$18,398
168
$3,090,787
MST
$4,447
382
$1,698,895
Total Savings
$4,789,682
*Program cost per youth is based on average across providers or sites for each program, using the average
BH-MCO rate as reported by providers in a 2015 Rate Survey and average length of stay for completed cases
in FY 2012-2013.
FY15/16
4,965 Youth and Adults Completing
FF
SAAF
TF-CBT
TripleP
TND
LST
PATHS
IYS Class-room
IYS Small…
IYS Advance
IYS BASIC
SFP
ART
BBBS…
BBBS
0
22 18
15 10
39
78
86
1834
200
248
102
65
187
207 221
256
142
235
Youth Completing
500
1000
Adults Completing
1500
2000
FY15/16: 4,965 Youth and Adults Completing
% Youth Improved Protective
Factors
% Decreased Risk Factors
64%
69%
50%
17%
22%
+ Peer Refusal
+ Academics
+ ATOD
Knowledge
12%
- Antisocial
Attitudes/Behs
- ATOD Intent
% Improved Risk & Protective Factors
76%
60%
59%
+ Parent ATOD
Rules
+ Parental
Discipline
- Parental
Aggression
- PTSD
FYs 12-14: FFT & MST Outcomes for 5,279 Youth
• 1,934 youth completed FFT
• 3,345 youth completed MST
Outcomes included:
• 88.2% of youth remained in the community at the end of
treatment;
• 88.0% committed no new criminal offenses while involved in
the program;
• 80.3% experienced improved family functioning;
• 70.1% of youth showed improved school attendance; and
• 68.7 % of youth showed improved school performance
SPEP Initiative Data
Strategic Planning and Outreach
Goal: Promote adoption, sustainment, cost-effeciency of EBPs
Promoting use of public health model;
PA Prevention Coalition Advisory Workgroup, white paper
Cultivating partnerships, buy-in for EBPs;
Restructuring DDAP needs assessment
Uncovering sustainability needs/barriers;
MST, FFT rate survey
Identifying strategic, timely needs;
“EBP Opioid Workgroup”
Resource Development & Training
Goal: sustainability, cost savings; improving outside TA
for PA
Supplementary resources to fill the gaps
– Repackaging of eCTC Facilitator Training prep work
– Extensive development of SPEP implementation materials
Statewide coordinated trainings for savings
– BHARP-EPIS TF-CBT training for 160 people
– In-state, training capacity building and trainer oversight (SFP,
IYS, SPEP)
Responsive
Proactive
Facilitating Networking
• Successes, Barriers
• Training in
specialized topics
(MA funding)
• All models
supported, ~1x
each quarter
ON THE HORIZON
SAILING INTO THE YELLOW SUNRISE A ABSTRACT SEASCAPE
BY ANDREW CORP
FOR REFERENCE
History of Research-Based Prevention in Pennsylvania
1994: Key state leaders introduce Communities that Care (CTC) in PA
1994-2002: Initiation of CTC funding by PCCD
–
16 cycles of CTC model introduced in ~120 communities
1996: PCCD Co-funding of research for Blueprints programs
1998: Process Study of CTC conducted by Prevention Research Center
–
–
Resulted in creation of statewide TA infrastructure to support CTC
Formalized connection between CTC and EBP Initiative
1998: Initiation of Evidence-based Program Initiative by PCCD
2001: Narrowed list of supported EBPs, aka “PA Blueprints”
2008: PCCD created Resource Center for Evidence-Based Prevention and
Intervention Programs and Practices
2012: Three components of Resource Center awarded to PRC EPISCenter
2014: PCCD’s Resource Center initiative/EPISCenter receive NCJA Outstanding
Criminal Justice Program award, NE region
Eric Barron,
President
Pennsylvania State University
Social Science Research Institute
Children, Youth, and Families Consortium
Ann Crouter,
Dean
College of Health & Human
Development
Diana “Denni”
Fishbein,
Director
The Network on Child
Protection and Well-Being
Susan McHale,
Director
Jennie Noll,
Director
Bennett Pierce Prevention Research Center (PRC)
PRO Health
Lab (alcohol, skin
cancer projects)
Family
Foundations
Prevention
Innovation
Lab
PEACE
(Mindfulness,
Compassion,
and Well-being
Studies)
EPISCenter
Women and Child
Maltreatment/
Life Events Studies
P-TRAN
The Prevention Research Center (PRC) is housed in the College of Health and Human Development at Penn State University.
The PRC is one of four “foundational” centers that make up the Network on Child Protection and Well-being.
The EPISCenter is an “affiliate” center of the Network.