Transcript Slide 1

Putting the evidence into
evidence based commissioning
Rosie Travers
Commissioning Strategies Group
NOMS, London
[email protected]
The context
150,500 offenders being supervised in the community
84,500 people in prison
135,000 new prisoners each year
800,000 prisoner movements annually
1 National Probation Service
21 Community Rehabilitation Companies
104 Public sector prisons providing 82% of prison places
14 Privately operated prisons
4 Immigration Removal Centres
1,120 commissioned beds for young people under 18
Why Evidence-Based Commissioning?
•
It makes the best use of our resources
•
It brings us the best outcomes (not solely reoffending)
•
It makes our commissioning strategic, consistent and defensible
How …?
•
With services demonstrated to make a difference to the outcomes we want
•
By targeting issues demonstrated to be related to reoffending
•
For the people known to get the most benefit from a particular service.
What is good enough evidence?
Evidence is:
•
Randomised or matched comparison
groups or predicted vs actual designs
(for outcome studies)
•
Strong design and analysis (for
prediction studies)
•
Peer reviewed and published
•
Suitable outcomes ( such as
reconviction or a proven proxy for
reconviction)
•
Good quality studies using qualitative
techniques to further our
understanding of how and why
Evidence is not:
•
•
•
•
•
•
•
•
Making a strong argument
Experience
Intuition
Good practice
Opinion – even if it is a
consensus of views
Success stories
Hopefulness
Commitment
The minimum standard for new services
•
There is a clear description of the service, including who it is for, and the
outcomes it aims to deliver
•
There is a clear, plausible rationale for why the service should work. The
rationale should include reference to published high-quality evidence that
supports the approach being taken. NOMS welcomes innovative approaches
that are less tried and tested, but they should still be grounded in a credible
theoretical model of change
•
There is a strategy in place for gathering evidence about whether the
service works, using a high quality methodology
Building capacity
•
In 2013/2014 NOMS asked Clinks (a voluntary sector umbrella organisation)
and New Philanthropy Capital to work together on Improving your
Evidence a series of resources for the voluntary sector to develop
evaluation skills and boost understanding of how to demonstrate
effectiveness
•
http://www.clinks.org/support/evaluation-and-effectiveness
•
Grants programme to support innovation and build the evidence base
particularly among smaller providers
•
Promoting a readiness for evidence
The segmentation approach
Evidence
into
Action
NOMS priorities
Safe and decent care * Protect the Public * Reduce Reoffending
Operational
environment
Constraints
Opportunities
Budgets
Policy/change
Physical estate
Profiling tools
Risk: OGRS
Need: Offence
Responsivity:
Gender & Age
Evidence
Risk Need Responsivity
What Works (what doesn’t!)
Desistance
Engagement
Rehabilitative culture
Commissioning
Strategy
Reflect priorities
Be lead by the evidence
Focus on quality
Target resources
responsively
Segmentation:
evidence based
commissioning for
correctional
services
Profiling those in our
care
Criminal history and OASys
risk/need assessments for
RNR profile to include
offence type, gender, age,
sentence & requirements
Evidence summaries
Commissioning guidance: what services for which people
NOMS outcomes
Safe and decent care * Protect the Public * Reduce Reoffending
Evaluate
&
Review
Example segmentation table
Percentage of young adult, male and female prisoners*
with different levels of risk of reoffending
40
35
30
%
25
20
% young adults
15
% adult male
% women
10
5
0
0-24
25-49
50-74
75-89
90-100
OGRS risk of reoffending
* from 2012/13
Translating the evidence
https://www.gov.uk/government/
uploads/system/uploads/attachm
ent_data/file/280924/evidencesegmentation-2014.pdf
Translating the evidence
The features of a rehabilitative prison – A hierarchy
Resettle
Address
attitudes
and thinking
Address drug & alcohol
problems
Rehabilitative culture;
Rehabilitative
staff prisoner relationships
Safety & Decency
Level of risk impacts on the size of treatment effect
Barnes et al. (2010)
randomised control trail
found that reducing
contact with low-risk
offenders does not
increase reoffending
rates.
Bonta, Wallace-Capretta
and Rooney (2000)
found low-risk offenders
who received minimal
treatment had a 15%
recidivism rate, whilst
low-risk offenders who
received intensive
treatment had a
recidivism rate of 32%.
Bonta et al also found
that intensive treatment
markedly decreased
reconviction rates for
high-risk offenders
Lovins, Lowenkamp and
Latessa (2009) residential sex offender
treatment was effective
for medium and high-risk
offenders, but not lowrisk offenders, who fared
better with less intensive
community interventions.
Shift in SOTP delivery in custody to focus on higher risk
1200
1000
High Intensity
SOTP
Targets
800
600
Low/Medium
Intensity SOTP
400
Total SOTP
200
0
2011-12
2012-13
2013-14
Years
2014-15
Criminal Justice and Behavior, September 2014
Who benefits from cognitive skills programmes?
Predicted and actual reconviction rates
80
60
A Predicted
40
B Actual
20
Main current offence
To
ta
l
r
Ot
he
Dr
ug
s
Ac
q
ui
sit
i
ve
ry
be
Ro
b
Vi
o
le
n
ce
al
0
Se
xu
2-year reconviction rate (%)
following ETS by offence type
Who benefits from cognitive skills programmes?
Reductions in reoffending after ETS
20
15
All offence types
Percentage point
reduction in
Non-acquisitive
offenders only
10
reconviction rates
5
0
0 <=10
21 - 30
41 - 50
61 - 70
81 - 90
Total
Predicted reconviction (OGRS)
Shifting provision from acquisitive offenders increases the % point reduction in reoffending
against predicted rate from 8 to 14
Redirecting the targeting of TSP to those most likely to benefit
(custody delivery: 2012 to 2014)
Next steps ...
•
Do we see the same differential response on a newer version of the
programme – TSP?
•
Do we see the same in community delivery?
•
Do we see the same when we have a matched comparison group?
•
How much might this effect be due to the risk tool we use?
•
Did the programme impact on sexual reoffending specifically?
•
Who were these violent offenders? Were they different to those who
receive violence-specific programmes? Can we learn something about
targeting and dosage in comparing outcomes form different programmes?
•
Why did those with robbery and acquisitive offences not respond?
Criminogenic needs and reoffending
(work in progress!)
There are some factors that have been repeatedly found to predict
reoffending:
Drug misuse
Alcohol misuse
Poor family/marital
relationships
Attitudes that support
crime
Lack of work
Impulsivity & low selfcontrol
Lack of structured and
meaningful
recreation/leisure
activities to make use
of free time
Social network engaged
in crime
Homelessness
Are these equally prevalent among men and women?
Is the relationship with recidivism equally strong for men and women?
NB: N=100,000; 08/09; both settings; OASys only; 20 items; Prevalence ≠ predictiveness;
Prevalence
PROVISIONAL FINDINGS
OASys item
Poor problem solving
Impulsivity
Unemployed
Poor finance skills
Lack of closeness with family
Current psychological problems
Adverse childhood experiences
Regular activities encourage offending
Men
Women
84.77%
84.23%
77.17%
69.94%
62.26%
67.64%
53.28%
62.72%
48.86%
57.00%
31.59%
56.17%
44.04%
53.61%
56.66%
49.65%
NB: Victim of domestic violence; psychological problems
PROVISIONAL FINDINGS
Predictiveness
Needs items where the Odds Ration was greater than 1.1 –
equivalent to a 2% difference to reconviction rate
All offending
Violent offending
Men
Women
Activities encourage offending
Class A Drug Use
Employment
Activities encourage offending
Class A Drug Use*
Employment
Accommodation
Problematic relationship
Binge drinking*
Impulsivity
Binge drinking
Temper control
Problematic drinking
Binge drinking*
Temper control*
Problematic drinking
Victim of DV
Lack of closeness with family*
Accommodation
Next steps
• Finish writing it up!!
• Refresh prevalence data – particularly around drug
misuse
• New analysis on the co-occurrence of needs to identify
clusters of people who may need similar packages of
care
Our plans ...
Segmentation
• Offence type: specialisation study and analysis of needs & motivations
• Incorporate risk of harm assessments (sexual and violent)
• Further our understanding of acquisitive offending (burglary; shoplifting;
robbery)
Profiling
• Clusters of criminogenic need
• Psychosocial maturity
Whole system
• Every contact matters: Five Minute Interventionist
• Benefits of prisoner-led services
• Explore adjudications as an opportunity for positive change
Evaluations
• TSP in community & replicate “who benefits”
• Further analysis of cognitive skills with sex offenders
• Violence programmes outcomes
Evidence makes a difference
• Answering important questions for our organisation around
effectiveness and value for money but also the necessary
conditions for success
• Finding gaps, problems and opportunities
• Promoting excellence and supporting innovation
• Supporting important values such as integrity, decency,
acceptability, and making every contact matter
• Increasing capability
– Better at capturing the right information – at the right time
– Better at judging the quality of evidence
– Better use of evidence to inform choices and improve delivery and
efficiency
– Increased capability to evaluate – meaningfully