Transcript Open - The Scottish Government

The NPF & Scotland Performs:
Analytical Underpinning and
Challenges
Mairi Spowage
Office of the Chief Statistician
9th June 2009
An outcome based approach
• This “New Approach” presented many
opportunities for analysts
• We could now bring to bear the whole
package of evidence to say something
meaningful about change
• A way to go, particularly in evaluation
Introduced since May 2007
National Performance Framework
Summary of National Performance
Framework
How were the indicators chosen?
There was a need to
be able to measure
progress against the
5 strategic
objectives and the
15 national
outcomes
How were the indicators chosen?
A selection of indicators
were chosen to act as a
representative set, so
when taken all together
they may be able to tell
us something about
progress on the
outcome
Longer, Healthier Lives
Scotland Performs
• Designed to show how the government is
performing against its key indicators and
targets
• New and innovative approach
• Big difference to previous administrations
The Indicators
• Mixture of many types of targets and
indicators, from existing targets to those
which were set down by legislation to
some which were not currently
measureable
• Many are direction of travel indicators
Existing Targets/ Those from
other frameworks
• HEAT – Health improvement, Efficiency,
Access and Treatment
• E.g.
Achieve annual milestones for reducing
inpatient or day case waiting times
culminating in the delivery of an 18
week referral to treatment time from
December 2011
Those contained in legislation
• All unintentionally homeless
households will be entitled to settled
accommodation by 2012
• Reduce the number of Scottish public
bodies by 25% by 2011
Those which could not be/were
not measured
• Improve knowledge transfer from
research activity in universities
• Increase the average score of adults on
the Warwick-Edinburgh Mental
Wellbeing Scale by 2011
• Reduce overall ecological footprint
Challenges
• Huge range of data sources
– Survey vs. Administrative Data
– Composite measures
– Some SG sources, some UK Departments,
some ONS, some from partner agencies
• Many are National Statistics, but many are
not, at least at the moment
• Differing frequencies of publication and
lags
Recent Change Arrows
• Each indicator has an arrow assigned to it
• Arrows comment on the change between
the last two data points
– Simple, easy to understand, can be
universally applied and is transparent
Indicator Page
Recent Change Arrow
Performance Improving
Performance Maintaining
Performance Worsening
Thresholds
• Thresholds are used to ensure that small
fluctuations are not claimed as worsening
or improvement, e.g. for smoking, NI 22;
any difference within +/- 0.5 percentage points of last
year's figure suggests that the position is more likely to
be maintaining than showing any change. A decrease of
0.5 percentage points or more suggests the position is
improving; whereas an increase of 0.5 percentage points
or more suggests the position is worsening.
Thresholds
Thresholds take into account:
– Known Variance
– Past trends
– Significance of change
– To some extent, the change required by the
target
Threshold Examples
• GDP
– 0.1 percentage points
• Social Economy
– £10M
• Housing Supply
– 1,000 Houses
Who sets the thresholds?
Scotland Performs Technical Assessment
Group (SPTAG)
– Chief Statistician (chair)
– Chief Researcher
– Head of OCEA
– Chief Scientist
• SPTAG make all analytical decisions to do
with Scotland Performs
When data need to be updated…
• Each indicator has a Lead Analyst assigned to it
from within Government
• We have built in the SP updates to the standard
publication process
• The analyst submits a recommendation to the
SPTAG
• The group make comments on the presentation
of the information, to improve accessibility
• They approve or reject recommendation
SPAN
• Made up of all of the analysts who are
involved across Government
• Network meets regularly to debate
technical issues which underpin Scotland
Performs
• Support to SPTAG
• Acts as a “peer-review” function
Future Challenges
• Reflecting the contribution of the wider
public sector – our partners
• Evaluation of outcomes – how do we know
we have achieved them?
• What does success look like?
Questions?
www.scotlandperforms.com
[email protected]