Transcript Measuring the Quality of the Scottish Education System

Quality-adjusted Output Measure
for the Scottish Education System
Richard Murray
Economic Adviser
Scottish Executive
Structure of Presentation

Section 1: Background
–
–


Structure of the Scottish Education System
Context for the work
Section 2: Quality-adjusted Output Index
Section 3: Concluding Remarks
–
–
Limitations of the New Measure
Summary
Section 1: Background
Structure of Scottish Education
System


Different education system in Scotland compared
with the rest of the UK
Primary education:
–
–

7 years of education (P1 to P7)
2,184 schools & 382,783 pupils
Secondary education:
–
–
–
6 years of education (S1 to S6)
Compulsory to S4
381 schools & 312,979 pupils
Structure of Scottish Education
System



Comprehensive system
Examinations sat in final three years of
secondary education (S4, S5 & S6)
No national testing before S4
Context to this Work

Atkinson Review
–
Specific recommendations for education
 Improve the way we record inputs
 Move towards an output-based approach
 Develop measures to track changes in quality
of output
 Extend coverage across whole of UK
Section 2: Quality-adjusted
Output Index for Education
Rationale for Adjusting for Quality
Why adjust for quality?
–
To capture improvements in the education
pupils receive
How do you measure quality?
–
–
Ideally, the quality of teaching pupils receive
Difficult to measure this, though attainment
provides a useful proxy
Differentiating Output by Attainment


Aim to capture wide spectrum of attainment a pupil
receives by end of secondary education
3 categories which capture all levels of attainment
at school
Category 1
ISCED 3
Category 2
Between ISCED 2 & 3
Category 3
ISCED 2
Proportion of Pupils Gaining Qualifications
Proportion of Pupils Gaining Qualifications
0.60
0.50
0.40
0.30
0.20
0.10
0.00
1997
1998
Category 1
1999
2000
2001
Category 2
2002
2003
Category 3
2004
Weighting the Different Levels of
Attainment



Each category of attainment needs to be
given a weight, relative to its contribution to
total output
Inappropriate to use cost weights
Expected future earnings provides a proxy
for the benefit pupils receive from education
Expected Future Earnings by Highest
Qualification
Expected future
earnings
=


Average
earnings by
highest
qualification
Employment
rate by
X highest
qualification
Results show that higher qualifications lead to
higher expected future earnings
Weightings of 48: 32: 20 for Categories 1: 2: 3
Quality-adjusted Output Index
Methodology
Step 1: Differentiate S6 pupils
by attainment
Step 2: Apply a weight (based
on expected future earnings) to
the attainment categories
Step 3: Calculate the weighted
total & convert to index
Quality-adjusted Output Index for Final
Year of Secondary Education
Number of
Pupils
2000
2001
2002
2003
2004
Category 1
18,486
18,583
18,479
18,494
18,315
Category 2
10,191
11,005
11,100
11,414
11,666
Category 3
28,203
28,024
28,079
28,870
28,531
Total Pupils in
Cohort
59,250
60,138
59,998
61,035
60,447
1,777,500
1,804,621
1,803,780
1,830,318
1,823,082
98.5
100.0
100.0
101.4
101.0
Weighted
Total
Output Index
Quality-adjusted Output for Education



Only measures the output of one year group
(i.e. S6)
Apply the differentiated output across all
pupils in primary & secondary education
This assumes quality will be the same for
each year group
Quality-adjusted Output for Scottish
Education System
Number of
Pupils
2000
2001
2002
2003
2004
Category 1
213,402
210,277
207,859
204,239
201,522
Category 2
117,645
124,533
124,850
126,048
128,362
Category 3
325,575
317,116
315,838
318,829
313,922
Total Pupils in
Cohort
683,980
680,507
674,867
674,056
665,090
Weighted
Total (000s)
20,519
20,421
20,289
20,214
20,059
Output Index
100.5
100.0
99.4
99.0
98.2
What is driving the New Quality
Adjusted Output Index?
104
103
102
101
100
99
98
97
96
95
94
93
1997
1998
Quantity
1999
2000
Quality
2001
2002
2003
Quality-adjusted Output
2004
Pupils Gaining No Qualifications


Believe this group should not have a positive
impact on the quality dimension of output
Still captured in the quantity dimension of the
quality-adjusted output
Sensitivity Analysis



Examined different combinations of
qualifications
Applied different weights by attributing part of
the future earnings associated with those
who go on to university
Overall, little impact on quality-adjusted
output index
Section 3: Concluding Remarks
Limitations of New Measure




Unable to track changes in quality across
each year group
Will take a number of years before
investment influences the measure
Wider benefits of education are excluded
Possibly not giving enough weight to high
attainment category
Summary



New Quality-adjusted output measure for
Scottish Education system
Output is differentiated by attainment and
weighted by expected future earnings
Output index has declined in recent years,
driven by fall in pupil numbers