Transcript slides - Editorial Express

Paolo Severati
Scientific Coordinator
of the Project and
Head of Training
Policies Evaluation Unit
ISFOL – Institute for the
Development of Vocational Training
for Workers
Roma, Corso d’Italia, 33
Post-secondary
vocational training
courses: are they
effective for Italian
unemployed youth with a
high school diploma?
COMPIE 2014 Conference
Rome, 27th November 2014
A national project
• Ministry of Labour and Social Policies
(applicant and provider of the co-financing of the
project)
• ISFOL - Institute for the Development of
Vocational Training for Workers
• ASVAPP & IRVAPP
• Coordinamento delle Regioni (National
Coordination Committee of the Italian Regions)
Outline
•
•
•
•
•
Evaluation design and methodology
Average impact estimates
Subgroup analysis
Conclusions and policy implications
References
EVALUATION DESIGN AND
METHODOLOGY
Policy intervention and target
population
 The study is focused on ‘post-diploma’ (PD) training courses
targeted mainly to 20-29 year old unemployed individuals with a
high school diploma
 Within the whole spectrum of the training programmes co-financed
by ESF in Italy, the features of PD interventions are quite
homogenous across the country
 PD training courses are very intensive and have a strong focus on
the acquisition specific job skills
 So, they should reasonably have a positive effect on individual
employability
Geographical scope
•
•
•
•
Piedmont
Trento
Veneto
Lazio
Time coverage
• Different from Region to Region, according to
the programming choices and availability of
data. For example, in Piedmont PD courses started between 2007
and 2011 were chosen (from 2-6 year coverage, outcome data
being available until the end of 2013)
• PD courses last from less than one to two years
• Lock-in effect (generally absorbed after maximum 2 years)
Methodology
• Propensity score matching
• Kernel matching estimator
• Blocking with regression adjustment estimator
(Imbens [2014])
Data requirement
• Propensity score matching is data hungry in terms of
the number of variables to estimate participation and
outcomes
• Three archives were used and merged for the
analysis:
1.
2.
3.
Regional archives on training policies (co-funded by ESF or not);
Public Employment Service Archives;
COB (Comunicazioni Obbligatorie) Archives, containing information
that every private employer or agent is legally bound to communicate online to the COB archive in order to initiate, modify or terminate any workrelated contract
Accurate description of the pre-intervention labour
market history is crucial to derive a credible
comparison group
• Pre-programme (un)employment is an important
predictor of programme entry and employment
outcomes
• (un)employment history can capture
unobservable characteristics (as, for instance,
motivation) which could influence participation
and outcomes
Design of evaluation
AVERAGE IMPACT
ESTIMATES
Average impact estimates
Evidence from our analysis shows:
1. Significant and positive effects on individual
employability
2. Lock-in effect (months after the beginning of the course the
treatment group exhibits a lower probability of employment, but a
positive effect on the probability of employment is evident in
subsequent months)
Piedmont - Trainees employment rate and
counterfactual estimate, months -24 to 48 since the
.3
.2
.1
employment rate
.4
.5
beginning of the training courses, years 2007-2011
-24
-12
0
12
24
36
months before and after beginning of training
trainees
counterfactual
48
Trento - Probability to be at work for participants and
controls after matching, months -36 to 27 since the beginning
of
the
training
courses,
years
2010
and
2011
Veneto - Probability to be at work for participants and
controls after matching, months -60 to 36 since the beginning
of the training course, years 2008 and 2009
Effects on other outcomes and
matching procedure
• It seems that participation to courses reduces the
probability of finding an open-ended contract
• Positive effect on number of weeks spent working in
a year
• Impact estimates do not depend on the matching
procedure used
Piedmont – Average impact estimates
Piedmont
- Sensitivity analysis
Trento - ATT on the probability to be at work estimating
using kernel matching (KM) and blocking with regression adjustment
(BRA)
Months s ince beginning of the cours e
Sample s ize
6
12
18
24
27
-0.2538 ***
0.1050 **
0.1606 ***
0.1532 ***
0.1268 ***
0.0431
0.0356
0.0402
0.0397
0.0975 **
0.1752 ***
0.1553 ***
0.1405 ***
0.0423
0.0422
0.0411
0.0398
Participants
Controls
Participants
excluded from
the analys is
Al l
KM
Se
BRA
(0-0.1042)
Se
0.0298
-0.2178 ***
0.0409
156
11301
15
147
11221
24
Trento - ATT on the probability to get an open – ended contract
and on the number of weeks at work estimating using kernel
matching (KM) and blocking with regression adjustment (BRA)
Sa mple size
Months s i nce begi nni ng of the cours e
Open-ended contract
6
KM -0.0243
Se
BRA
(0-0.1042)
Se
Yea rl y weeks a t work
KM
Se
BRA
(0-0.1042)
Se
Participants Controls
12
-0.0254
18
-0.0159
24
0.0035
27
-0.0022
0.0157
0.0215
0.0267
0.0298
0.0292
-0.0223
-0.0175
-0.0235
0.0047
-0.0017
0.0195
0.0239
0.0268
0.0278
0.0278
0-12
-8.3094 ***
0.9599
-7.1689 ***
1.3971
12-24
7.2843 ***
Participants
excluded
from the
analysis
156
11301
15
147
11221
24
156
11301
15
147
11221
24
1.3957
8.0987 ***
1.7611
Veneto - ATT on the probability to be at work estimating
using kernel matching (KM) and blocking with regression adjustment
(BRA)
Months s ince beginning of the cours e
6
12
18
24
30
Sample size
36
Participants
excluded
Participants Controls
from the
analysis
Al l
KM -0.1712 *** 0.0441
Se
BRA
(0-0.0309)
Se
0.0214
0.0281
-0.1740 *** 0.0397
0.0271
0.0285
0.1129 *** 0.0931 *** 0.0901 *** 0.0947 ***
0.0294
0.0337
0.0335
0.0306
0.0312
97,992
0
239
97,819
9
0.0316
0.1153 *** 0.0973 *** 0.0877 *** 0.0960 ***
0.0301
248
0.0313
Veneto - ATT on the probability to get an open – ended contract
and on the number of weeks at work estimating using kernel matching
(KM) and blocking with regression adjustment (BRA)
Sa mple size
Participants
excluded
Participants Controls
from the
analysis
Months s i nce begi nni ng of the cours e
Openended
6
12
contra ct
KM -0.0574 *** -0.0120
Se
0.0105
18
24
30
36
-0.0065
0.0128
0.0215
0.0147
0.0209
0.0232
0.0262
0.0281
0.0289
-0.0167
-0.0096
0.0084
0.0104
0.0050
0.0197
0.0222
0.0237
0.0252
0.0259
248
97,992
0
239
97,819
9
248
97,992
0
239
97,819
9
BRA
(0-0.309) -0.0628 ***
Se 0.0165
Yea rl y
weeks at
work
KM
Se
BRA
(0-0.309)
Se
0-12
-4.9437 ***
0.6635
-5.2639 ***
0.9768
12-24
24-36
4.9188 ***
5.1016 ***
1.3489
1.4461
4.8239 ***
4.9653 ***
1.3282
1.4272
Lazio - ATT on the probability to be at work at 12, 24, 36
months after the beginning of the treatment
ATT nearest neighbor
ATT stratification
ATT Kernel
12
24
36
-0.077*** -0.048** 0.018
-0.097*** -0.057 *** -0.000
-0.127*** -0.077*** -0.009
*, **, *** stands for 10%, 5%, and 1% statistical signif icance. Common
support imposed
Lazio - ATT on the probability to get an open-ended
contract
at 12, 24, 36 months after the beginning of the treatment
ATT nearest neighbor
ATT stratification
ATT Kernel
12
24
36
-0.026*** -0.026*** -0.022
-0.038*** -0.037*** -0.041***
-0.055*** -0.053*** -0.052***
*, **, *** stands for 10%, 5%, and 1% statistical signif icance. Common
support imposed
SUBGROUP ANALYSIS
Effects on subgroups are different in
each Region
• Age. In Piedmont, the impact seems to be higher for younger people
(18-20) who just got a diploma. In Veneto the impact is larger for
older participants.
• Gender. In Piedmont, few differences between women and men. In
Veneto the effect is larger for females
• Length: in Piedmont some evidence on the existence of positive
correlation between course length and employability. In Veneto
slight evidence of a positive correlation between (short) length and
the (short) size of the lock-in effect
Piedmont – Average impact estimates, by age
Piedmont - Average impact estimates, by gender and
nationality
Trento - Average impact estimates, by age and gender
Months s i nce begi nni ng of the cours e
By a ge
<=23
KM
Se
BRA
(0 - 0.1222)
Se
6
12
-0.2795 ***
0.1384 **
0.1731 ***
0.0974 *
0.1134 *
0.0541
0.0513
0.0566
0.0579
0.1750 *** 0.1964 ***
0.1224 **
0.1215 **
0.0588
0.0589
0.0570
0.0540
0.1017
0.1282 **
0.1624 **
0.1069
0.0728
0.0625
0.0696
0.0720
0.0900
0.1527 **
0.2043 ***
0.1018
0.0728
0.0733
0.0700
0.0686
0.2139 *** 0.1712 ***
0.1600 ***
0.1350 **
0.0493
0.0536
0.0545
0.1999 *** 0.1801 ***
0.1714 ***
0.1309 ***
0.0535
0.0532
0.0522
0.0500
0.1773 **
0.1974 **
0.1422 *
0.0759
0.0851
0.0783
0.2571 ***
0.2332 ***
0.1528 **
0.0787
0.0770
0.0742
0.0447
-0.2722 ***
0.0563
18
24
Sa mple s ize
27
Pa rti ci pants
Control s
Pa rti ci pa nts
excl uded from
the a na l ys i s
87
5321
9
83
5277
13
60
5980
15
44
5697
31
101
6992
18
91
6886
28
49
4110
3
41
4037
11
>23
KM
Se
BRA
(0 - 0.0677)
Se
By gender
Fema les
KM
Se
BRA
(0 - 0.1026)
Se
-0.2256 ***
0.0532
-0.1368 *
0.0711
-0.2334 ***
0.0393
-0.2151 ***
0.0521
0.0477
Ma l es
KM
Se
BRA
(0 -0.0847)
Se
-0.2281 *** -0.0401
0.0650
-0.1840 **
0.0774
0.0742
-0.0059
0.0788
Veneto - Average impact estimates, by age and gender
Months s i nce begi nni ng of the cours e
6
12
Sa mple size
18
24
30
36
0.0730
* 0.0769
* 0.0515
0.0655
0.0412
0.0410
0.0394
0.0410
0.1078 *** 0.0989
Participants Controls
Participants
excluded
from the
analysis
By a ge
<=23
KM -0.1674 *** 0.0166
Se
BRA
0.0293
0.0423
41,406
2
*
140
41,312
9
**
99
56,586
0
**
89
56,436
10
** 0.1202 ***
128
57,988
0
125
57,909
3
116
40,004
4
110
39,788
10
** 0.0701
* 0.0792
0.0389
0.0407
0.0413
0.0418
0.0422
KM -0.1866 *** 0.0459
0.1334
** 0.0763
0.1191
** 0.1105
0.0520
0.0496
0.0505
0.0485
-0.1923 *** 0.0797 * 0.1189
** 0.0892
* 0.1109
** 0.1064
0.0488
0.0498
0.0501
0.1226 *** 0.0993
** 0.0899
(0-0.0504)
Se
-0.1610 *** 0.0333
147
0.0362
>23
Se
BRA
(0-0.0262)
Se
0.0344
0.0439
0.0469
0.0453
By gender
Fema les
KM -0.1619 *** 0.0483
Se
BRA
(0-0.0316)
Se
0.0303
0.0382
-0.1580 *** 0.0356
0.0385
0.0482
0.0403
0.0448
0.0447
0.1427 *** 0.0983
** 0.0863
0.0434
** 0.1254 ***
0.0403
0.0422
0.0428
0.0431
0.0434
KM -0.1685 *** 0.0461
Ma l es
0.0942
* 0.0969
** 0.0978
** 0.0779
0.0430
0.0504
0.0491
0.0451
0.0479
BRA -0.2275 *** 0.0431
0.0685
0.1014
** 0.0608
0.0569
0.0451
0.0456
0.0466
0.0469
Se
(0-0.0354)
Se
0.0274
0.0400
0.0417
Conclusions and policy implications
• Training courses have a positive effect on
individual employability (and on number or
weeks worked in a year)
• Effects on different target groups are
different from Region to Region
• Length of courses and lock-in: interesting
but slight evidence. More analysis is needed
References
•
•
•
•
•
Ashenfelter,O. (1978), Estimating the effects of training programs on earnings, The
Review of Economics and Statistics, Vol.60: 47-57
Crépon, B., Ferracci, M. and Fougère, D. (2007). Training the unemployed in France:
how does it affect unemployment duration and recurrence? Bonn: IZA discussion
paper n. 3215
Heckman, J.J., LaLonde, R.J. and Smith, J.A. (1999). The economics and
econometrics of active labor market programs. In Ashenfelter, A. and Card, D.
(Edited by) Handbook of Labour Economics, Volume 3: 1856- 2097. Elsevier
Imbens G. (2014), ‘Matching Methods in Practice: Three Examples’, IZA Discussion
Paper n. 8049
Lechner, M. (1999). Earnings and employment effects of continuous off-the-job
training in East Germany after reunification. Journal of Business & Economic
Statistics, 17(1): 74-90
References
•
•
•
Lechner, M., Miquel, R. and Wunsch, C. (2007). The curse and blessing of training
the unemployed in changing economy: the case of East Germany after unification.
German Economic Review, 8(4): 468-509
van Ours, J.C. (2004). The locking-in effect of subsidized jobs. Journal of
Comparative Economics, 32(1): 37-55
Sianesi, B. (2004). An evaluation of the Swedish system of active labor market
programs in the 1990s. Review of Economics and Statistics, 86(1): 133-155
Thank you for your attention
[email protected]