Transcript Slide 1

EPAG Impact Evaluation:
Preliminary Midline Results
EPAG Study Team
(World Bank / MoGD/ Subah-Belleh Associates)
Making Cents / Youth Economic Opportunities Conference
12 September 2012
The Adolescent Girls Initiative
• Countries: Liberia, Nepal, South Sudan,
Afghanistan, Rwanda, Jordan, Laos, Haiti
Afghanistan
Jordan
Haiti
Nepal
South Sudan
Liberia
Rwanda
Laos
The Adolescent Girls Initiative
• Objective: Vocational and entrepreneurship
training for young women
• Design Features:
– Private sector/NGO training providers competitively
selected to provide market-relevant skills
– Vocational Skills for wage or self employment
– Life skills training to address girls’ vulnerabilities
– Financial literacy and business development
– Stipend (Liberia, Rwanda, Afghanistan)
– Job placement through performance-based
contracts
Global Perspective of the AGI
• Cross-country learning
– Core evaluation team works across countries
– Reports shared across countries
– International workshops at different stages of project
to share lessons learned
• Expansion: Potential to influence AGI programs in
new countries
• Sustainability: By comparing outcomes, we can learn
what methods work best.
These lessons can inform design of future programs
and the scale-up of these pilot projects
AGI in Liberia (EPAG)
• Implemented by Ministry of Gender and Development
• Design Features:
– 4 service providers competitively selected (plus 4 more
sub-contracted)
– Training delivered in 2 rounds in 9 communities
– In Round 1: Job Skills (35%) and Business Development
Services (65%)
– Wide variety of job skills areas: painting, hotel/
restaurant work, driving, etc.
– Coverage: 2500 girls in Greater Monrovia and Kakata
(1191 in Round 1; 1300 in Round 2)
– Six month follow-up period for job placement
Impact Evaluation
• Like other evaluations, IE focuses on outcomes
– What is the effect of a specific program on specific
outcomes?
• But unlike other evaluations:
1. IE starts before the project begins
2. IE compares the beneficiary group with a similar group of
individuals who do NOT receive the project
• This method allows us to attribute causality:
– How much better off are beneficiaries because of the
program?
– How can we know that the outcomes we see are due to the
program, rather than other factors?
Research Design
• Objective: To measure the impact of the program on
the well-being of participants and their families
• Methodology: Randomized selection to treatment and
comparison groups
• Data collection using a series of household and
individual surveys:
– Baseline: 2010
– Midline: 2011
– Endline: 2012 (in the field now)
Research Questions
What we will learn:
• Does the program improve the economic well-being of young
women who participate? What is the impact on employment,
earnings, investment, savings, borrowing, and lending?
• What is the impact of the program on a wide range of
socioeconomic behaviors and outcomes, such as reproductive
health, time management, experience of gender-based and
other violence, and attitudes toward risk?
• Does the program promote the empowerment of participants,
as measured by proxies such as decision-making, aspirations for
the future, and control over household resources?
• How do the program impacts vary according to the demographic
and personal characteristics of the participants?
Research Design
Sample size
2106
Originally recruited
25
Very pregnant
(assigned to round 2)
808
1273
Assigned to
round 2
Assigned to
round 1
39
116
1157
Never started
training
Started
training
Re-assigned to
round 1
34
Started
training
1191
Entered round 1
1131
Completed round 1
769
Control group
Response Rates
Table 1. Response rates for baseline and midline surveys
Midline
Not
interviewed
Not interviewed
Baseline
Interviewed
Total
56
56
112
Interviewed
314
1680
1994
Total
370
1736
2106
Baseline Survey
• Benchmark the socioeconomic status and demographic
characteristics of the young women who would be
participating in the EPAG program to provide a basis for
comparison in the future
• Validate whether the treatment and control groups are
statistically viable comparison groups for the impact
evaluation
• Should not be used to make generalizations about
adolescent girls and young women in Liberia overall
– EPAG girls are more educated than the “average” girl
– EPAG girls live in more urban areas than the “average” girl
Results: Employment
Likelihood of working
Percent with at east 1 IGA
0.70
0.67
0.65
0.60
0.55
Control
0.50
0.45
0.45
0.40
0.35
Treatment
0.39
0.38
0.30
Baseline
Midline
Notes:
• The treatment group in the graph includes only those who actually started the
training.
• The overall midline employment rate for the treatment group is 67%. This includes
the Job Skills (52%) and BDS (76%) tracks.
Results: Earnings
Table 2. Impact on Weekly Earnings (LD)
Indicator
(1) Entire sample
(2) Those who took part in program
Treatment
Effect
Tstatistic
Number of
Observations
947.88
2.66***
3608
1,019.40
2.79***
3467
(3) Those working
902.56
1.14
1648
(4) Those who took part in program and
are working
977.83
1.21
1596
*** significant at 1%
Note: An increase of 1,109 LD per week is roughly equal to 58 USD per month. The
average earnings at baseline was 43 USD per month.
Results: Savings
Table 3. Impact on Savings
(1) Any Savings? (Yes/No)
Treatment
Effect
T- Statistic
Number of
Observations
0.458
14.90***
3591
(2) Amount saved at home (USD)
-1.681
0.79
3582
(3) Amount saved at bank (USD)
30.188
7.35***
3590
(4) Amount saved at credit group( USD)
4.289
1.80*
3608
(5) Amount saved at susu( USD)
6.201
2.09**
3607
(6) Amount saved at Nigerian susu (USD)
4.487
2.97***
3608
44.526
6.81***
3563
(7) Amount saved total (USD)
* significant at 10%; ** significant at 5%; *** significant at 1%
Note: EPAG participants were given small stipends and a $25 bonus for completing the
course.
Results: Self-Confidence
Table 4. CHANGES IN ATTITUDES OVER THE PAST YEAR (only asked at midline)
Treatment
effect
T- statistic
Number of
observations
(1) I feel more able to work well with people now than a year ago
0.178
4.45***
1666
(2) I feel more comfortable with who I am now than a year ago
0.243
6.37***
1670
(3) I feel more in control of my life now than a year ago
0.170
4.08***
1670
(4) I feel more able to call upon my friends for support than I was
0.164
3.04***
1666
(5) I am more able to help my friends now than I was a year ago
0.288
5.98***
1668
(6) I am more comfortable in situations now with people I do not
0.201
4.82***
1665
0.167
2.98***
1650
a year ago
know than I was a year ago
(7) I am more outgoing now than I was a year ago
Note: A higher number indicates more likely to agree or strongly agree with the statement.
* significant at 10%; ** significant at 5%; *** significant at 1%
Summary
What have we learned?
• Large increases in employment (55% increase)
and earnings (115% increase)
• Positive impacts on savings and selfconfidence
• Stronger effects for Business Skills trainees
than for Job Skills trainees
– But the business skills income is total enterprise
revenue, not earnings or profits.
Summary
Next Steps:
• Examine impact on households
• Investigate heterogeneous impacts: by
community, type of training, age, etc.
• (After completion of endline survey) Look at
longer-term outcomes: do these positive
effects persist, grow, or weaken over time?
• Understand profits and incomes of businesses.
Thank you