Transcript Presentation Inka Pibilovax

The Most Significant Change
ProLearn Project in India
Inka Píbilová
Impact Evaluation Conference in Wageningen, 25 March 2012
A „tsunami relief“ project
 2004 first tsunami relief in Andra Pradesh India funded by
individual donors of ADRA Czech Republic
 2007 needs assessment with Education Office among 20
schools
 2008 – 2009 infrastructure development in 8 schools to
increase and retain the number of children
–
–
–
–
–
Classrooms construction
Teacher training
Learning aids
Community sensitisation
Children participation
Evaluation Objectives
• Show evidence of the project impact on access to
education, increased education quality and child-friendly
learning environment, thus achieving learning outcomes
and higher literacy.
• Show evidence of attitudinal changes of community,
school staff, children and government representatives in
education.
• Assess sustainability of the “Pro Learn” project after
ADRA withdrew support.
• Draw lessons learnt and recommendations for
improvements in future projects with respect to
planning, implementation, monitoring and evaluation.
Evaluation Methodology - Factors
1. Logical matrix did not correspond to reality
2. Audit of the construction done, client interested in
impacts and sustainability.
3. Donor did not have experience with evaluations
and did not have a specific method in mind.
4. Lack of funding to conduct the evaluation.
5. Lack of baseline data.
6. Field mission restricted to 3 weeks
Interviews
The Most
Significant
Change
Community
conversations
Observations
Evaluator
Selection
Final
debriefing
of project team
and
community
volunteers
Draft evaluation
report
Communication with the Project Partners
Inception
phase
Field
research
Reporting
phase
1 month
1 months
2 months
Final evaluation report
Desk study
Preliminary findings & conclusions
Initial briefing and inception
Terms of Reference – Objectives,
scope, stakeholders, questions,
budget, schedule, outputs, use.
Evaluation Methodology
http://www.mande.co.uk/docs/MSCGuide.pdf
http://www.mande.co.uk/docs/MSCGuide.pdf
http://www.mande.co.uk/docs/MSCGuide.pdf
How was the technique applied
MSC Training of
community
volunteers
Collection of
the MSC
stories by
volunteers
Selection of the
MSC with
community
volunteers and
project team
MSC drawings by
children
How we selected the „MSC story“
What were the findings
What were the findings
Communication of findings
• Field mission closed after evaluation finished - key
communication had to be done at the end of the evaluation.
• The local community was to a big extent illiterate - community
volunteers were expected to debrief the respective
communities.
• Headquarters in India unavailable for debriefing – done by
Skype, donor organisation debriefed back in Europe.
• Donor organisation decided to delay the communication and
finally did not published the evaluation report.
• Though a short debriefing was held with District Education
Office, despite the original plan findings were not officially
communicated to school management, teachers and students.
• Stories had already been used in local media.
Utilization of findings
• Utilisation was affected by dismissal of the whole project
team at the end of evaluation and lack of capacities of the
heasquarters in Delhi.
• The government school staff did not own much the project
(teachers´ absenteeism was a local issue) - principals did not
attend the final debriefing and did not receive the report.
• Evaluation report did not reach the authorities, though the
interviews revealed a good awareness on the project.
• Evaluation report was used only internally in ADRA CR,
especially general lessons learnt.
Any questions?
Inka Píbilová
[email protected]