Transcript Ghana
Impact Evaluations: Experiences from Sierra Leone and Ghana 1 Country Summary Sierra Leone Ghana Political Post Conflict Relative stability Governance MII: 37th (48 SSA) MII: 7th (48 SSA in 2007) Economic GDP/C: US$286 PPP: US$806 GDP GR: 4.5% (2005) GDP/C: US$650 (2007) PPP: US$2,480 (2005) GDP GR: 6% (2006) Poverty 70% 28% Human Dev’t HDI rank: 177th (177) 135 (177 in 2005) Population 6 million 23 million (2008) Education 5,000 pri. Schools 1.3 m enrollment 15,200 pri. Schools 3.1 m enrollment (2007) IE Question Textbooks and TT SMCs 2 Key Evaluation Questions Sierra Leone Does textbook distribution improve basic pupils’ learning outcomes? Does the training of teachers in the use of textbooks improve pupils’ learning outcomes? Ghana What is the Impact of School Management Committees (SMCs) on learning outcomes at basic level? 3 Background to Key Evaluation Questions Sierra Leone Significant expenditures on textbooks --Target of 1:1 is yet to be reached even in intervention areas after several years of projects Teacher in-service training is very important --An estimated 40% of Sierra Leone’s 35,000 to 40,000 teachers are untrained and unqualified 4 Background to Key Evaluation Questions Ghana Significant improvements in Primary Completion Rate has led to questions of how to improve quality of basic education Community and school based interventions have become very critical with the roll out of decentralization in the management and governance of basic schools. 5 Evaluation Design Sierra Leone Four Local Councils out of 19 LCs were selected each representing one of the four regions in Sierra Leone— Local Councillors were invited to a meeting to randomly select by lottery LCs who will participate in the Impact Evaluation. Schools were randomly selected into three groups from the EMIS data as follows: -- control (30 primary schools), -- textbook treatment (30 primary primary schools), -- textbooks and teacher training treatment (30 primary schools) 6 Evaluation Design Sierra Leone Pupils in Primary 4 and 5 were selected for the IE because they could read. Primary 6 was left out because they are preparing for the NPSE and it was agreed that the IE should not be intrusive. --Baseline concentrated on achievement test scores for P3 and P4 because these cohorts would move to P4 and P5 by the time of the actual evaluation. 7 Evaluation Design Ghana 53 deprived districts are already receiving additional resources from government— these districts were selected based on a number of indicators spanning inputs, access, quality of learning. 212 public primary schools were selected randomly from these 53 distircts. 8 Evaluation Design Ghana Three groups were randomly obtained from these 212 schoolsbased on the EMIS as follows: --control group (70 basic schools) --information about SMC roles and responsibilities group (71 basic schools) --information + capacity building of SMCs group (71 basic schools) 9 Components of a Successful IE Start-up workshop was/is critical—Abuja, Dakar Field Coordinator—whose key focus on IE Government ownership and commitment, Ministry of Education involvement at concept stage—but remember to diversify involvement WB involvement; TTL’s role in trying to coordinate all stakeholders 10 Elements of a Successful IE Timely availability of funds (EPDF is great!) Follow-up vigorously on the intervention budget Understand the issues and take risks Keep impact evaluation on the radar at all times—take every opportunity to communicate effectively on IE 11 Issues and Risks Watch out for possible derailment of the process and be prepared for this e.g. changes in the sociopolitical environment; problems with funding the intervention… Political changes, changes in government Ethical concerns, who gets textbooks first? Manage potential bad press Manage the expectations, because we may find “nothing” Create the stage for the next IE 12 Political Change-Sierra Leone Change in government in Sierra Leone --New councilors were elected last August, thank goodness they thought to involve the more permanent Chief Administrators in the district and school selection workshop Commission of Inquiry in Sierra Leone --All activities have sort of come to a stand-still because of appearances before the commission. Textbooks distribution in Pujehun by the MCSL and RADA have halted because of appearances before the commission. 13 Political Change-Ghana Elections in Ghana --All sector resources are understandably concentrated on the elections --First round was on Dec. 7, run-off on Dec. 28. Critical implications for interventions budget --A new government means a new budget, new priorities etc. 14 Status of Impact Evaluation Sierra Leone --EPDF application and funds obtained --Field Coordinator is on board --Recruited Statistics Sierra Leone for data collection, data entry --Textbooks are still being distributed to treatment schools --Teacher Training Manual is being revised 15 Status of Impact Evaluation Sierra Leone—next 12 months --Complete textbooks distribution --Complete manual revision --Roll out teacher training; Roll out textbooks --Plans to have another small survey, probably testing teachers as well --Obtain the intervention budget—EFA FTI; GoSL --Data collection, analysis for follow-up survey --IE Report 16 Status of Impact Evaluation Ghana --EPDF application and funds obtained --Field Coordinator is on board --Questionnaires have been finalized --Recruitment of firm for baseline is far advanced—technical and financial evaluation reports being finalized 17 Status of Impact Evaluation Ghana—next 18 months --Baseline survey planned for Jan. 2009 --Data entry and analysis Jun. 2009 --Training of trainers for SMCs --Training of SMCs, capacity building 18 Status of Impact Evaluation Ghana—next 18 months --Mid-Term Review; classroom observations and surprise visits to measure pupil and teacher attendance --SMC survey; --Final Data Collection, data entry and analysis --IE Report 19