Transcript survey - World Bank Internet Error Page AutoRedirect

Assessing Information
Needs and Survey
Alternatives
Kathleen Beegle, DECRG
Poverty and Inequality Course
Module 1: Multi-topic Household Surveys
January 23, 2008
1
The Demand for Data
1.
Performance-based management
has created pressures on developing
countries to improve the quantity
and quality of their macro and
micro-data:




Is the public sector delivering good
services?
Are country policies/poverty reduction
strategies reducing poverty?
Is aid supporting poverty reduction?
In the World Bank: “Results-based”
CASs, for example
2
The Demand for Data
1.
2.
Performance-based management
Millennium Development Goals (MDGs)
reflects the strong commitment on the
part of the international development
community and national governments to
monitor and evaluate the results of their
policies and programs (United Nations,
2000).
3
The Demand for Data
1.
2.
3.
Performance-based management
MDGs
HIPC, Poverty Reduction Strategies
(PRSPs)
The HPIC (debt reduction initiative for heavilyindebted, low-income countries) requires

a plan for poverty reduction

that can be measured and monitored

access to concessional lending from the
World Bank and International Monetary
Fund (IMF/IDA, 1999a and 1999b).
4
The Demand for Data

Much of the increased demand for data
has been focused on household-level
data.




It is not possible to monitor PRSPs or MDGS
without solid household data
Measurement of welfare and other key social
indicators. (Muñoz and Scott, 2005).
Decisions on appropriate policies to reach the
MDGs or the targets in the PRSPs need
household data.
Although they do not exclusively
need/require only household-level data,
although they often imply survey data
5
What these demands look like

PRSP
 Measure welfare/poverty
 Identify problems--magnitude, causes
 Alternative policies
 Cost/benefit
 Monitor
 Evaluate
6
What these demands look like


PRSP
MDGs
MDG
MDG
MDG
MDG
MDG
MDG
MDG
MDG
1:
2:
3:
4:
5:
6:
7:
8:
Eradicate extreme poverty and hunger
Achieve universal primary education
Promote gender awareness, empower women
Reduce child mortality
Improve maternal health
Combat HIV/AIDS, malaria and others
Ensure environmental sustainability
Develop a global partnership for development
7
What these demands look like



PRSP
MDGs
General Demand







Poverty and Inequality
Benefit Incidence Analysis
Public services
Determinants of observed outcomes
Assessment of alternative policies
Impact Evaluation
Inputs to Program Design
8
Recent Efforts to Increase Data:
Quantity and Quality


Partnership in Statistics for Development in
the 21st Century (PARIS21) was initiated to
“…to act as a catalyst for promoting a culture
of evidence-based policymaking and
monitoring in all countries, and especially in
developing countries.” (PARIS21 web site).
1999
Trust Fund for Statistical Capacity Building
(TFSCB), managed by the World Bank, set up
in 2000 to help build the capacity of statistical
systems/ statistical plans
9
Recent Efforts to Increase Data:
Quantity and Quality




World Bank line of credit for Statistical
Capacity Building to support countries in the
implementation of these statistical master
plans.
The Program to Improve Surveys of Living
Conditions in Latin America (MECOVI), (cosponsored by the IADB, ECLAC, WB, regional
program to improve quality and data
SPARC- Eastern Carribean Initative to improve
surveys, UNDP, CDB, WB IADB, OECS, inter
alia. (2004)
LSMS Phase IV: Methodological Research on
measurement, field work and technological
advances
10
Focus of Presentation




Assessing Information Needs
Sources of data
Need for multiple sources of data
Role of household surveys
11
Assessing Information Needs
Inputs
Internal
 Financial & physical indicators of
inputs (monthly)

Outputs
 Achievement/performance
indicators (annually)

Outcomes
External
indicators
 Benefits/usage
(annually)

Impact
 Indicators of improvements in
living standards (~ 5 years) 12
Data Sources






National accounts
Current public expenditure statistics
Program of Price collection (cons./prod.)
Administrative Records (from line
ministries)
Qualitative Work
Surveys:







Multi-topic w/ welfare focus (LSMS/IS)
Monitoring (CWIQ, PS)
Income and Expenditure (IES, HBS)
Single topic (Labor Force Surveys (LFS)
Demographic and Health (DHS))
Enterprise
Facilities
13
Surveys and Policy Analysis
Gov’t Programs
Conditional
Cash Transfers
Day care
centers
Public Health
Campaign
Social Goals
Households
Individuals
Firms
Increase
enrollment
Increase female
LFP
Lower infant
mortality
14
Surveys: Going Beyond Rates
Understanding secondary school enrollments, 12-18 year olds, Albania 2002
100%
90%
80%
70%
Percent
60%
50%
40%
30%
• In almost all countries
we have a single
statistic: mean
enrollment at the
national level. In this
case it is 61%.
Average
•This is interesting for
monitoring purposes,
but it doesn’t say much
about poverty or other
factors.
20%
10%
0%
•... A regional
disaggregation would
be useful
Understanding secondary school enrollments, 12-18 year olds,
Albania 2002
• In some countries we
100%
90%
80%
Urban
have regional
breakdowns, with
marked contrasts
70%
Percent
Average •The contrast between
60%
50%
40%
Rural
urban and rural rates
emphasizes the
disadvantages faced by
rural communities.
30%
20%
10%
0%
• Other breakdowns
would be useful
Understanding secondary school enrollments, 12-18 year olds,
Albania 2002
• …possibly, official
100%
statistics can add the
gender dimension
90%
80%
70%
Percent 60%
50%
Male
Urban
Female
Male
Female
Rural
•…the figures show
that, in urban areas,
Average there is no gender
differential but a large
gap in rural areas.
40%
30%
20%
10%
0%
•But we still don’t
know much about
who sends their
children to school
Understanding secondary school enrollments, 12-18 year olds,
Albania 2002
100%
Female, urban
90%
Male, urban
80%
Male, rural
70%
Female, rural
Percent 60%
Average
50%
•…With a survey we
can show enrollment
rates broken down
by consumption level-and thus understand
an additional
dimension
40%
30%
20%
10%
0%
Q1
Q2
Q3
Q4
Consumption quintile
Q5
Gathering information
through surveys (household or other)


There is a range of options
They can be ordered along two main
dimensions:


degree of representativeness
subjective/objective dimension
19
Degree of Representativeness
Case
study
Purposive
selection
Quota
sampling
Small prob.
sample
Large prob.
sample
Census
Subjective/Objective Dimension
Direct measurement
Questionnaire
(quantitative)
Questionnaire
(Qualitative)
Case
study
Purposive
selection
Quota
sampling
Structured
interview
Small prob.
sample
Open
meetings
Conversations
Subjective
assessments
Large prob.
sample
Census
Wonderful World of Surveys
Direct measurement
Questionnaire
(quantitative)
Participatory Poverty
Assessments
Case
study
Questionnaire
(Qualitative)
Sentinel Site
Surveillance
Purposive
selection
Participant
observation
Windscreen
Survey
Census
Household Budget
Survey
LSMS/ IS
CWIQ/PS
Quota
sampling
Structured
interview
Small prob.
sample
Beneficiary
Open
Assessment meetings
Conversations
Subjective
assessments
Large prob.
sample
Community
Surveys
Census
Wonderful World of Surveys:
“Statistical Surveys”
Direct measurement
Questionnaire
(quantitative)
Household Budget
Survey
LSMS/ IS
CWIQ/PS
Census
Questionnaire
(Qualitative)
Case
study
Purposive
selection
Quota
sampling
Structured
interview
Small prob.
sample
Open
meetings
Conversations
Subjective
assessments
Large prob.
sample
Census
Features of a Statistical Survey


Structured Questionnaire
Random/Probability Sample
24
Tradeoffs to Consider When Planning a
Survey as Part of a System of Surveys

Overall scope






Single vs. Multi-topic
Probability vs. Purposive Sampling
Sampling vs. Non-Sampling Errors
Time vs. Cost
Data vs. Capacity Building
Surveys over time: repeated cross
sections, panels, rotating
25
Summary

Surveys are one source of information
among many (system of information)

Consider all the key elements of a
National Poverty Monitoring System
26
Key Elements of a National Poverty
Monitoring System








Timely Annual National Accounts
Current Public Expenditure Statistics
Program of Consumer and Producer Price
Statistics
In-depth Welfare Survey (LSMS/IS, IES?)
‘Light’ Annual Monitoring Surveys (CWIQ,
PS)
Longitudinal Studies
Qualitative work on key topics
Specific tools for project/program/policy
monitoring and evaluation
27
Summary


Surveys are one source of information
among many (system of information)
No one survey can meet all data needs:
System of Household Surveys
28
System of Household Surveys

Goal: System able to respond to
evolving needs: not produce data X or
survey Y
 Determine data needs before they
are URGENT
 Identify appropriate instruments,
 Implement them properly, timely
fashion,
 Analyze the resulting data
29
Improving the SHS



Linking Users and Producers
Providing adequate resources
Continuous Survey Program


Not necessarily permanent survey
Benefits






Avoid loss of capacity
Create greater levels of capacity (building on
existing)
Economies of scale
Policy makers know when data will be available
Protects NSO from pressures for ad hoc surveys
Ongoing system actually allows more flexibility
30
and responsiveness
Summary




Expanding demand for timely,
relevant data
Need to determine the range of data
needs to begin to define a system of
information
Surveys are one, important, source
of information among many
No one survey can meet all data
needs: System of Household Surveys
31
References




United Nations (2000). “United Nations Millennium
Declaration.” United Nations’ General Assembly, Fifty-fifth
Session, New York, New York.
International Monetary Fund and International
Development Association (1999a). “Building Poverty
Reduction Strategies in Developing Countries.” Report to
the Board of Directors, International Monetary Fund and
International Development Association, Washington, D.C.
International Monetary Fund and International
Development Association (1999b). “Heavily Indebted Poor
Countries (HIPC) Initiative: Strengthening the Link
between Debt Relief and Poverty Reduction.” International
Monetary Fund and International Development
Association, Washington, D.C.
Muñoz, Juan and Kinnon Scott (2005). “Household
Surveys and the Millennium Development Goals.” Paris21,
32
processed.
References



DECRG, (2006) “LSMS IV: Research for
Improving Survey Data”, processed
LSMS Web page:
http://www.worldbank.org/lsms/
ISLC/ MECOVI web site:
http://worldbank.org/lac
then search on MECOVI
33