Transcript - UNDP-ALM

Farm Household Surveys Design and Sampling for Collection of Data
Ernest L. Molua
(University of Buea, Cameroon)
Data Collection for
Impact Evaluation
• Develop the sampling frames of households and randomly
select households for the survey
• Develop, translate and test the survey instrument
• Recruit and train data collection and data entry staff
• Develop and test the data entry system
• Implement the survey
• Enter the data
• Clean the data through basic analysis
• Document the data sets
4/8/2015
Training on Ecomomic Analyses
of Climate Change
2
Develop the sampling frames
• Obtain a list of the sites to be included in the
evaluation, along with the villages in each site.
• Select a sample of villages in each site (province,
region, division, etc.) for the household survey.
– It may include random sampling of villages per site.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
3
Develop the sampling frames
• Obtain or create a list of households in each
village.
– Census or other administrative data may be
used for this purpose if they are available and it
is determined that they can be used to build
such a list.
– If the data are not available, you will be
required to visit selected villages to identify
households for the sample frame.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
4
Develop the sampling frames
– Once the list of households is established, the
researcher will use this list to select a random sample
of households in each village for the survey.
– This list must be both ordered and large enough for the
households to be screened for eligibility at the time of
interview and to achieve the requisite number of
eligible households for the study.
– The research team will typically supply the screening
criteria, which will allow interviewers to identify
farming households.
• Then estimate your potential sample size
4/8/2015
Training on Ecomomic Analyses
of Climate Change
5
Village/Community selection
• Villages may be selected in consultation with ongoing
development programmes and national and local
authorities. Some are typically selected on the basis
of the following criteria:
– Cover the main climate variability related hazards and
choose villages with different levels of exposure (e.g. low
drought risk vs. high drought risk).
– Cover the main agricultural systems in the country.
– Choose some villages which are participating in ongoing
development programmes.
– Employ appropriate sampling techniques
4/8/2015
Training on Ecomomic Analyses
of Climate Change
6
Develop, translate and test the survey instrument
Questionnaire
• Household questionnaire may contain five categories of
questions.
1. First, for creating a household profile (H), questions are
raised on household composition, education, occupation,
migratory behavior, crops cultivated, livestock owned
and assets owned.
2. Outputs and inputs used – quantities and prices
Why? Q = f(X), where Q is output depending on X
which are various inputs.
Prices are used to estimate revenues and costs, total
revenues and total costs, so that Net Revenue = Total
Revenue – Total Cost.
In African mixed cropping systems, revenue is the
preferred dependent variable
Develop, translate and test the survey instrument
Questionnaire
3. Information is collected about the effects or types of
hazards (C) the household is facing in order to find out
whether climate related hazards are indeed perceived as
important problems.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
8
Develop, translate and test the survey instrument
Questionnaire
4. Information is elicited about the choice of
adaptation strategies (A). Which strategies have
been adopted in the past, for reducing which
hazard, and what were the necessary investments
for this?
5. Finally, questions are raised about the assistance
institutions (I) provide for adopting adaptation
strategies. Which institutions did assist, what type
of assistance did they provide and was this
assistance helpful?
4/8/2015
Training on Ecomomic Analyses
of Climate Change
9
Develop, translate and test the survey instrument
Questionnaire
It must contain questions to allow us generate data
to estimate output (or revenue) and its determinants,
as in the following equations:
Revenue = f(H,C,A,I)
Assuming a linear relationship:
R = k + aHi + bC + cA+ dI + error
Assuming a Ricardian non-linear relationship:
Log R = k + aHi + b1R + b2T + b3R2 + b4T2 +
b5R*T + cA + dI + error
4/8/2015
Training on Ecomomic Analyses
of Climate Change
10
Sample Questionnaire
4/8/2015
Training on Ecomomic Analyses
of Climate Change
11
Develop, translate and test the survey instrument
Questionnaire
• translate the English version into the appropriate local
languages, format it, and pretest it in several of the
languages.
See French questionnaire
– ensure the quality and accuracy of the translations as well
as the cultural relevance, -contract professional
translators.
– train field staff to conduct the pretest, and for entering the
pretest data into Excel,
– Questionnaires used must be made available by authors
upon request.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
12
Recruit and train data collection and data entry staff
• Recruit data collection and data entry staff.
– The ideal interviewer will be proficient in (1)
interacting with all kinds of people, (2) building a
rapport with respondents, and (3) dealing with
quantitative data.
– Given the complexity of the survey instrument, it is
likely that interviewers will, at a minimum, need the
equivalent of a primary-level education and a high
level of literacy in the language(s) of the instrument.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
13
• develop and provide interviewers with a training guide that
includes an in-depth explanation of the survey questions.
Lead researcher will review and provide feedback on the
training manual and on all other training materials.
– Data entry clerks should also attend the interviewer
training.
– Training participants will be required to attend all sessions
and prove competence through a test administered at the
end of training.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
14
Field trips
Locating & Selecting Households
• In consultation with local authorities, within each
village, households are randomly selected and
institutional stakeholders identified (remember to
employ appropriate sampling techniques)
• Moreover, people are selected for the focus
group discussion in such a way that different
sexes, age classes and wealth classes are present.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
15
Field trips
Institutional Stakeholder selection
• Institutional stakeholder interviews should focus
on the types of services they provide, their role in
assisting households with adaptation, their
linkages with authorities and the main constraints
limiting their activities.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
16
Field trips
Focus group selection
• Focus group discussion can be organized with a group of
between 10 to 20 people.
– These group discussions give additional information
about the perceptions on the main (climate) hazards and
their changes; about the reasons for adopting strategies;
about the advantages, disadvantages, bottlenecks, strong
points or weak points of the different adaptation options;
about the institutions facilitating the adoption of
strategies; and about differences in strategy adoption
between socio-economic groups.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
17
Other data collection strategy
4/8/2015
Training on Ecomomic Analyses
of Climate Change
18
Develop and test the data entry system
• After the survey instrument has been finalized, you
develop the data entry system and provide a protocol for
data entry and cleaning
• Test the data entry system by entering mock data from
surveys filled out by interviewers (e.g. pretests) and fix
any problems that are identified.
• Remember to train data entry clerks, manage the double
data entry of all questionnaires (and reconcile any
differences) and the cleaning of the data.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
19
Implement the Survey
• Logistics must be available (enough questionnaire copies,
transport, accommodation, etc.)
• Field supervisors should be in charge of ensuring the quality
of the collected data.
– They should review each questionnaire soon after it is
completed and ensure interviewers return to respondents if
questions are skipped, answers are ambiguous, or other
problems with questionnaires are identified.
– Supervisors will also re-train interviewers or otherwise
ameliorate difficulties if systematic problems are found.
• During survey implementation period, submit weekly updates
on the number of households contacted, the number of
refusals, and the number of completed surveys.
20
Implement the Survey
• Data should be entered as it is collected, and provide
feedback to the data collection team as problems are
identified.
• Lead researcher should examine thoroughly after the first
five percent of cases are entered.
• Conduct random audits of a sample of questionnaires to
ensure that the data collected and entered are reliable and
accurate.
4/8/2015
Training on Ecomomic Analyses
of Climate Change
21
Enter, Clean and Document Survey Data
• Double-enter the data, compare them, and reconcile any
differences.
• Check the entered data for logical inconsistencies and return to the
original questionnaires to resolve them.
– If inconsistencies found in the original questionnaire data are
not caught by the field supervisor while in the field, re-visiting
respondents should be considered. If it is not possible to return
to the field for re-interviews, missing values should be coded
consistently.
• Lead researcher should also ensure that all variables are named and
labeled according to agreed specifications.
• Once the data are cleaned and labeled, the lead researcher should
provide a codebook that will include a description of all variables
collected.
4/8/2015
Training on Ecomomic Analyses
22
of Climate Change
Key Personnel
Survey Director
• To guide the data collection effort, ensure that it is
implemented effectively and oversee technical aspects
Field Supervisor
• To oversee data collection in the field,
– including assuring proper dispatching of interviewers to the correct
survey sites, determining which interviewers will be assigned
specific cases, and ensuring cases are completed. Conduct quality
assurance checks ideally while interviewers are still in the field and
could return to respondents if errors require. Correct any data
collection problems, including re-training interviewers when
necessary for systematic problems or changes to protocols or
instruments.
4/8/2015
Training on Ecomomic Analyses
23
of Climate Change
Key Personnel
Data Entry Manager
• To oversee the technical aspects of developing and testing
the data entry system; recruit and train data entry staff;
manage data entry, track progress, and monitor quality.
Enumerators / interviewers
• About ten, and well trained to obtain and record responses
Data Entry Clerks
• About 4, and well trained to transcribe information from
questionnaire into excel sheets
4/8/2015
Training on Ecomomic Analyses
of Climate Change
END
24