Transcript Chapter

Chapter 13
Survey Designs
Power Point Slides by Ronald J. Shope in collaboration with John W. Creswell
Key Ideas
•
•
•
•
•
•
Intent and use of survey research
Types of survey designs
Key characteristics of a survey
Constructing survey questionnaires
Designing and conducting survey interviews
What are the steps in the process of
conducting survey research?
• How do you evaluate survey research?
Educational Research 2e: Creswell
What is survey research?
• Survey research designs are
procedures in quantitative research in
which investigators administer a survey
to a sample or to the entire population
of people in order to describe the
attitudes, opinions, behaviors, or
characteristics of the population.
Educational Research 2e: Creswell
When do you use a survey?
•
•
•
•
Assess trends
Opinions, beliefs, and attitudes
Follow-up analyses
Evaluations
Educational Research 2e: Creswell
How did survey research
develop?
• 1817 International Survey of Education Systems
• 1890 Stanley Hall survey of children
• 1907 the Pittsburgh Survey examined social
problems
• WWI and WWII sampling techniques improved
• Applications began to vary from social science to
media
• Universities established social science research
centers
• Private organizations such as Gallup, Rand
Corporation, and Roper formed
Educational Research 2e: Creswell
What are the types of survey
designs?
Time of Data Collection
Study Over Time
Longitudinal
Study at One Point in Time
Cross-sectional
Changes
in a
Changes
Trends in sub-population
in the
the same group identified
same
population by a common
people
over time characteristic
over time
over time
Trend
Cohort
Attitudes
and
Practices
Panel
Educational Research 2e: Creswell
Community Program
Needs
Evaluation
Group
Comparisons
National
Assessment
What are key characteristics
of survey research?
• Sampling from a population
• Collecting data through questionnaires
or interviews
• Designing instruments for data
collection
• Obtaining a high response rate
Educational Research 2e: Creswell
Population, Target Population,
Sampling Frame, Sample
The Population
The group of individuals having one characteristic
that distinguishes them from other groups.
The Target Population or Sampling Frame
The actual list of sampling units from which the
sample is selected.
The Sample
The group of participants in a study selected
from the target population from which the
researcher generalizes to the target population.
Educational Research 2e: Creswell
Reducing error in survey
research
• Reducing Coverage Error: have a good
sampling frame list on which to select
individuals
• Reducing Sampling Error: select as large a
sample from the population as possible
• Reducing Measurement Error: use a good
instrument with clear, unambiguous questions
and response options
• Reducing non-response error: use rigorous
administration procedures to achieve as large
a return rate as possible
Educational Research 2e: Creswell
Forms of data collection based on who
completes or records the data
Who Completes or Records the Data?
Participant
Mailed
Questionnaire
Electronic
Questionnaire
Researcher
One on To a
Group
One
Over
Telephone
Individual Focus Group Telephone
Interview Interview
Interview
Educational Research 2e: Creswell
Aspects of survey instrument
design
•
•
•
•
•
•
Personal
Attitudinal
Behavioral
Sensitive
Scale
Open-ended
Educational Research 2e: Creswell
Question construction
problems
•
•
•
•
•
•
•
•
•
Question is unclear
Multiple questions
Question is wordy
Question is negatively worded
Overlapping responses
Unbalanced response options
Mismatch between questions and answers
Question includes overly technical language
Not all questions are applicable to all the
participants
Educational Research 2e: Creswell
Pilot testing
• Test on a small number of individuals in
the sample
• Ask for written feedback on the
questions
• Revise the survey based on the written
comments
• Exclude the pilot participants from the
final sample for the study
Educational Research 2e: Creswell
Response rate
• Mail out original questionnaire with
cover letter
• Follow it in two weeks with a second
mailing of questionnaire to those who
have not responded
• Follow again in two weeks with a
postcard to non-respondents reminding
them to complete the questionnaire
Educational Research 2e: Creswell
Three phase survey
administration procedure
Step 1:
First Mailing
of Survey
2 Weeks
Step 2:
Second Mailing
of Survey
2 Weeks
Step 3:
Postcard
Mailing
2 Weeks
6 Weeks
Start
Time
Educational Research 2e: Creswell
Planning and conducting a
mailed questionnaire survey
• Write a cover letter to invite the
participants to complete the
questionnaire
• Form and construct the questionnaire
• Identify what statistical procedures will
be used to analyze data from the mailed
questionnaire
Educational Research 2e: Creswell
Planning and designing an
interview survey
• The interviewer should maintain a
neutral stance during the interview
• Train the interviewers prior to the
interview
• Take good notes of responses or use a
tape recorder
• For telephone interviews, develop a
telephone interview guide prior to the
interview
Educational Research 2e: Creswell
Steps in conducting survey
research
• Decide if a survey is the best deign to use
• Identify the research questions or hypotheses
• Identify the population, the sampling frame and
the sample
• Determine the survey design and data collection
procedures
• Develop or locate an instrument
• Administer the instrument
• Analyze the data to address the research
questions or hypotheses
• Write the report
Educational Research 2e: Creswell
Evaluating survey research
• Was the target population or sampling frame clearly
specified?
• Was the sampling procedure specified? If a random
sample was not used, were modifications from it
explained?
• Was the sample clearly identified and the basis on
which it was chosen specified?
• Did the type of survey match the questions or
hypotheses advanced by the author?
• Was it clear whether a questionnaire or an interview
survey comprised the form of data collection? Did the
researcher identify the basis for selecting or
developing the instrument?
Educational Research 2e: Creswell
Evaluating survey research
• Was information reported on reliability and validity of
scores from past use of the questionnaire or
interview?
• Did the author mention the dates on which the
questionnaire or interview was administered?
• Were the administration procedures clearly
identified?
• If a questionnaire was used, were the procedures for
obtaining responses identified? Were follow-up
procedures used to ensure a high response rate?
• Did the data analysis match the research questions
or hypothesis to be answered in the study?
Educational Research 2e: Creswell
Evaluating survey research
• If an interview was used, were the field
procedures for collecting this information
specified? Were demographic characteristics,
previous experience, training, and monitoring
discussed for the interviewer(s)?
• Did the researcher check on the reliability and
validity of scores from the data collection in the
study?
• Was the study written scientifically and ethically?
Educational Research 2e: Creswell