Transcript Questionnaires - University of Colorado Boulder

Survey questionnaires, a data
collection instrument
1.
2.
3.
4.
5.
Outline
What is a questionnaire?
Open vs. Closed Questions
Modes of questionnaire distribution
Piloting
Take-a-ways
Quiz time!
A questionnaire . . .
A.
B.
C.
D.
Is what that guy on the Hill has on his clipboard.
Is something you throw away with Tuesday’s junk mail.
Causes you to hang-up on someone around election time.
Is a cost effective means of gathering original data from a
large sample population that can be performed via personal
interview, telephone, or mail.
E. All of the above.
What is a questionnaire?
• A questionnaire is a form of data collection, a survey
methodology
• Questionnaires “pose standardized, formally structured
questions to a group of individuals, often presumed to be a
sample of a broader population . . . Questionnaires are useful
for gathering original data about people, their behavior and
social interactions, attitudes, and opinions, and awareness of
events (McGurik and O'Neill 2005).”
• Can be used as part of a mixed methodology
Geographical issues that
questionnaires can explore
•
•
•
•
•
•
•
•
•
Perceptions of risk
Social networks
Coping behaviors (HIV, unemployed)
Environmental attitudes
Travel patterns
Mental maps
Power relations in a particular setting
Gender roles in the household
Access to employment, social services, education . . .
Strengths and Weaknesses
Strengths of Questionnaires
• Gather original data about opinions, behaviors, attitudes and
awareness
• Provide insights to relevant social trends
• Cost-effective meaning you can extend your research over a
large area
• Very flexible i.e. can be combined with other methodologies
Weaknesses of Questionnaires
• Structure of questions can influence responses
• Depth and originality of answers can be limited
• Low response rates for the more cost effective modes of
delivery
Things to keep in mind
• Requires a lot of preparation and forethought, design is
critical
• Make sure you can’t get quality data from somewhere
else, don’t just use this to generate your “own” data
• Questions should be relevant to and focused on your
research question
• Each question must have a clear purpose and be relevant
to your critical analysis
• Be aware of the social and cultural limits of what your
respondents are willing or able to answer
• Can the respondents understand the questions?
• Do they have the knowledge to answer them?
• Do your questions make them uncomfortable?
Closed Questions
Closed/ Fixed
• Benefits
• Can get quantitative data from these
• Select an answer, rank, select a point on a scale
• Major benefit is that they are easy to code and analyze
• Challenges
• Demanding to design
• Have to know how the question will be interpreted and must
assume that everyone will understand the meaning of the words
and concepts to be the same
• Respondents are limited to the range of answers provided
Open-ended Questions
Open questions
• Benefits
• Less structured
• Respondents get to recall experiences or explain understanding in
their own words and terms
• Allows respondents to challenge the structure of the
questionnaire
• You can get valuable and unanticipated insights
• Major benefit is it gives you the most robust perspective, major
drawback is that it’s more challenging to analyze
• Challenges
• Can’t use this data for statistical analysis
• Requires a lot of effort to interpret
Quantitative, Qualitative, Mixed
• Fixed questions allow you to do quantitative analysis on
sample populations, used to make statistical claims about the
population at large
• In qualitative analysis questionnaires are used usually as part
of mixed methods i.e. non-probability sampling, looking for
trends, themes
• Start your mixed method questionnaire with quantitative
(closed-ended) questions and progress to the qualitative
(open-ended questions)
• *DON’T try to turn open-ended answers into quantitative
data, its mixed methods!*
Mode of Questionnaire Distribution
Mail
• The Good
• low cost
• large coverage area
• good for sensitive subjects, anonymity
• The Bad
• length and complexity limits
• once its mailed it’s out of your hands
• low response rate 30-40%
Email
• will limit your audience to those with email accounts
(obviously problematic in many researach sites)
• some report high (and detailed) response rates
Mode of Questionnaire Distribution
Face to face (interviewer)
• The Good
• 80-85% response rate
• Can usually get longer more detailed answers verbally than in
writing
• The interviewer can explain uncertain terms and insist on more
detail
• The Bad
•
•
•
•
Expensive (especially if you hire professionals), time consuming
Spatially limited
Data quality is dependent on interviewers
Interviewees may self-censor in the presence of an interviewer
Mode of Questionnaire Distribution
Telephone
• The Good
• Historically good response rates
• Cheaper and less spatially restricted than face to face interviews
• Can give the feeling of anonymity overcoming self-censorship
• The Bad
• Can introduce bias since not every has or reacts the same on the
telephone
• Not everyone has a land line, do you really expect them to use
there cellphone minutes on a questionnaire?
Piloting, pre-testing, road-testing
. . . is essential.
• Do it with a subset of your target audience, this gives you a
chance to work out kinks
• Did they interpret the questions as you hoped?
• Did they answer open ended questions fully?
• Were they uncomfortable with the questions?
• How long did it take to complete the questionnaire? (30 min
should be the max)
• If you are doing face to face interviews this is a time to build
your confidence
• Vet your questionnaire with experienced surveyors,
experience is priceless in crafting a good questionnaire
Take-a-ways
• Survey questionnaires can be a useful part of you research
toolkit
• Survey questionnaires should be viewed as mixed method
• Survey questionnaires can be helpful in reaching large areas
cost affectively
• There are tradeoffs between the depth of answers, cost
effectiveness of the questionnaire, and response rate.
Bibliography
• McGurik, Pauline M., and Phillip O'Neill. "Using Quesitonnaires
in Qualitative Human Geography." In Qualitative
Research Methods in Human Geography, edited by Iain
Hay, 147-162. New York: Oxford University Press, 2005.
• McLafferty, Sara L. "Conducting Questionnaire Surveys." In Key
Methods in Geography, edited by Nicholas J. Clifford, &
Gill Valentine, 87-100. London: Sage Publications Ltd.,
2003.
Questions and Comments