Transcript Experimental_Chp4_Final

Survey Research
Constructing Surveys
Measuring Responses
Important Considerations for Survey Items
Collecting Survey Data
Evaluating Surveys and Survey Data
Sampling
Probability Sampling
Nonprobability Sampling
Describe survey research.
Survey research obtains data about opinions,
attitudes, preferences, and behaviors using
questionnaires or interviews.
The survey approach allows researchers to
study private experience, which cannot be
directly observed.
Survey Research
What are the advantages of the survey approach?
We can efficiently collect large amounts of data.
Anonymous surveys can increase the accuracy
of answers to sensitive questions.
Surveys can allow us to draw inferences about
the causes of behavior and can complement
laboratory and field experiments.
Survey Research
What is the most important limitation of the survey
approach?
The survey approach does not allow us to test
hypotheses about causal relationships because
we do not manipulate independent variables and
control extraneous variables.
Therefore surveys are low in manipulation of
antecedents. However they can be low or high in
imposition of units.
Surveys are usually written or face to face.
Survey Research
What are the major steps in constructing surveys?
1.
Identify specific research objectives. If you want to measure
peoples’ beliefs about animal research, you need to figure out
which specific part of animal research you wish to ask about.
2. Decide on the degree of imposition of units
(degree of response restriction). Do you want low (can say or
write what they want) or high (must answer yes or no) imposition of
units?
3. Decide how you will analyze the survey
data. This depends upon the imposition of units. High imposition
needs statistics, low uses qualitative methods.
Constructing Surveys
Describe the major question types.
Closed questions (structured questions)
can be answered using a limited number of
alternatives and have a high imposition of units.
For example, “How do you feel about the job our
president is doing? Answer either very good,
good, fair, or poor.
Constructing Surveys
Describe the major question types.
Open-ended questions (open questions)
require that participants respond with more than
a yes or 1-10 rating and have a low imposition
of units.
For example, “Why did your choose your major?”
Constructing Surveys
Difficulty with questions
• Many questions on surveys have problems in how they were written.
They can be ambiguous, too complex or double barreled.
• Ambiguous- people don’t understand what the question is asking.
For instance, the question, Have you ever considered the idea of
abortion?
• Too complex – use a double negative in a sentence or a word
whose meaning is difficult to understand. “I don’t usually dislike not
being alone.”
• Double barreled – two ideas are present in the question. Do you feel
the country is going in the right direction and the president is doing a
good job?
How do researchers analyze data from each
question type?
The number or percent of responses can be
reported for closed questions.
Open-ended questions can be analyzed using
content analysis, like Yepez’s INTERSECT, in
which responses are assigned to categories
using objective rules.
Constructing Surveys
Describe a nominal scale.
Simplest level of measurement is a nominal
scale. A nominal scale assigns items to two or
more distinct categories that can be named
using a shared feature, but does not measure
their magnitude. True, false, male, female.
For example, you can sort professors into
exciting and dull categories.
Measuring Responses
Describe an ordinal scale.
An ordinal scale measures the magnitude of
the dependent variable using ranks, but does
not assign precise values.
For example, ask a subject to list his favorite
soda from favorite to least favorite. You don’t
really know how much he likes his third ranked
soda.
Measuring Responses
Describe an interval scale.
An interval scale measures the magnitude
of the DV using equal intervals between values
with no absolute zero point.
For example, Fahrenheit or Centigrade
temperatures, and Sarnoff and Zimbardo’s
(1961) 0-100 scale. Zero temperature is not a
true zero, true zero is when there is a total
absence of something. How much do you like
this professor, 0 = not at all, 1 = very little, 2 = a
little, 3 = like him, 4 like him a lot
Measuring Responses
Describe a ratio scale.
A ratio scale measures the magnitude of
the dependent variable using equal intervals
between values and an absolute zero.
This scale allows us to state that a 2-meter
board is twice as long as a 1-meter board.
For example, distance in meters, time in
seconds.
Measuring Responses
Which to choose
• Ordinal gives you more information than
nominal. So knowing which candidate
came in first, second etc (ordinal) is more
informative than just knowing who won
and who lost (nominal). But get more
information knowing percentage of people
who voted for each (ratio scale).
How should we select measurement scales?
The best type of scale depends on the
variable you are studying and the level
of precision you desire. Marital status would be nominal,
years married would be ratio.
Since psychological variables like traits, attitudes, and
preferences represent a continuous dimension. Each
individual can fall at any point along the dimension, such
as high sociability or low sociability. Different scales can
be used to measure continuous dimensions including
interval, or ratio.
Measuring Responses
How should we select measurement scales?
When working with variables like sociability,
psychologists often select the highest scale
possible since it provides more information and
allows analysis using more powerful statistics.
Measuring Responses
What should you consider when creating survey
items?
Subjects decide to refuse to answer surveys
during the start or first few questions.
Engage subjects from the start by asking
interesting questions they will not mind
answering.
Important Considerations for Survey Items
What should you consider when creating survey
items?
The first survey question should be:
1. relevant to the survey’s central topic
2. easy to answer
3. interesting
4. answerable by most respondents
5. closed format (so they can’t say “I don’t know”
Important Considerations for Survey Items
What should you consider when creating survey
items?
Whenever possible, use commonly used
response options.
Avoid value-laden questions that might
make a response seem embarrassing.
Example:
1. Should medical researchers be allowed to kill animals in the
name of important science?
2. Should medical researchers be allowed to experiment on
animals for important scientitifc reasons even if it might cost the
animal’s life?
Important Considerations for Survey Items
What is a response style?
Response styles are tendencies to respond
to questions or test items without regard to their
actual wording.
People differ in their willingness to answer,
position preference, and yea-saying and naysaying.
Important Considerations for Survey Items
Explain the willingness to answer response style.
Willingness to answer is the tendency to
guess or omit items when unsure. If tell subjects
that there are no right answers may be more
responses.
Important Considerations for Survey Items
Explain the position preference response style.
Position preference is selecting an answer
based on its position.
For example, students choosing “c" on multiplechoice exams.
Or consistently answering “true” without reading
the question because all the other statements
were “true.”
Important Considerations for Survey Items
What is manifest content?
Manifest content is the plain meaning of the
words printed on the page.
While we expect subjects to respond to the
manifest content of questionnaires, they may
ignore it when answering questions about their
feelings or attitudes. Subjects may just say “yes”
to everything rather than pay attention to the
question.
Important Considerations for Survey Items
What are yea-saying and nay-saying?
Yea-saying is agreeing with an item
regardless of its manifest content.
Nay-saying is disagreeing with an item
regardless of its manifest content.
To control for this, must mix up questions with some needed a “no”
response and some needing a “yes” response.
Example: I am happy most of the time.
I enjoy being with other people
When I can, I avoid noisy places.
Important Considerations for Survey Items
What are context effects?
Context effects are changes in question
interpretation due to their position within a
survey.
This problem is especially likely when two
questions are related and not separated by
buffer items (unrelated questions).
Important Considerations for Survey Items
Example of context effects
• You want people to rate Miley Cyrus on this scale:
• Nice-----------------------Nasty
• If this item appeared right under another item asking the person to
rate Kerry Washington as:
• Not sexy ____________sexy
• The subject might rate Miley Cyrus according to the “sexy” question
that appeared before it. To control for this, you put a buffer question
in between, like,
• Are the Koch brothers:
• Kind ___________________cruel
Explain the social desirability response set.
The social desirability response set is representing ourselves in a
socially appropriate fashion when responding to a question’s latent
content (underlying meaning). It is when a subject responds to a
question in a way to make himself look good in the experimenter’s
eyes.
For example, a question may ask whether you are able to get along
well with loud-mouthed obnoxious people. The person responds
“yes” to be more likeable.
There are scales available to show if a person has high social
desirability. If he does, you may not be able to use his data in your
analyses. For instance, the Marlow-Crowne Social Desirability Scale.
Collecting Survey Data
Social Desirability Scale
http://www.cengage.com/resource_uploads/
downloads/0495092746_63626.pdf
Collecting survey data
•
•
•
•
•
•
Written questionnaires
Mail surveys
Telephone surveys
Internet surveys
Interviews
Focus groups
Written questionnaires
•
•
•
•
Keep instructions simple and clear
Control for reactivity by giving privacy
Keep anonymous
If questions can be embarrassing, be
prepared to minimize discomfort by giving
privacy, not looking at their responses,
assuring anonymity
Mail
•
•
•
•
•
Always include a cover letter that is polite
Include a stamped envelope for return
Response rates are between 45% to 75%
If possible, include a small gift
Keep track of those who do not return it; consider a
second mailing.
• When nonreturn rate is high, this compromises your
results (perhaps only those interested in the topic
returned it, volunteer bias)
Computer and internet
• Can use programs such as
surveymonkey.com and surveygizmo.com
• Allows for easier collection and analyses
• Less concern with social desirability, feels
more anonymous
• Allow for larger pool of collection data
• Cannot tell if person takes survey multiple
times
Telephone
• Large scale telephone surveys use random digit dialing,
not a phone book to make calls.
• Allows for a wider sample
• Response rate for phone surveys is 60 to 90%
• Caller ID allows more people to refuse to answer call.
• Male telephone interviewers are more effective than
females.
Interviews
• Most expensive and time consuming method
• Female interviewers tend to be more successful than
male
• Must be able to establish a rapport
• Best results come when interviewer matches the
respondent on race, physical appearance,
socioeconomic status.
• Will interview be structured or unstructured
Compare structured and unstructured interviews.
In structured interviews, questions are asked
the same way each time. It is read from a script.
This provides more usable, quantifiable data.
In unstructured interviews, the interviewer can
explore interesting topics as they arise. These
data may not be usable for content analysis.
Collecting Survey Data
Focus groups
• Small groups of people with similar
characteristics, all women, or all teachers.
• The interviewer is called the facilitator
• Facilitator guides the group through a discussion
of specific issues.
• Are usually open-ended questions
Evaluating
• How good a survey is depends upon how reliable and
how valid it is
• Reliability comes in different forms, but in general, it
refers to consistency.
• Responses to similar questions should be consistent;
the survey generate similar responses across different
survey-givers, the survey should generate a very similar
response if it is given to the same subject more than
once
Evaluating
• Validity- Does the survey measure what
it’s supposed to measure?
What is the relationship of a sample to its
population?
Sampling is deciding who will fill out your survey.
A population consists of all people, animals, or objects
that share at least one characteristic. For instance, all
undergraduate students, all senior citizens.
A sample is a subset of the population of interest. Data
collected from the sample can be used to draw
inferences about the population.
Sampling
Generalizability
• How accurately we can generalize our findings
from a given sample to a population depends
upon its representativeness.
• Representativeness is how closely the sample
mirrors the larger population. How closely the
sample responses reflect what we would obtain
if we could sample the entire population.
• There are 2 sampling approaches, probability
sampling and nonprobability sampling
Probability sampling
• Probability sampling- selecting subjects in such a way
that the odds of their being in the study are known or can
be calculated. If target population is all undergraduate
students in city college, we could get a count from city
records, then we would know the odds of any one
person being in the study.
• Researcher must use an unbiased method of choosing
the participants, such as flipping a count, taking names
out of a hat, using a table of random numbers. This is
called random selection.
What are two advantages of probability sampling
over nonprobability sampling?
1. A probability sample is more likely to represent
the population (external validity) than a
nonprobability sample.
2. We know the exact odds of members of the
population being included in our sample. This
tells us whom the sample represents.
Probability Sampling
Which are the main probability sampling methods?
The four main probability sampling methods are:
 simple random sampling – a portion of the whole
sample is selected in a random way (close your eyes and pick out 20
names from all the names if need 20 people in your study)
 systematic random sampling – list all the people in
the population in unbiased way and take every nth one

Probability Sampling
Probability sampling
 stratified random sampling- randomly sampling
from people in each subgroup in the same proportions as they exist
in the population. If school is made up of 70% female and 30 %
male, then want your sample to also be 70% female and 30% male.
 cluster sampling – sample entire clusters or naturally
occurring groups that exist in the population. Randomly select from
clusters that already exist, such as zip codes and survey everyone
in that zip code. Only works if the cluster is similar to the rest of the
population. Used when the population is very large.
Nonprobability sampling
• Probability sampling is best, but can’t
always be used. Most studies are actually
done using nonprobability sampling. Here
subjects are not chosen at random.
Which are the main nonprobability sampling
methods?
The four main nonprobability sampling methods include:
 quota sampling- Researcher has quotas she must fill for her
data, needs 50 white and black men to answer questions. Doesn’t
matter how they are selected, as long as you fill quota. Goes to Iona
College and gets first 50 white and black men that she sees to fill ut
her surveys.
 convenience sampling – using groups that happen to be
available, your class, your choir group. Is commonly used. Aslo
called accidental sampling.

Nonprobability Sampling
Nonprobability sampling
 purposive sampling – selecting a sample who are
needed for the purpose of a study. If purpose of study is to compare
freshman to seniors on psychology comprehension, then this is
purposive sampling.
 snowball sampling- researcher locates a few people
who fit the sample criterion and asks them to locate additional
individuals. If want to sample men who are sports enthusiasts, then
you locate a few, an ask them to give the survey to some of their
friends who are also sports enthusiasts.