Transcript Week Three Readings

Week Three Readings: Context
and Response Order Effects
BA531
Hongyan Shi and Adam Branson
January 31, 2006
Cognitive Models: Answering
Questions Processes, Context, &
Response Order Effects On Attitude
Measurement
SBS, Tourangeau et. al., & Krosnick et. al.
Survey
Survey Recap, Dissection, and Discussion
Additional Questions Related to Readings
Week Three Review
Chapter 4, 5, and 6
Tourangeau and Rasinski (1988)
Krosnick and Alwin (1987)
Chapter Four and Tourangeau
 Context Effects on Attitude Measurement at
each stage of the Cognitive Model
Conditions at each stage in the response model
Variables
content, mood
reciprocity and norms
anchoring measurement, extremes, intervals,
perspective and frequency effects
rank orders and rating scales
social desirability, self-presentation, consistency
survey format and mode
Chapter Five and Tourangeau
 Substantive Information being brought to mind by preceding
questions?
 Inclusion/Exclusion Model
 Assimilation and Carry Over
 size of assimilation, expertise, # of preceding questions
 Contrast and Backfire
 sub-traction based and comparison based effects
 What triggers these effects? Any variables that affect the
categorization of information.
 attitude stability and survey mode
 Six Implications (Variables) for Questionnaire Construction
 Preceding Content, # of Preceding Questions, Target Generality,
Related Question Spacing, Introductions to Block or Non-Block of
Questions, Questionnaire Layout
 Attitudes: cognitive, affective, or conative or evaluative judgments
 What are attitudes? Implications of File Drawer & Construal Model
 Over Time, Strength, Accessibility (short-term or long-term memory)
Chapter Six and Krosnick
 Order Effects Within A Question
 primacy and recency effects
 open response and forced choice (both large and small)
 factual and judgment questions
 Response Order Effect Causes
 satisficing, memory limitations, option order, earlier item content
 survey mode, time constraints, persuasive opinions, distractions
 Cognitive Elaboration
 serial position, presentation mode, and item plausibility
 Judgmental Contrast Effects
 dimensional and nondimensional, asymmetric, response
continuums
Sample Survey
 Overview and Instructions
 How do you see material from chapters four,
five, six and the outside readings incorporated
into this survey?
 How does this survey provide context effects?
Assimilation? Contrast? Response Order?
 What are particular items you could see affecting
judgments of general audience pools?
Survey Questions
 1. People may associate foods with timely cultural, religious, and
political festivals. Likewise, certain festivals may be associated with
any numbers of foods. For example, people may celebrate New
Years Eve by consuming Champagne or other sparkling wines. Can
you think of any occasions associating consumable items and
festivals? 9 Yes/ 0 No
 2. Which of the following foods do you most closely associate with
Christmas? Please select one box.
2a. Response Options:
1 _Turkey 1 _Ham _Hamburgers 1 _Lobster
2b. Response Options:
1_Ice Cream _Pumpkin Pie 2 _Candy Cane 2 _Chocolate
Survey Questions
 3. Which of the following foods do you most closely
associate with Thanksgiving? Please select one box.
 3a. Response Options
_Turkey 3 _Ham _Hamburgers _Lobster
 3b. Response Options:
1 Ice Cream 4 Pumpkin Pie _Candy Cane Chocolate
Survey Questions
 4. Which of the following foods do you most closely
associate with the Fourth of July? Please select one box.
 4a. Response Options
_Turkey _Ham 3 _Hamburgers _Lobster
 4b. Response Options:
2 Ice Cream _Pumpkin Pie 3 Candy Cane Chocolate
Survey Questions
 5. Which of the following foods do you most closely
associate with St. Valentine’s Day? Please select one
box.
 5a. Response Options
_Turkey _Ham 1 _Hamburgers 2 _Lobster
 5b. Response Options:
_Ice Cream _Pumpkin Pie_Candy Cane 6 Chocolate
Survey Questions
 6. Please think of a food item from your own culture.
Write, in the following space, the name of this
food._apple pie, foie gras, doner, halwa, sweet bean
soup, dumpling, pizza, ham stuffed bread, grits
 7. Is the food you wrote down in the prior question
associated with a festival from your own culture?
4 Yes/ 5 No
* If yes, please share the name of the
festival._Christmas (2), New Year, Diwali
Survey Questions
 8. Is the food you wrote down in the previous question
similar to the types of foods mentioned in this survey?
I.e. Is it sweet? Is it a side-dish? Is it a meat dish?
5 Yes/ 3 No
Survey Questions
 9. On a five-point scale, please mark the corresponding
box of how much you like this food.
 9a. Response Option
One-Really Like
Four-5
Two-4
3
Two-2
1 Really Dislike
Survey Questions
 10. On a five-point scale, please mark the corresponding
box of how much you enjoy Chocolate Chip Cookies.
 10b. Response Option
Really Dislike
One-Moderately Dislike
One-Neutral
Three-Like Moderately
Three-Really Like
Survey Questions
 11. Check one. In answering how much you enjoy
Chocolate Chip Cookies, is the cookie number value
based on a comparison with:
__ Other Cookies
_2 Other Desserts
_1 Your Written Item
_5 All Foods
Survey Dissection
 Is Q1 general enough? Does the example serve a
purpose? The repetitive word choice? What about the
word choice for beverages, festivals, and “consumable”?
 Are the different sets of Q2-Q5 useful? Should visual
cues of the answer choices be provided? How would
respondents react to a survey presented orally? Should
response order options employ the “Latin Square”?
What do you think about when you read the questions?
Because these may not be salient events, do you
“satisfice”? Were you lead by preceding questions? As
you went on to the next question, did you want to go
back and change any answers? Your attitudes?
Survey Dissection
 Did Q6 lead you to believe that you should enter
something that was primed? What would a
respondent feel about going from closed
questions to open response?
 For Q6 and Q7, how would respondent and
interviewer rapport influence the survey? What
would be the various influences on the answer
development processes?
 Were you lead to thinking about a similar food
category in Q8 as mentioned in Q2-Q5? Is this
an indication of assimilation/carryover or did it
indicate contrast/backfire effects?
Survey Dissection
 In Q9, what was your basis for a comparison? Was your
feeling and attitude about the item strong? Neutral?
 Did it matter whether you began with really dislike and
assigned a value of 1? Or, if you had a choice of really
like and a value of 5? Should word options be the
ranking scale or numbers? Should the range be
broader? How did you anchor the item and compare it
with something neutral, deplorable, favorable?
 Did you compare it to items previously mentioned in the
survey during questions Q2 – Q5?
Survey Dissection
 Did having a switch from an item you wrote
down to a seemingly innocuous item in Q10
create any anxiety?
 Did you start guessing what might be next in the
questionnaire? Were you comparing any
festivals at this point to the target item?
 Did having a reward for completing the survey
bias your responses?
 In Q11, were you comparing your rankings to
other items not provided as a choice? Did
having a closed set of options limit or force you
to make comparisons you did not want to make?
Week Three
General Questions
Questions on Assumptions
Questions on the Inclusion/Exclusion
Model
Chapter Four & Tourangeau General
Questions
 Do we agree human judgment is always context dependant? (SBS
81)
 What would be necessary to answer differently?
 What problems do you see with the model assumptions?
 What can researchers be doing to ensure intentional priming
materials are used by respondents? Does it matter?
 If individuals seek consistency during surveys, what does this imply
for group or organizational level surveys? Will we see a normal
probability distribution function? Or, is it group consistent?
 Are there occasions when greater familiarity or expertise with
subject matter does not result in greater consistency over time?
When are we not prone to context effects?
 What are the impacts of “textbook” and “sensory experienced”
knowledge on attitudes and judgments?
Chapter Five General Questions
 Why are context effects not stable or replicable? What
does this mean for the surveying discipline?
 What explains when preceding items do not result in
assimilation or contrast effects? Can we determine this?
 What reasoning best explains why expertise results in
mixed results of context effects?
 What are respondents answering when they provide
information on their attitudes?
 Does the SBS pg. 109 flow chart present a complete
portrayal of how effects emerge?
 What are the strengths and weakness of the construal
model?
 What other variables may elicit inclusion and exclusion?
 How can we isolate these context effects?
Inclusion/Exclusion Model Assumptions
 Dynamic nature of knowledge representation.
Or, structures in long term memory?
 Representation of target stimulus and the
representation of the standard are, in part,
context-dependent…. ?
 Respondents who are highly knowledgeable
about an issue should be less susceptible to
context effects than those who are less
knowledgeable…. ?
Chapter Six & Krosnick General
Questions
 Do we agree it is possible to predict response order effects? Is it
possible to mitigate or isolate response order effects?
 How would primacy or recency effects appear in open-ended
questions?
 Do primacy effects arise only because of respondent personality and
their interaction with the survey instrument?
 Is it more likely that primacy and recency effects can cancel each
other out across large population samples? Or, will we have
distortions?
 Given this information, why is it or is it not possible to predict and
mitigate context effects?
 What would we see in the Krosnick study if the examination was
based on an aural/oral survey?
 What is the value in increasing respondent motivation to consider all
possible options? Is there a trade-off? Do we initiate other context
effects?
SBS Ch. Six Assumptions and Questions
 Memory limitations are not the primary source of
response order effects?
 Limitations are more likely to result in recency
effects for complex questions?
 Should we consider the complexity of each
alternative? 134, 248, 100, 752, 553
 Interaction of serial position, presentation mode,
and plausibility/persuasiveness?
 Do you concur response order effects are less
pronounced when a formed judgment can be
recalled from memory?
Other Topics/Interesting Items
What are your other thoughts and
comments regarding the improper and
proper use of techniques employed in this
survey and those techniques discussed in
the readings for this week?
Week Three-Conclusion
Cognitive Models: Answering Questions
Processes, Context, & Response Order
Effects on Judgments and Attitudes
SBS, Tourangeau et. al., & Krosnick et. al.
Additional Questions Related to Readings
Survey
Survey Recap, Dissection, and Discussion
On To Autobiographical Memory
Thank you!