Transcript No Slide Title - University of Idaho

JAMM 444: Public Opinion
Survey Methodology
& Question Wording
Feb. 12, 2008
Tuesday, Feb. 12






Quiz 2
Schedule update
Research paper topic memo
Democratic, Republican contests
Survey methods & question wording
End of class: Return news clipping
assignment
Quiz 2



1. What term is given to governors, senators,
and other “party bigwigs” who have voting
privileges at the Democratic national
convention?
2. Which Republican candidate pulled out of the
race after Super Tuesday?
3. What might explain why Barack Obama
received more support in primaries in Alabama
and Georgia than pre-election polls indicated?
Quiz 2



4. Why might a potential candidate for public
office commission a poll before deciding
whether to run? (Give one reason.)
5. In Travis Ridout’s opinion, which of the
two remaining Democratic candidates has
the best chance of winning the nomination?
Why?
BONUS: What color tie did Travis wear
during his guest lecture on Feb. 7?
Schedule update

TODAY: Chapter 3, Question Wording


Thursday: Chapter 4: Sampling


Turn in extra-credit papers on Idaho
caucus
Guest speaker: Chris Williams,
Department of Statistics
Feb. 19: Interviewing & Data Collection

Research topic memo due (new date)
Research Topic Memo


Due Tuesday, Feb. 19
See Web site:
http://www.class.uidaho.edu/jamm444/assignments.htm
Possible Research Topics
1.
2.
3.
4.
5.
Practice (methodology)
Issue or controversy
Person (historical figure)
Presidential candidate
Primary or caucus (one state)
Elements of memo
1.
2.
3.
Topic, why you have chosen it, and its
importance to public opinion.
At least three research questions
(things you hope to discover).
At least five sources:
1.
2.
One must be from a scholarly journal,
such as Public Opinion Quarterly
No more than one Web site
Superdelegates

Delegates drawn from the Democratic
National Committee, members of
Congress, governors and distinguished
party leaders, such as former
presidents, vice presidents, and
congressional leaders.

Former House Speaker Tom Foley is a
superdelegate from Washington state.
Why superdelegates matter

Current tally of superdelegates (CNN)




Clinton – 224
Obama – 135
Neutral/undecided/no public preference – 437*
TOTAL: 796*

*This number may change over the next several
months as people die, leave office or leave the
Democratic Party.
The race for the nomination

Democrats




Clinton
Obama
Edwards
Needed for nomination:
1,148
1,121
26
2,025
The race for the nomination

Republicans





McCain
783
Romney
286
Huckabee
217
Paul
16
Needed for nomination
1,191
Brokered convention


A situation in U.S. politics in which no candidate
obtains a majority of delegates during the primary
and caucus process. Because no candidate will
receive enough votes on the first ballot to win the
nomination, the convention is brokered through
political horse-trading and/or multiple ballots.
Last brokered conventions:


1948: Republicans (Thomas Dewey)
1952: Democrats (Adlai Stevenson)
Survey methodology
Why do a survey?
 Measure opinion of a population too
large to contact individually
 Save time, save money

“Surveys can be used in a scientific way
to realize the great benefits of
interviewing a representative sample
instead of the whole population.”

Salant & Dillman, 1994
Surveys work, if...




Sample is large enough to yield
desired precision.
Everyone in population has equal
chance of being selected.
Questions enable respondents to give
accurate answers.
Sampled respondents have similar
characteristics to non-respondents.
5 key elements of a survey
1.
2.
3.
4.
5.
Identify population
Choose type of sample
Choose method(s) for gathering data
Write (and test) questionnaire
Analyze the results
Choosing a sample

Identify population of interest:




University of Idaho students
Moscow residents
Idaho voters
‘The American public’
Choosing a sample

Non-probability sampling


1936, Literary Digest
Probability sampling




Simple random sampling (rarely used)
Systematic random samples (i.e., every 30th
name in student directory)
Stratified sample: divided into groups (strata)
Cluster sampling (spread over a wide area)
Cluster sampling

Used for many national surveys:



National Election Survey (NES)
General Social Survey (GSS)
Gallup Organization



350 geographical segments
Sample to reflect U.S. as a whole
(geographic area, size of community)
Random telephone numbers in each area
How big is a typical sample?



UI or Moscow: 200 to 500
Idaho: 500 to 800
National: 1,000 to 1,500
For Thursday

Read: P&P 4: Sampling Techniques



Non-probability vs. probability samples
Sample size
Sampling error
Chapter 3: Question wording

1. How can a sponsor with an ax to
grind manipulate wording of
questions?

What types of problems with question
wording are more subtle but just as
harmful to the accuracy of responses?
Chapter 3: Question wording
2. Describe the potential problems with the
following types of questions:
a) Compound (multi-topic) questions
b) Factual questions
c) Double-negative questions
d) Leading (argumentative) questions
e) Branching (follow-up questions)
f) Use of multiple items or indexes
Chapter 3: Question wording
3. The book gives several questions of how
alternative wording of questions influenced
responses. Discuss the examples
presented in these cases:
a) Impeachment of Bill Clinton
b) Use of government wiretaps to fight terrorism
c) The National Security Agency’s collection of telephone
records of U.S. citizens
Impeachment of Bill Clinton
Q: If the full House votes to send
impeachment articles to the Senate for a
trial, then do you think it would be better for
the country if Bill Clinton resigned from
office, or not?
Q: If the full House votes to impeach Bill
Clinton, then do you think it would be better
for the country if Bill Clinton resigned, or
not?
Use of government wiretaps
Q: After 9/11, President Bush authorized government
wiretaps on some phone calls in the U.S. without
getting court warrants, saying this was necessary to
reduce the threat of terrorism. Do you approve or
disapprove?
Q: After 9/11, President Bush authorized government
wiretaps on some phone calls in the U.S. without
getting court warrants. Do you approve or
disapprove of George W. Bush doing this?
NSA collection of phone records
Q: It’s been reported that the NSA has been collecting
phone records of millions of Americans. It then
analyzes calling patterns in an effort to identify
possible terrorist suspects… Would you consider
this an acceptable way for the federal government
to investigate terrorism?
Q: Based on what you have heard about the program
to collect phone records, would you say you
approve or disapprove of this government program?
Chapter 3: Question wording

4. How can the order in which
questions are asked affect responses?
Give an example.



Context effect: When the preceding question
influences a response
Q: Would you say that traffic contributes more or
less to air pollution than industry?
Q: Would you say that industry contributes more
or less to air pollution than traffic?
Tips for better question wording

1. Include filter questions



to exclude some respondents
based on interest or knowledge of issue
2. Use a neutral tone


Unbiased phrasing of controversial topics
Avoid words that lead respondent to
preferred answer (pseudo-surveys)

Examples: Bill Sali constituent surveys
Question wording

3. Provide middle alternatives



Offer a position between extremes
Likert scale: 5 options, from ‘strongly
agree’ to ‘neutral’ to ‘strongly disagree’
4. Be aware of context effect:



preceding questions affect responses
hard knowledge vs. attitude
general vs. specific
Question wording

5. Note the order of alternatives
(especially in phone surveys)



Primacy (first asked)
Recency (last asked)
Solution: rotate the alternatives.
Reading for Thursday

P&P 4: Sampling Techniques




Non-probability vs. probability samples
Sample size
Sampling error
Pick up news clipping assignments