Transcript June 9

Survey Research. Sampling and
Inference
June 9, 2008
Ivan Katchanovski, Ph.D.
POL 242Y-Y
Survey Research
• Surveys: One of the main methods of data collection
and research in political science and sociology
• Polls: Used in mass media and business
• Advantages
• Reliable
• Involves relatively small numbers of respondents
• Relatively cheap and fast
– Compare with census
• Disadvantages
– Sampling error
– Validity
2
Example of Polls: Bush Approval
3
Observation and Sampling
• Polls and surveys are based on observations
• Population: universe of subjects the researcher
wants to sample
– Adult population in Canada
– Voters in Ontario
• Sample: cases or observations drawn from a
population
4
Random Sample
• Random selection: Each element has an equal chance
of selection independent of any other event in the
selection process
– Are internet polls random?
– Are telephone polls random?
• Weighting: Giving some cases in a survey dataset
more weight than others
– Weights examples
5
Parameters and Sample Statistics
• Parameters: Characteristics of a population
– Proportion of the vote for political parties in national or local
elections
– Median personal income in Canada
• Sample Statistics: Estimates of population parameters,
based on a sample drawn from a population
– Exit poll estimate of proportion of the vote for political
parties in national or local elections
– Survey estimate of median personal income in Canada
• Inference about the characteristics (parameters) of a
population based on samples
6
Confidence Level
• Surveys and polls: margin of error
• Random sampling error:
– Decreases with increase in sample size
• Confidence Level
– The estimated probability that a population parameter lies
within a given confidence interval
• Example: we might be 95% confident that between
48 and 52% of all voters favor Candidate A.
7
Election Eve Polls - U.S. Presidential
Candidates, 2004
Date
Begun
Agency
Bush
Kerry
10/29
Gallup/CNN/USA
49
51
10/29
NBC/WSJ
51
49
10/29
Harris
52
48
10/29
10/30
10/30
10/31
11/2
CBS
Fox/OpinDynamics
TIPP
GWU Battleground
Actual Vote
51
49
51
52
52
49
52
49
48
48
8
Types of Survey Questions
• Open-ended questions
– Respondent is asked to provide his or her own answer to
the question
• Closed-ended questions
– Respondent is asked to select an answer from among a list
provided by the researcher
• Example: different % correct responses in openended and closed-ended questions
9
Example: Two Types of Questions in a
2007 Pew Poll of Americans
Closed-Ended Question, %
Open-Ended Question %
Who is the President of
Russia? Is it…
(A) Vladimir Putin
60
(B) Boris Yeltsin
7
(C) Mikhail Gorbachev 7
(D) or is it someone else 8
Don’t know (Vol.)
18
Total
100
Can you tell me the name
of the president of Russia?
Yes, Vladimir Putin 36
Yes, other incorrect
3
No, don’t know
61
Total,
100
10
Contingency Questions and Biased
Questions
• Contingency questions: Questions which are
contingent on response to previous questions
• Example of a contingency question:
Q14.1a. When it comes to political matters, do you ever think of yourself
in terms of Left and Right?
(1) Yes (2) No (9) DK, etc.
(IF YES TO Q.14.1a, ASK Q.14.1b)
14.1b) How would you place your views on this scale, generally
speaking – where 1 means “very Left” and 10 means “very Right”?
• Biased questions: Questions that encourage
respondents to answer in a particular way
11
Response Rate
• Number of people participating in a survey
divided by the number selected in the sample
• Acceptable response rates
– 50% - adequate for analysis and reporting
– 60% - good
– 70% - very good
• Lower response rates
– Can be used if non-respondents are similar to
respondents
12