Transcript 702_8

SURVEY RESEARCH
Topics Appropriate to Survey
Research
• Descriptive
• Exploratory
• Explanatory
Guidelines for Asking Questions
•
•
•
•
Choose appropriate question forms.
Make items clear.
Avoid double-barreled questions.
Respondents must be competent to
answer.
Guidelines for Asking Questions
•
•
•
•
•
Respondents must be willing to answer.
Questions should be relevant.
Short items are best.
Avoid negative items.
Avoid biased items and terms.
Guidelines for Questionnaire
Construction
• One question per line.
• Use contingency questions when
necessary.
• Format matrix questions so they are easily
answered.
Guidelines for Questionnaire
Construction
• Be aware of issues with ordering items.
• Include instructions for the questionnaire.
• Pretest all or part of the questionnaire.
Question/Response Wording
•
•
•
•
•
•
Likert Scales
Thermometer ratings
Agree/Disagree statements.
Ordered responses.
Open responses.
Check all that apply.
Acceptable Response Rates
• 50% - adequate for analysis and reporting
• 60% - good
• 70% - very good
Guidelines for Survey
Interviewing
• Dress in a similar manner to the people
who will be interviewed.
• Study and become familiar with the
questionnaire.
• Follow question wording exactly.
• Record responses exactly.
• Probe for responses when necessary.
Telephone Surveys
Advantages:
• Money and time.
• Control over data collection.
Disadvantages:
• Surveys that are really ad campaigns.
• Answering machines.
Strengths of Survey Research
• Useful in describing the characteristics of
a large population.
• Make large samples feasible.
• Flexible - many questions can be asked on
a given topic.
Weaknesses of Survey
Research
• Can seldom deal with the context of social
life.
• Inflexible in some ways.
• Subject to artificiality.
• Weak on validity.
Survey Problems
• Reader’s Digest presidential survey of
1936; Alf Landon vs. FDR
• Hite Sexuality Survey – 70% of women
married 5 years or more are having sex
outside of marriage (4,500 completed
surveys out of 100,000).
• Recorded phone surveys.
• Push & Entertainment polling: political
telemarketing masquerading as a poll
Bush campaign in South Carolina asked the
following:
"John McCain calls the campaign finance
system corrupt, but as chairman of the
Senate Commerce Committee, he raises
money and travels on the private jets of
corporations with legislative proposals
before his committee. In view of this, are you
much more likely to vote for him, somewhat
more likely to vote for him, somewhat more
likely to vote against him or much more likely
to vote against him?"
Independent Sample T-test
Formula
t=
X1  X 2
s X1  X 2
 N1s1  N 2 s2
 
 N1  N 2  2
2
s x1  x2
2
 N1  N 2 


 N N 
 1 2 
Independent-samples t-tests
•
But what if you have more than two groups?
•
One suggestion: pairwise comparisons (t-tests)
Multiple independent-samples t-tests
# groups
2 groups
3 groups
4 groups
5 groups
...
10 groups
•
That’s a lot of tests!
# tests
= 1 t-test
= 3 t-tests
= 6 t-tests
= 10 t-tests
= 45 t-tests
Inflation of familywise error rate
• Familywise error rate – the probability of making
at least one Type I error (rejecting the Null
Hypothesis when the null is true)
• Every hypothesis test has a probability of making a
Type I error (a).
• For example, if two t-tests are each conducted
using a = .05, there is a .0975 probability of
committing at least one Type I error.
Inflation of familywise error rate
• The formula for familywise error rate:
1  1  a 
# groups
# tests
c
nominal alpha
familywise alpha
2 groups
1 t-test
.05
1  1  a   1  .95   .05
3 groups
3 t-tests
.05
1  1  a   1  .95   .14
4 groups
6 t-tests
.05
1  1  a   1  .95   .26
5 groups
10 t-tests
.05
1  1  a   1  .95   .40
45 t-tests
.05
1  1  a   1  .95   .90
c
c
c
c
1
3
6
10
...
10 groups
c
45
Analysis of Variance: Purpose
• Are there differences in the central
tendency (mean) of groups?
• Inferential: Could the observed differences
be due to chance?
Assumptions of ANOVA
•
Normality – scores should be normally distributed within
each group.
•
Homogeneity of variance – scores should have the same
variance within each group.
•
Independence of observations – observations are
randomly selected.
Logic of Analysis of Variance
• Null hypothesis (Ho): Population
means from different conditions are
equal
– m1 = m2 = m3 = m4
• Alternative hypothesis: H1
– Not all population means equal.
Logic of Analysis of Variance
• Null hypothesis (Ho): Population
means from different conditions are
equal
– m1 = m2 = m3 = m4
• Alternative hypothesis: H1
– Not all population means equal.
Lets visualize total amount of
variance in an experiment
Total Variance = Mean Square Total
Between Group Differences
(Mean Square Group)
Error Variance
(Individual Differences + Random Variance)
Mean Square Error
F ratio is a proportion of the MS group/MS Error.
The larger the group differences, the bigger the F
The larger the error variance, the smaller the F
Logic--cont.
• Create a measure of variability among
group means
– MSgroup
• Create a measure of variability within
groups
– MSerror
Example: Test Scores and
Attitudes on Statistics
Loves Statistics
Hates Statistics
Indifferent
9
4
8
7
7
4
6
5
5
11
6
6
12
3
7
•
•
Find the sum of squares between groups
SSbetween   N groupX
2
group
 Ntotal X
2
total
Find the sum of squares within groups
2
2
SS within   X total
  N group X group
•
Total sum of squares = sum of between
group and within group sums of squares.
SStotal   X
2
total
 Ntotal X
2
total
•
•
•
•
•
•
•
•
To find the mean squares: divide each
sum of squares by the degrees of
freedom (2 different dfs)
Degrees of freedom between groups =
k-1, where k = # of groups
Degrees of freedom within groups = n-k
MSbetween= SSbetween/dfbetween
MSwithin= SSwithin/dfwithin
F = MSbetween / MSwithin
Compare your F with the F in Table D