Transcript Day 6- Research Design I

Research Designs
Agenda
Experimental Designs
Natural Experiments
Time Series and Panel Designs
Cross Sectional Designs

Surveys
Announcements
Next week- half of you meet in library
Tuesday, half come here, flip for Thursday
A through McGrath meet in Library
Tuesday, Here Thursday
Mickelson-Z meet here Tuesday, Library
Thursday
Don’t read Johnson 6, 11
Recap
Hypotheses
Variables
Units of Analysis
Positive, negative, spurious,
antecedent/intervening/mediating,
moderating
Intro to Iyengar and Kinder
Media and Public Opinion: Hypodermic
Needles to Minimal Effects to ??
How might you study media effects
Why might experiments be useful?
Experiment
Allows for the greatest level of researcher
control
Systematically vary one variable at a time
Control Group
Treatment Group
Stimulus/Treatment
Experiment
Assignment


Matching
Random
Within Subjects Design
Between Subjects Design
Classic Experiment
Randomly assign groups
Test at time 1
Provide treatment to one group
Provide no treatment to other group
Test at time 2
Both between and within subjects design
Iyengar and Kinder
Hypotheses- Multiple



Exposure influences agenda
Exposure influences accountability
Qualities of story matter
Recruit a random sample of citizens
Participants watch the news
Mild deception- watch with us to avoid
distractions
Iyengar and Kinder
Pretest
What are key IVs/DVs?
Show TV news


Control group- Unaltered News
Treatment group- Added Story
Post Test measures
Findings- Agenda Setting and Priming on
Most issues
Iyengar and Kinder
Interesting Features?
Create natural environment
Tight Control of Stimulus
Number of Conditions
No pre test
Is it desirable to pretest?
Between Subjects Test
Example- Druckman and Holmes 2005
Comparison between groups
Random Assignment makes comparison
valid
Quasi Experimental Approaches
Increased control over non experimental
techniques
Less control than true experiment
Sidanius and Pratto
Social Dominance Theory
Secret Shopper Paradigm
Send different races, genders to do various
tasks
Look at and bargain for apartment
Bargain for car
Apply for job
Controls
Results- White men do best, African-American
Men do worst
Natural Experiments
Two similarly matched entities
Differ in “one” key way
E.g. Mondak Study of Newspapers in
Cleveland and Pittsburgh
E.g. political coverage on different
networks
Social Programs
Concern- Lack of Control
Experimental Methods-Advantages
Allow the researcher control over the
situation
Allows for more certainty of measurement
Allow for stronger causal inferences- High
Internal Validity

Control of extraneous factors
Experimental MethodsDisadvantage
Limits to External Validity




Subject Population
Reality of Situation
Cannot create some circumstances
Demand Effects
Cannot manipulate all factors


Personality
Ethical Considerations
Somewhat weak relationship between attitudes
and behaviors?
Experiments- When Appropriate
When feasible
When ethical
When factors of interest are controllable
Public Opinion Change
Media Studies
Time Series
Look at cases at multiple times
Panel Study- Re-interview same cases at
set points
Presidential Approval



Track from 50s to present
Look at Approval rating of president as
Dependent Variable
Look at fluctuations in other factors
Time Series- Advantages
Causality- follows temporal patterns
Gives best insight into over time change
Panels- good for understanding change
Time Series- Disadvantages
Must have data over time
Consistency of measures over time
Causality- confusion?
Challenge- empirical generalization
Panels



Drop outs
Systematic bias in sample
Learning
Time Series- When Appropriate
When looking at change over time
When looking at historical events
When looking at development processes
When looking at factors that cannot be
easily manipulated in lab or do not vary
cross-sectionally
Cross Sectional
One time
Look at multiple cases
Cases should have variation
Example- National Election Study
Cross Sectional Studies- Survey
Random sample of population of interest
Ideal Sample greater than 500
Fixed questionnaire
Ask all at roughly same time
Ideal for public opinion/voting behavior
Cross Sectional DesignsAdvantages
Does not require as much advanced
planning
Good for comparing groups of things
Snapshot
Cross Sectional DesignDisadvantages
Does not work for certain types of
phenomena
Limited ability to make causal connections

Temporal issues
Limited number of cases
Case Study
Focus intense study on one case
Often qualitative
Challenge- generality
Surveys
More a tool than a design
Randomly select large number of people
Administer same questionnaire
May be cross sectional or time series
Allows rich data collection
Often most generalizable
Expensive
Not social
Inflexible
Focus Groups
4-10 people together
Have them discuss
Relatively unstructured
Advantage- study more social phenomena
Generate new ideas
Concern- generalizability
Interpretation