Transcript Chapter 8 Experiments

Topics Appropriate to Experiments
• Projects with limited and well-defined
concepts.
• Projects that are exploratory rather than
descriptive.
• Studies of small group interaction.
Components of Experiments
• Independent and dependent variables
• Pretesting and posttesting
• Experimental and control groups
– Stimulus and no stimulus
Experimental and
Control Groups
• Must be as similar as possible.
• Control or Comparison group represents what
the experimental group would have been like
had it not been exposed to the stimulus.
Experiment
Two & Half Men
• Short clip making fun of the experimental
design in testing drugs
Selecting Subjects
• Probability sampling
• Randomization
• Matching
Randomization and Matching
• May not know which variables will be relevant
for matching process.
• Most statistics used to analyze results assume
randomization.
• Randomization only makes sense if you have a
large pool of subjects.
Internal Validity
• Refers to the possibility that the conclusions
drawn from experimental results may not
accurately reflect what went on in the
experiment itself.
• Did something other than the experimental
stimulus affect the dependent variable?
Sources of Internal Invalidity
• Historical events may occur during the course
of the experiment.
• Maturation of the subjects.
• Testing and retesting can influence behavior.
• Instrumentation
Sources of Internal Invalidity
• Selection biases.
• Experimental mortality - subjects drop out of
the study before it's completed.
External Validity
• Refers to the possibility that conclusions
drawn from experimental results may not be
generalizable to the “real” world.
• Is the experimental setting unrealistic?
• Is pre-testing influencing the subjects (i.e.
cueing them in on what the researcher wants)
Limiting External Invalidity
Solomon Four Group Design - Four groups of
subjects, assigned randomly:
• Groups 1 and 2 are the control and
experimental group.
• Group 3 does not have the pre-test.
• Group 4 is only posttested.
Solomon Four-Group Design
– G1:Pretest-stimulus-posttest
– G2:Pretest-posttest
– G3:Stimulus-posttest
– G4:posttest
Solomon Four-Group Design
Example
• Champney and Edleman. 2010. “Assessing
Student Learning Outcomes in United States
Government Courses”
• Available on WebCampus (not required
reading).
Posttest-only Control Group Design
• Includes Groups 3 and 4 of the Solomon
design.
• With proper randomization, only these groups
are needed to control problems of internal
invalidity and the interaction between testing
and stimulus.
• Commonly done in the social sciences
Web-based Experiments
• Increasingly, researchers are using the World
Wide Web to conduct experiments.
• Because representative samples are not
essential in some experiments, researchers
use volunteers who respond to invitations
online.
Experimental Method
Strengths:
• Isolation of the experimental variable over
time.
• Experiments can be replicated several times
using different groups of subjects.
Experimental Method
Weaknesses:
• Artificiality of laboratory setting.
• Social processes that occur in a lab might not
occur in a more natural social setting.
Example of Experiments in Political Science
• Mendelberg, Tali. 1997. Executing Hortons:
Racial Crime in the 1988 Presidential
Campaign. The Public Opinion Quarterly 61(1,
Spring): 134-157.
• Available on WebCampus (not required
reading).
Methodology
•
•
•
•
Experiment
77 white students at U of M
Median age was 18
Treatment group shown the ad, control group
was not
• OLS regression (not a T-test)
Findings
• Students shown the Horton ad were more
likely to have negative views on race and racial
policies
• Students shown the Horton ad did not have
different views on crime
The End
• Read Levin and Fox Ch. 7
– Will go over homework assignment
• Read Matland, Richard E. 1994. “Putting
Scandinavian Equality to the Test: An Experimental Evaluation
of Gender Stereotyping of Political Candidates in a Sample of
Norwegian Voters.” British Journal of Political Science 24, 2:
273-92.
• Available on WebCampus