Transcript Laboratory Experiments in the Social Sciences: An Introduction

LABORATORY
EXPERIMENTS IN THE
SOCIAL SCIENCES:
AN INTRODUCTION
Stephen
Benard
Depar tment of
Sociology
Indiana
University
OVERVIEW
A sample question
What is an experiment?
Basics of experimental design
What can we learn from experiments?
Ethics of experiments
AN (ENCOURAGING) DISCLAIMER
Just a small sample of the:
 Questions
 Experimental designs
 Independent and dependent variables
Many, many possibilities
SAMPLE QUESTION: WHAT PREDICTS
HELPING IN AN EMERGENCY?
If we notice someone in need:
 Are we more likely to help when alone, or in the
presence of others?
 “Diffusion of responsibility”
ARE PEOPLE LESS LIKELY TO HELP
OTHERS WHEN IN A GROUP?
Challenging to study through observation
 Emergency events are rare and hard to predict
 May vary in countless ways
Many alternative explanations
 People in groups less likely to notice
 More groups at busier times of the day – less time
 Unhelpful people more likely to travel in groups
STUDYING HELPING IN AN EXPERIMENT
 Would be useful to repeatedly observe responses
to the same emergency under different condition
 E.g., when many or few people observe the
emergency
 Could be staged in a laboratory
 (Darley and Latane 1968 in JPSP)
 Laboratory discussion group
 One person appears to have a seizure
 Manipulate number of people present
 Measure proportion who helped, speed
A FEW MORE EXAMPLES
 Does violent media make people aggressive, or do aggressive
people prefer violent media (Bandura, Ross, and Ross 1961)?
 Does intergroup contact reduce or exacerbate intergroup conflict
(Sherif 1958)?
 Does positive mood make people more altruistic, or are more
altruistic people happier ( Isen et al 1978)?
 Do our attitudes determine our behavior, or does our behavior
determine our attitudes (Festinger and Carlsmith 1959)?
 Does the gender/race/age/criminal record/other characteristic
of a job applicant affect the likelihood of being hired (e.g., Pager
2003)?
 Is support for a policy determined by the content of the policy, or
the identity of the party supporting it (Cohen 2003)?
 Does lack of control over our environment turn us into conspiracy
theorists (Whitson and Galinsky 2008)?
 Does the status of an author’s institution affect their chances of
having an article accepted (Peters and Ceci 1982)?
WHY CONDUCT AN EXPERIMENT?
Identifying causes
Addressing alternative explanations
Identifying moderators and mediators
Examining hard-to-observe or rare events
WHAT IS AN EXPERIMENT?
THREE PRINCIPLES
Manipulation of the independent variable
Random assignment to condition
Controlled measurement
MANIPULATION
In experiments you must manipulate an
independent variable (IV)
This creates 2 (or more) levels of the IV
The levels of the IV are called conditions
Conditions identical except for the
manipulated IV
E.g., number of people present when an
emergency occurs
RANDOM ASSIGNMENT
How do we distinguish the effects of our IV
from extraneous variables?
Perhaps personal interest in helping others
confounded with group size
Experimenter places people into experimental
conditions by chance
Equal likelihood of being in each condition
Individual differences cancel out
RANDOM ASSIGNMENT
Colors symbolize
any differentiating
attribute among
the individuals
(e.g., personal
interest in helping
others)
Before Random
Assignment
R
After Random
Assignment
Small
crowd
Large
Experimental Groups crowd
WHAT IF PEOPLE CHOSE THEIR
CONDITION?
Colors symbolize
any differentiating
attribute among
the individuals
(e.g., personal
interest in helping
others)
Before choosing
C
Systematic
error
Small
crowd
Self-selected Groups
Large
crowd
CONTROLLED MEASUREMENT
Systematically observe changes in the
dependent variable as a function of changes
in the independent variable
Important to avoid bias in recording the DM
Participant blind to hypotheses
Experimenter blind to hypotheses
Experimenter blind to condition
A SIMPLIFIED HELPING STUDY
( B A S E D O N DA R L E Y A N D L ATA N E 1 9 6 8 )
Experimental setting: a laboratory discussion
group
Simulate an emergency (seizure)
Manipulate number of other people present in
group
 E.g., zero vs. three
Randomly assign participants to the “alone”
condition or the “group” condition
Measure proportion helping, time to help
EXPERIMENTAL DESIGN
Two condition, treatment-control design
 Similar to medical study with placebo
 Simplest possible design
 Often very effective, but also limited
 Additional treatment conditions
 Factorial designs
ADDITIONAL TREATMENT CONDITIONS
Perhaps group size has a non-linear effect
 Add additional condition with 6 total group members
 Sometimes it is useful to have a “baseline”
condition
 E.g., a study of whether a text is evaluated more
positively when the author is a man vs. a woman
 May wish to compare to a condition with no author
information
 Is it that men receive a boost relative to the baseline,
or women receive a penalty?
FACTORIAL DESIGNS
Multiple IVs
Every combination
of every level of IV
Interaction effects
 Predict an interaction
 Or evaluate generality
Similarity
to
Victim
Similar
Not
Group
Large
No help
No
help
Size
Small
Help
No
help
A 2 x 2 Factorial Design
BETWEEN VS. WITHIN-SUBJECTS
DESIGNS
Between-subjects design: Each participant is
exposed to one level of the independent
variable
E.g., study of helping
BETWEEN VS. WITHIN-SUBJECTS
DESIGNS
Within-subjects design: Each participant
exposed to multiple levels of the dependent
variable
 E.g. Text evaluation study
More efficient
But possibly easier to guess hypotheses
Requires counterbalancing
Rarely possible in high-impact designs
INDEPENDENT VARIABLES
 How do we know operational independent variable
accurately measures theoretical dependent variable?
 E.g., positive mood
 How do we know the manipulation had the expected
effect on participants?
 Manipulation Checks
DEPENDENT VARIABLES
(AND SOME BROAD GENERALIZATIONS)
Examples
Difficulty/
Cost
Sensitivity to Participant
Social
Engagement
Desirability
Useful
for/When
Verbal
Reported
attitudes &
emotions,
vignettes
Low
High
Low
Goal is
measure
internal state
Behavioral
Help victim,
donate to
charity
High
Low
High
Goal is predict
behavior
Behavioroid
Choose partner, Moderate
agree to
something
Moderate
High (predecision)
Commitment
of more
interest than
behavior
Physiological
fMRI, cortisol,
heart rate
Very Low
High (but
possible
discomfort)
Biological
mediators/
moderators
Very High
WHAT CAN WE LEARN FROM
EXPERIMENTS?
 High degree of control provides high internal validity
 Experiments provide the strongest possible evidence
for causality
 But, external validity of laboratory experiments is
often criticized
 Settings don’t always resemble “real world”
 Participants don’t resemble other populations
 Samples are generally non-random
 Small samples, at least by survey data standards
 Participants are often college undergraduates
 Participants are often WEIRD: Western, educated,
industrialized, rich, democratic
MUNDANE VS EXPERIMENTAL REALISM
Mundane Realism: the extent to which an
experiment looks like the real world
Experimental realism: the extent to which
experience is psychologically real and
important to participants
Rarely come to a lab for a group discussion
GENERALIZING FROM…?
 Should not generalize
directly from an
experiment to a real
world situation
 Experiments test
theories
 Theory bridges
empirical studies and
the real world
 See Zelditch, 1969, “Can
you really study an army
in the laboratory”
Theory
•Provide
evidence for
or against a
theory
Experiment
•Provide
explanation
for real world
phenomena
•Complex
phenomena
to be
explained
“Real World”
SCOPE CONDITIONS
 Criticism that findings
won’t generalize
 Often explicitly or
implicitly signal possible
scope conditions
 These can be tested to
further refine the
theory
 Example: College
students discriminate
against women in hiring
simulation
 Maybe more than real
managers: less
experience
 Maybe less than real
managers: more
egalitarian
CONVERGENT VALIDIT Y
Useful to think of different methods as
complementary, not competing
Survey data
 May have high external validity, but limited ability to
show causality
 Experiments
 High internal validity, but limited generality
EXPERIMENTAL ETHICS:
Three core principles for all research
Respect
Beneficience
Justice
Deception
Necessary to test some hypotheses
But should be used only as a last resort
And fully explained to participants
Debriefing
SUMMARY
 Experiments are excellent for answering
questions about causality, exploring alternative
explanations, and examining rare or hard to
observe events
 Many different types and approaches to
experiments, can (must) be tailored to the
research question
 Facilitate systematic replication and theory
development
 Strengths/weaknesses complement other
methods
THANK YOU!
Stephen
Benard
Depar tment
of Sociology
Indiana
University