Transcript Design

Design
Experimental Control
Experimental control allows causal inference
(IV caused observed change in DV)
Experiment has internal validity when it
fulfills 3 conditions for causal inference
1) covariation
2) time-order relationship
3) elimination of plausible alternatives
Specify variables to be controlled
• Controlling extraneous variables
•
•
•
•
1) elimination
2) holding conditions constant
3) randomization/balancing
4) counterbalance
1) Elimination
If possible eliminate the extraneous variable
Eg noise
a) As a confound; group A measured during
high traffic Group B low traffic noises
b) Nuisance variable (may not be a
confound). Random noises from heating
system.
2) Hold conditions constant
Minimize variability
• Time of day
• Lighting
• Instructions
• Stimuli
• Procedure….
Loftus and Burns ( 1982)
• Two groups both saw a film of a bank robbery. Only the
ending differed.
• Group A violent ending
• Group B nonviolent
• Both groups asked questions about events that
happened prior to end scenes
• Eg the number on a t-shirt worn by a bystander
• Correct recall group A 4% Group B 28%
• Same film, same instructions, same questions, same
room…
• Did not control same temperature or weather…
• Only factors thought to impact DV
3) Randomization/Balance
• Especially useful if unsure what
extraneous variables may be operating
Between Subjects Design
only choice if
• a) subject variable eg smoker and nonsmoker
• b) if manipulation of IV makes repeats
impossible or undesirable (deception or
carryover effects)
the number of groups = the number of levels
of IV
disadvantages:
• many subjects needed
• individual variation and selection effects
statistical tests
• compare variability between groups to variability
within groups
sources of variability are
• a) the IV
• b) confounds –systematic
• c) error – unsystematic (individual variability)
Design problems
• The equivalent group
Equivalent Groups
• - try to compensate for selection effect
• - groups are equal to each other in
important ways
• - the number of groups = the number of
levels of IV
Random Assignment
a)Every participant has equal chance of being in
each group, the individual variation is spread
through the groups evenly
this works well with big N
b) Block Randomization
use random number table to assign order
if have 5 groups then use numbers 1-5
list the numbers in the order they appear – must finish
sequence before repeating a number
c) Matching
if small N then a few individuals assigned by chance can
have a big impact
test participants on a variable and pair scores – each group
gets similar scores
•
-you need a priori reason to match on a variable
•
-it adds logistical complexity
•
-may give away hypothesis ( bias and reactivity
problem)
Example
weights
156 167 183 170 145
143 152 145 181 162
175 159 169 174 161
order
143 145 145 152 156 159 161 162 167 169 170 174 175 181 183
Matching
Group 1
Group 2
Group 3
145
143
145
152
159
156
161
167
162
169
170
174
181
175
183
161.8
162.8
164
Block randomization
Group 1
Group 2
Group 3
143
156
175
167
159
152
183
169
145
170
181
174
161
162
145
164.8
165.4
158.2
156 167 183 170 145
143 152 145 181 162
175 159 169 174 161
21113
13332
32221
Balancing
• Cannot control characteristics of
participants.
• Try to evenly spread the individual
differences between the levels of IV
• Random assignment
• Eg if in the Loftus and Burns study groups
differed in attention or memory then
problem
Between Subjects
Design problems
• The equivalent group
• Solution – randomize or balance
Within Subjects Design
(repeated measures)
• Each participant exposed to each level of
the IV
• Fewer people needed (economical)
• Individual variability removed as source of
error (more power in testing)
Great for rare events/species/diseases
BUT sequence or order effects can be problematic
Progressive effects
Practice improves performance
Fatigue worsens performance
Carryover effects
Doing task A has bigger impact on task B than
the reverse
Uneven impact
Within Subjects
Design problem
• Sequence effects
4) Counterbalance
a) complete counterbalancing – use all
possible sequences of orders at least once
good if few conditions (3 or less) (n!
possible)
3 groups gives ? possible combinations
4 groups ? possible….
4) Counterbalance
a) complete counterbalancing – use all
possible sequences of orders at least once
good if few conditions (3 or less) (n!
possible)
3 groups gives 6 possible combinations
4 groups 24 possible….
b) partial counterbalancing
- take random sample of all possible
sequences , reduces systematic bias
c) Latin squares
every condition appears equally often in
every sequential position
- if balanced Latin square then each
condition precedes and follows every other
once
Latin Squares
order
participant
1
2
3
4
1
1
2
3
4
2
2
3
4
1
3
3
4
4
Latin Squares
order
participant
1
2
3
4
1
1
2
3
4
2
2
3
4
1
3
3
4
4
4
Latin Squares
order
participant
1
2
3
4
1
1
2
3
4
2
2
3
4
1
3
3
4
1
2
4
4
1
Latin Squares
order
participant
1
2
3
4
1
1
2
3
4
2
2
3
4
1
3
3
4
1
2
4
4
1
2
3
Balanced square
Rule is first row 1,2,n, 3, n-1, 4,n-2 ,5….
Second row add one
order
participant
1
2
3
4
1
1
2
4
3
2
2
3
3
3
4
4
Balanced square
Rule is first row 1,2,n, 3, n-1, 4,n-2 ,5….
Second row add one
order
participant
1
2
3
4
1
1
2
4
3
2
2
3
1
3
3
4
4
Balanced square
Rule is first row 1,2,n, 3, n-1, 4,n-2 ,5….
Second row add one
order
participant
1
2
3
4
1
1
2
4
3
2
2
3
1
4
3
3
4
4
Balanced square
Rule is first row 1,2,n, 3, n-1, 4,n-2 ,5….
Second row add one
order
participant
1
2
3
4
1
1
2
4
3
2
2
3
3
3
4
4
4
1
1
2
3
4
1
2
Within Subjects
Design problem
• Sequence effects
• Solution - counterbalance
Experimental Control
Dependant Variable
• validity
• reliability
• multiple measures
Independent Variable
Vary in a systematic way
• Control confounds related to IV
Eliminate
Hold constant
Balance (groups)
Counterbalance (order)
Randomize
Plan for experimenter bias
Participant Effects
•
•
•
•
Random assignment
Pilot measures for social desirability
Consider floor/ceiling
Yes/no bias
Single group
A single group threat includes history,
maturation, testing, instrumentation,
mortality and regression to mean threats.
Multiple Groups
• These multiple group threats are called a
selection bias or selection threat.
• These include selection history, selection
maturation, selection testing, selection
instrumentation, selection mortality and
selection regression threats
Double pretest
The design includes two measures as denoted by two
"Os" prior to the program.
This design can rule out selection maturation threat
and a selection regression threat. It will help to make
sure that the two groups are comparable before the
treatment
Switching Replication Design
Good at solving the social threats to internal validity
compensatory rivalry,
compensatory equalization,
resentful demoralization.
Both groups get same program so no inequity
• control group – assumes extraneous
variables operate on both experimental
and control equally
• more than one control group can be used
to assess different variables
Before training
training
After
O
X
O
Single Group
Multiple Groups
Before training
training
After
experimental
O
X
O
control
O
O
Before training
Training
After
experimental
O
X
O
Control 1
O
Control2
O
O
Solomon 4 group design
Before training
Training
After
experimental
O
X
O
Control 1
O
Control2
Control3
O
X
O
O
testing threat
The design consists of four groups of randomly assigned.
Two of them receive the treatment as denoted by " X" and the other two do not.
Determine extraneous variables
Will not
influence
DV
Might influence DV
Cannot be
controlled
Can be controlled
ignore
Randomize
Continue
experiment
Continue
experiment
Continue
experiment
Cannot
randomize
Abandon
experiment