Transcript External validity

External validity
 Generalization based upon representation
 If you can’t see it you can’t talk about it.
EXTERNAL VALIDITY
FOUR ISSUES
GENERALIZATION
Setting
Sampling issues
subjects
Divisions
Operatonalizations
X
Manipulations
Y Measures
timING
X when given?,
Spacing
Y when
measured?
SAME AMOUNT
: HOW DOLED OUT?
Factorial
Quality-type
Right after
statistical
partitioning
Quantity-amount
much latter
GENERALIZATION
 One can only generalize to that which one
has examined.
 If ya haven’t looked it ya can’t talk about it!
 It “it” compares with what you’re examining
you can apply what you found.
 Note the “judgment” which must enter in here.
 Hence science is tentative.
Setting
 Are other variables combining with X which
influence the X-Y relationship so that there is
more than one relationship (statistical
interaction; context)?
 Can we / do we see that X acts differently on
Y in diffferent situations (i.e. in combination
with additional variables)?
Setting
 EX of pretest.
 R OXO
Classical design
R O O
 Vs.
R
R
XO
O
Posttest only design
Solomon 4 Design
 R OXO
RO O
 R XO
R
O
 Look at the XY effect when there is a pretest
vs one when there isn’t.
Factorial
 R OXO
 R ZXO
 RO O
 RZ O
 R XO
 R XO
 R
 R
O
 O and Z are Sam Ting
O
 Pretest is just example
of any second variable
Z
Factorial
 We can now see how X and Z interact.
Example
 Drug rehab program
 Negotiating in neutral territory vs. under
threat…
 Driving on dry vs. icy road.
Statistical partitioning
 Same as factorial without randomizing the Zs.
 Without randomizing Zs we don’t know what
other variables may be confounded with Z
and be the real causal culprit.
Subjects
 We’ve already considered generalization
when looking at sampling…We know our
odds of being wrong in rejecting no
relationship in a population on the basis of a
sample.
 also…whatever you want to study…make
certain that these are the types of people you
are studying!
Subjects
 Because of statistical interaction, we won’t
know how a given experiment generalizes
without specifically examining each “Z” that
can influence the X-Y relationship.
 This is an issue raised by experimental
psychologists that sociologists “know” but
don’t often pass on to their students.
Subjects
 …so for each type of subject to which we
want to generalize…we must examine the XY relationship within different categories of
the relevant subject characteristic.
 Note also…whatever you want to
study…make certain that these are the types
of people you are studying!
How?
 Factorial
 Statistical partitioning
 Note also…whatever you want to
study…make certain that these are the types
of people you are studying!
Operationalizations
 Manipulations …we “play god” and “decide”
what amount of X the subject will receive by
using random assignment…the subject
doesn’t decide the amount…so we’re doing
more than measuring the independent
variable.
Operationalizations
 We measure the effect of Y.
 X manipulates Y we don’t.
X
 Type
 Example






Aggression
Pain pill
Counseling
Tutoring
Diet program
Drug rehab program
X
 Amount




Total hours of counseling
Total hours of tutoring
Total hours of studying
Total amount of penicillin
Y
 Type
 (amount is measured but we don’t give
different amounts as we could with X)

Aggression


Hitting Bobo doll vs. person
Tutoring

One on one vs. group
timING
 We’re not talking about the “times”…that’s a setting
issue.
 We’re not talking about time…that’s a manipulation
issue.
 We’re talking about when we administer the total
amount of X…how we dole X out.
 & we’re talking about when we assess the effects of
X…do we measure Y right after X occurs or later after
it’s had time to have an effect or lose its effect?
Timing of X
 …all at once or in measured doses?
 E.g. tutoring, studying, counseling, exercising,
dieting, drugging…etc
Timing of Y
 Right after or later?
 Some effects occur immediately and then
fade…e.g.: diet programs, inspirational
speeches or sales pitches
 Others have a sleeper effect…they don't “kick
in” ‘till later…e.g.: your appreciation of this
great class,