Transcript No Slide Title

Between Groups & Within-Groups
ANOVA
• BG & WG ANOVA
– Partitioning Variation
– “making” F
– “making” effect sizes
• Things that “influence” F
– Confounding
– Inflated within-condition variability
• Integrating “stats” & “methods”
ANOVA  ANalysis Of VAriance
Variance means “variation”
• Sum of Squares (SS) is the most common variation index
• SS stands for, “Sum of squared deviations between each of a
set of values and the mean of those values”
SS = ∑ (value – mean)2
So, Analysis Of Variance translates to “partitioning of SS”
In order to understand something about “how ANOVA works” we
need to understand how BG and WG ANOVAs partition the SS
differently and how F is constructed by each.
Variance partitioning for a BG design
Mean
Variation among all
the participants –
represents variation
due to “treatment
effects” and “individual
differences”
SSTotal =
Tx
C
20
30
10
30
10
20
20
20
15
25
Variation between the
conditions – represents
variation due to
“treatment effects”
SSEffect +
Called “error”
because we can’t
account for why
the folks in a
condition -- who
were all treated
the same – have
different scores.
Variation among
participants within
each condition –
represents “individual
differences”
SSError
How a BG
F is constructed
Mean Square is the SS converted to a “mean”  dividing it
by “the number of things”
SSTotal = SSEffect +
SSError
dfeffect = k - 1
represents #
conditions in design
F =
MSeffect
MSerror
=
SSeffect / dfeffect
SSerror / dferror
dferror = ∑n - k
represents #
participants in study
How a BG
r is constructed
r2 = effect / (effect + error)
 conceptual formula
=
SSeffect / ( SSeffect + SSerror )
 definitional formula
=
F / (F + dferror)
 computational forumla
F =
MSeffect
MSerror
=
SSeffect / dfeffect
SSerror / dferror
An Example …
i
p
O
S
S
m
S
u
d
F
S
a
i
e
N
e
a
v
B
4
1
4
2
7
1
0
W
0
8
0
2
0
T
4
9
T
0
SStotal
= SSeffect +
SSerror
1757.574 = 605.574 + 1152.000
r2 =
SSeffect
/
( SSeffect + SSerror )
= 605.574 / ( 605.574 + 1152.000 ) = .34
r2 =
=
F / (F + dferror)
9.462 / ( 9.462 + 18) = .34
Variance partitioning for a WG design
Mean
Tx
C
Sum
Dif
20
30
50
10
10
30
40
20
10
20
30
10
20
20
40
0
15
25
Variation among
participants – estimable
because “S” is a
composite score (sum)
SSTotal =
SSEffect
+
SSSubj
Called “error”
because we can’t
account for why folks
who were in the same
two conditions -- who
were all treated the
same two ways –
have different
difference scores.
Variation among
participant’s difference
scores – represents
“individual differences”
+
SSError
How a WG
F is constructed
Mean Square is the SS converted to a “mean”  dividing it
by “the number of things”
SSTotal = SSEffect + SSSubj + SSError
dfeffect = k - 1
F =
MSeffect
MSerror
=
SSeffect / dfeffect
represents #
conditions in design
SSerror / dferror
dferror = (k-1)*(n-1)
represents # data
points in study
How a WG
r is constructed
r2 = effect / (effect + error)
 conceptual formula
=
SSeffect / ( SSeffect + SSerror )
 definitional formula
=
F / (F + dferror)
 computational forumla
F =
MSeffect
MSerror
=
SSeffect / dfeffect
SSerror / dferror
An Example …
Don’t ever do this with
e
N
e
a
real data !!!!!!
S
4
0
0
e
2
0
0
S
Tests of W ithin-Subjects Effects
Measure: MEASURE_1
Source
factor1
Error(factor1)
Sphericity Assumed
Sphericity Assumed
Type III Sum
of Squares
605.574
281.676
df
1
9
Mean Square
605.574
31.297
n
F
19.349
Sig.
.002
-
M
T
I
I
q
d
F
S
i
u
S
g
f
I
1
1
1
4
0
E
5
9
3
SStotal = SSeffect + SSsubj
1757.574 = 605.574 + 281.676
Professional
statistician on a
closed course.
Do not try at
home!
+
+
SSerror
870.325
r2 = SSeffect /
( SSeffect + SSerror )
= 605.574 / ( 605.574 + 281.676 ) = .68
r2 =
F / (F + dferror)
= 19.349 / ( 19.349 + 9) = .68
What happened?????
Same data. Same means & Std.
Same total variance. Different F ???
BG ANOVA SSTotal = SSEffect +
WG ANOVA
SSError
SSTotal = SSEffect + SSSubj + SSError
The variation that is called “error” for the BG ANOVA is divided
between “subject” and “error” variation in the WG ANOVA.
Thus, the WG F is based on a smaller error term than the BG
F  and so, the WG F is generally larger than the BG F.
What happened?????
Same data. Same means & Std.
Same total variance. Different r ???
r2 = effect / (effect + error)
 conceptual formula
=
SSeffect / ( SSeffect + SSerror )
 definitional formula
=
F / (F + dferror)
 computational forumla
The variation that is called “error” for the BG ANOVA is divided
between “subject” and “error” variation in the WG ANOVA.
Thus, the WG r is based on a smaller error term than the BG r
 and so, the WG r is generally larger than the BG r.
Both of these models assume there are no confounds, and
that the individual differences are the only source of withincondition variability
BG SSTotal = SSEffect + SSError
WG SSTotal = SSEffect+SSSubj+SSError
A “more realistic” model of F
F=
SSeffect / dfeffect
SSerror / dferror
IndDif  individual differences
BG SSTotal = SSEffect + SSconfound + SSIndDif + SSwcvar
WG SSTotal = SSEffect + SSconfound +SSSubj + SSIndDif + SSwcvar
SSconfound  between condition variability caused by
anything(s) other than the IV (confounds)
SSwcvar  inflated within condition variability caused by
anything(s) other than “natural population individual
differences”
Imagine an educational study that compares the effects of two
types of math instruction (IV) upon performance (% - DV)
Participants were randomly assigned to conditons, treated, then
allowed to practice (Prac) as many problems as they wanted to
before taking the DV-producing test
IV
• compare Ss 5&2 - 7&4
Control Grp
Exper. Grp
Prac DV
S1
S3
5
75
Prac DV
S2
10
82
5
74
S4
10
84
S5 10
78
S6
15
88
S7 10
79
S8
Confounding due to Prac
• mean prac dif between cond
15 89
WG variability inflated by Prac
• wg corrrelation or prac & DV
Individual differences
• compare Ss 1&3, 5&7, 2&4,
or 6&8
The problem is that the F-formula will …
•
Ignore the confounding caused by differential practice
between the groups and attribute all BG variation to the type
of instruction (IV)  overestimating the effect
•
Ignore the inflation in within-condition variation caused by
differential practice within the groups and attribute all WG
variation to individual differences  overestimating the error
•
As a result, the F & r values won’t properly reflect the
relationship between type of math instruction and
performance  we will make a statistical conclusion error !
•
Our inability to procedurally control variables like this will lead
us to statistical models that can “statistically control” them
F=
SSeffect / dfeffect
r
SSerror / dferror
= F / (F + dferror)
How research design impacts F  integrating stats & methods!
SSTotal = SSEffect+SSconfound+SSIndDif+SSwcvar
F=
SSeffect / dfeffect
SSerror / dferror
SSEffect  “bigger” manipulations produce larger mean
difference between the conditions  larger F
SSconfound  between group differences – other than the IV -change mean difference  changing F
• if the confound “augments” the IV  F will be inflated
• if the confound “counters” the IV  F will be underestimated
SSIndDif  more heterogeneous populations have larger withincondition differences  smaller F
SSwcvar  within-group differences – other than natural
individual differences  smaller F
• could be “procedural”  differential treatment within-conditions
• could be “sampling”  obtain a sample that is “more
heterogeneous than the target population”