Transcript k-group Effect Size & Power

Effect Sizes & Power Analyses for
group Designs
•
•
•
•
Effect Size Estimates for k-group ANOVA designs
Power Analysis for k-group ANOVA designs
Effect Size Estimates for k-group X2 designs
Power Analysis for k-group X2 designs
k-
k-BG Effect Sizes
When you have more than 2 groups, it is possible to
compute the effect size for “the whole study”.
Include the F-value, the df
(both for the effect and
the error), and click the
button for the type of
design you have (BG or
WG)
However, this type of effect size is not very helpful, because:
-- you don’t know which pairwise comparison(s) make up the r
-- it can only be compared to other designs with exactly the
same combination of conditions
k-BG Effect Sizes
Just as RH: for k-group designs involve comparing 2 groups
at a time (pairwise comparisons)… The most useful effect
sizes for k-group designs are computed as the effect size
for 2 groups (effect sizes for pairwise comparisons)
Since you won’t have F-values for the pairwise comparisons,
you will use Computator to complete a 2-step computation
Using info from the SPSPS output
d = (M1 - M2 ) /  MSerror
r =

d²
---------d² + 4
Pairwise effect sizes computation for k-BG designs
For no therapy vs.
weekly therapy …
Descriptives
outcome variable -- larger scores are better
N
no therapy
weekly therapy
daily therapy
Total
25
24
30
79
Mean
42.1400
43.9500
50.9600
46.2607
Std. Deviation
12.06261
9.39588
14.59920
12.80355
O
o
m
d
F
S
i
a
g
f
B
2
0
6
5
W
6
5
T
8
For a BG design
be sure to press
k-WG Effect Sizes
Just as RH: for k-group designs involve comparing 2 groups
at a time (pairwise comparisons)… Effect sizes for k-group
designs are computed as the effect size for 2 groups (effect
sizes for pairwise comparisons)
Since you won’t have F-values for the pairwise comparisons,
you will use Computator to complete a 3-step computation
Using info from the SPSPS output
d = (M1 - M2 ) /  (MSerror * 2)
dw = d * 2
dw²
r =
--------- dw² + 4
Pairwise effect sizes computation for k-WG designs
For no intake vs. mid …
Descriptive Statistics
INTAKE
MID
FINAL
-
Mean
26.6200
33.8900
30.6200
Std. Deviation
8.99399
9.02793
8.45050
N
48
48
48
S
u
M
I
I
I
S
q
d
S
F
S
u
i
f
g
a
o
q
T
S
I
p
4
2
2
0
0
G
4
4
7
0
0
Hu
4
3
8
0
0
L
o
4
0
4
0
3
E
S
r
p
9
4
0
G
9
9
7
Hu
9
8
8
L
o
9
0
0
For a WG design
be sure to press
Determining the power you need ..
For a 2-condition design...
• the omnibus-F is sufficient -- retain or reject, you’re done !
• you can easily determine the sample size needed to test any
expected effect size with a given amount of power
For a k-condition design …
• the power of the omnibus-F - isn’t what matters !
• a significant omnibus-F only tells you that the “two most
different” means are significantly different
• follow-up (pairwise) analyses will be needed to test if the
pattern of the mean differences matches the RH:
• you don’t want to have a “pattern of results” that is really just a
“pattern of differential statistical power”
• you need to assure that you have sufficient power for the
smallest pairwise effect needed to test your specific RH:
k-group Power Analyses
As before, there are two kids of power analyses;;;
A priori power analyses
• conducted before the study is begun
• start with r & desired power to determine the needed N
Post hoc power analysis
• conducted after retaining H0:
• start with r & N and determine power & Type II probability
Power Analyses for k-BG designs
Important Symbols
S is the total # of participants in that pairwise comp
n = S / 2 is the # of participants in each condition
of that pairwise comparison
N = n * k is the total number or participants in the study
Example
• the smallest pairwise effect size for a 3-BG study was .25
• with r = .25 and 80% power S = 120
• for each of the 2 conditions
n = S / 2 = 120 / 2 = 60
• for the whole study
N = n * k = 60 * 3 = 180
Power Analyses for k-WG designs
Important Symbols
S is the total # of participants in that pairwise comp
For WG designs, every participant is in every condition, so…
S is also the number of participants in each condition
Example
• the smallest pairwise effect size for a 3-WG study was .20
• with r = .20 and 80% power S = 191
• for each condition of a WG design n = S = 191
• for the whole study
N = S = 191
Combing LSD & r …
Cx
mean
M dif
r
Tx1
M dif
r
Cx 20.3
Tx1 24.6
4.3
.32
Tx2 32.1
11.8*
.54
7.5*
.41
* indicates mean difference is significant based on LSD criterion (min dif = 6.1)
Something to notice …
• The effect size of Cx vs. Tx1 is substantial (Cohen calls .30
“medium effect”), but is not significant, suggesting we should
check the power of the study for testing an effect of this size.
k-group Effect Sizes
When you have more than 2 groups, it is possible to
compute the effect size for “the whole study”.
Include the X², the total N and
click the button for df > 1
However, this type of effect size is not very helpful, because:
-- you don’t know which pairwise comparison(s) make up the r
-- it can only be compared to other designs with exactly the
same combination of conditions
Pairwise Effect Sizes
Just as RH: for k-group designs involve comparing 2 groups
at a time (pairwise comparisons)… The most useful effect
sizes for k-group designs are computed as the effect size
for 2 groups (effect sizes for pairwise comparisons)
The effect size
computator calculates
the effect size for each
pairwise X² it computes
k-group Power Analyses
As before, there are two kinds of power analyses;;;
A priori power analyses
• conducted before the study is begun
• start with r & desired power to determine the needed N
Post hoc power analysis
• conducted after retaining H0:
• start with r & N and determine power & Type II probability
Power Analyses for k-group designs
Important Symbols
S is the total # of participants in that pairwise comp
n = S / 2 is the # of participants in each condition
of that pairwise comparison
N = n * k is the total number or participants in the study
Example
• the smallest pairwise X² effect size for a 3-BG study was .25
• with r = .25 and 80% power S = 120
• for each of the 2 conditions
n = S / 2 = 120 / 2 = 60
• for the whole study
N = n * k = 60 *3 = 180