Transcript General Linear Model 2 - University of South Florida

General Linear Model 2
Intro to ANOVA
Questions
 ANOVA makes assumptions about error for
significance tests. What are the
assumptions?
 What might happen (why would it be a
problem) if the assumption of {normality,
equality of error, independence of error}
turned out to be false?
 What is an expected mean square? Why is
it important?
 Why do we use the F test to decide whether
means are equal in ANOVA?
Questions (2)
 Correctly interpret ANOVA summary
tables.
 Find correct values of critical F from tabled
values for a given test.
 Suppose someone has worked out that a
one-way ANOVA with 6 levels has a power
of .80 for the overall F test. What does this
mean?
 Describe (make up) a concrete example of a
one-way ANOVA where it makes sense to
use an overall F test. Explain why ANOVA
(not t, chi-square or something else) is the
best method for the analysis.
New Distributions
• So far, the normal (z) and its short, fat
relative, the t distribution.
• The normal has two children, chi2
square (  ) and F.
• Chi-square is made of the sum of v
squared deviations from the unit
normal. It essentially show the
sampling distribution of the variance.
• F is the ratio of two chi-squares.
ANOVA Assumptions
•
•
Recall we can partition total SS into
between (treatment) and within (error)
SS. No assumptions needed.
To conduct tests about population
effects, have to make assumptions:
1. Within cells (treatments) error is normal.
2. Homogeneity of error variance.
3. Independent errors.
Assumptions
• Normality – sampling distribution of
means,variances; not bad if N is large; e.g.
reaction time
• Homogeneity – pooled estimate of population
value. Where are means different? Assumed
equal error for each. E.g., ceiling effects in
training.
• Independence – sampling distribution again;
e.g., cheating on exam, nesting (schools, labs)
Mean Square Between Groups
 (2v )
• Mean square = SS/df =
= variance
v
estimate.
(J treatments)
• MS between = SS between
J 1
• E(MS between) =
 e2 
2
n

 j j
j
J 1
• If there is no treatment effect, MS between =
error variance.
• If there is a treatment effect, MS between is
bigger than error variance.
Mean Square Within Groups
• MS within =
(N is total sample size
and J is number of
groups.)
SS within
NJ
2

• E(MS within) = e
• Expected mean square for error is  e2.
Expected mean square for treatment is same
2
n

 j j.
plus treatment effect:
 e2 
j
J 1
• When there is no treatment effect, between
and within estimate same thing.
Review
ANOVA makes assumptions about error for
significance tests. What are the assumptions?
What might happen (why would it be a
problem) if the assumption of {normality,
equality of error, independence of error}
turned out to be false?
What is an expected mean square? Why is it
important?
The F Test (1)
• Suppose
H0 :  j  0, for all j
H1 : j  0, for some j
• The null is equivalent to:
• If the null is true, then
MS between
~ F( J 1, N  J )
MS witin
1  2  ...   j  
The ratio of the two variance
estimates will be distributed as
F with J-1 and N-J degrees of
freedom.
The F Test (2)
MS between
~ F( J 1, N  J )
MS witin
This is a big deal because we can use variance estimates to test the
hypothesis that any number of population means are equal. Equality
of means is same as testing population treatment effect(s).
For a treatment effect to be detected, F must be larger than 1. F
is one-tailed in the tables which show upper tail values of F
given the two df.
p value (significance)
1.0
0.8
0.6
N o Effec t (n.s .)
0.4
Signifc ant
(Alpha = .05)
0.2
0.0
0
1
2
3
4
5
Obtained F (2 and 10 df)
6
F Table – Critical Values
Numerator df: dfB
dfW
1
2
3
4
5
5 5%
1%
10 5%
1%
6.61
16.3
4.96
10.0
5.79
13.3
4.10
7.56
5.41
12.1
3.71
6.55
5.19
11.4
3.48
5.99
5.05
11.0
3.33
5.64
12 5%
1%
4.75
9.33
3.89
6.94
3.49
5.95
3.26
5.41
3.11
5.06
14 5%
1%
4.60
8.86
3.74
6.51
3.34
5.56
3.11
5.04
2.96
4.70
Review
Why do we use the F test to decide
whether means are equal in ANOVA?
Suppose we have an ANOVA design
with 3 cells and 5 people per cell. What
is the critical value of F at alpha = .05?
Calculating F – 1 Way
ANOVA
Sums of squares (squared deviations from the mean) tell the story
of variance. The simple ANOVA designs have 3 sums of
squares.
The total sum of squares comes from the
2
SStot  ( X ij  X ) distance of all the scores from the grand
mean. This is the total; it’s all you have.

SSW   ( X ij  X j )2
SSB   n j ( X j  X )2
SSTOT  SSB  SSW
The within-group or within-cell sum of
squares comes from the distance of the
observations to the cell means. This
indicates error.
The between-cells or between-groups
sum of squares tells of the distance of
the cell means from the grand mean.
This indicates IV effects.
Computational Example:
Caffeine on Test Scores
G1: Control
G2: Mild
Test Scores
G3: Jolt
75
77
79
80
82
84
70
72
74
81
83
76
78
79
86
88
Means
84
3.16
SDs (N-1)
3.16
74
3.16
SStot  ( X ij  X )2
G1
75
79
16
Control
77
79
4
M=79
79
79
0
SD=3.16
81
79
4
83
79
16
G2
80
79
1
M=84
82
79
9
SD=3.16
84
79
25
86
79
49
88
79
81
G3
70
79
81
M=74
72
79
49
SD=3.16
74
79
25
76
79
9
78
79
1
Sum
370
Total
Sum of
Squares
SSW   ( X ij  X j )2
X ij
Xj
( X ij  X j )2
G1
75
79
16
Control
77
79
4
M=79
79
79
0
SD=3.16
81
79
4
83
79
16
G2
80
84
16
M=84
82
84
4
SD=3.16
84
84
0
86
84
4
88
84
16
G3
70
74
16
M=74
72
74
4
SD=3.16
74
74
0
76
74
4
78
74
16
Sum
120
Within
Sum of
Squares
SSB  n j ( X j  X )2
Xj
X
( X j  X )2
G1
79
79
0
Control
79
79
0
M=79
79
79
0
SD=3.16
79
79
0
79
79
0
G2
84
79
25
M=84
84
79
25
SD=3.16
84
79
25
84
79
25
84
79
25
G3
74
79
25
M=74
74
79
25
SD=3.16
74
79
25
74
79
25
74
79
25
Sum
250
Between
Sum of
Squares
ANOVA Source (Summary) Table
Source
SS
df
Between 250 J-1=
Groups
3-1=2
Within
Groups
Total
MS
SS/df
250/2=
125
=MSB
120 N-J=
120/12 =
15-3=12 10 =
MSW
370 N-1=
15-1=14
F
F=
MSB/MSW
= 125/10
=12.5
F( .05,2,12)  3.89
ANOVA Summary
• Calculate SS (total, between, within)
• Each SS has associated df to calculate
MS
• F is ratio of MSb to MSw
• Compare obtained F (12.5) to critical
value (3.89). Significant if obtained F
is larger than critical.
• One-tailed test makes sense for F.
Review
• Suppose we have 4 groups and 10
people per group. We find that SSB =
60 and SSW = 40. Construct an
ANOVA summary table and test for
significance of the overall effect.
ANOVA Descriptive Stats
• Because SStot = SSb+SSw we can figure
proportion of total variance due to
treatment.
• Proportion of total variance due to
treatment is:
• R2= SSb/SStot.
• Varies from 0 (no effect) to 1 (no error).
• Sample value is biased (too large).
Estimating Power
• Power for what? For one-way
ANOVA, power usually means for the
overall F, i.e., at least 1 group mean is
different from the others.
• Howell uses noncentral F for sample
size calculation.
'
2
(



)
/k
 j
  ' n
 e2
2
n  '2

Where k is the number of treatment
goups; n is sample size per group.
Variance of error is MSE in the
population (variance of DV within
cells). Mu(j) are treatment means;
mu is grand mean.
SAS Power calculation
• SAS will compute sample size
requirements for a given scenario.
• You input the expected means and a
common (within cell) standard
deviation, (along with alpha and desired
power) and it will tell you the sample
size you need.
run;
SAS Input
**********************************************************
* Power computation example from Howell, 2010, p. 350.
* Note the standard deviation is the square root of the
* provided MSE: sqrt(240.35) = ~ 15.5.
**********************************************************;
proc power ;
onewayanova
groupmeans = 34 | 50.8 | 60.33 | 48.5 | 38.1
stddev = 15.5
alpha = 0.05
npergroup = .
power = .8;
SAS Output
The POWER Procedure
Overall F Test for One-Way ANOVA
Fixed Scenario Elements
Method
Exact
Alpha
0.05
Group Means
34 50.8 60.33 48.5 38.1
Standard Deviation
15.5
Nominal Power
0.8
Computed N Per Group
Actual
Power
0.831
N Per
Group
8
Review
Suppose someone has worked out that a
one-way ANOVA with 6 levels has a
power of .80 for the overall F test.
What does this mean?
Describe (make up) a concrete example
of a one-way ANOVA where it makes
sense to use an overall F test. Explain
why ANOVA (not t, chi-square or
something else) is the best method for
the analysis.