Transcript 002_arizona_LM2x

More on the linear
model
Categorical predictors
1500
men
1000
500
0
Reaction time
RT ~ Noise + Gender
Noise
women
12
resp ~ Condition
6
4
2
0
resp
8
10
mean B
equals:
1
A
B
12
10
6
4
2
0
resp
8
mean B
equals:
3
A
B
12
10
6
4
2
0
resp
8
mean B
equals:
8
A
B
Demo
set.seed(666)
pred = c(rep(0,20),rep(1,20))
resp = c(rnorm(20,mean=2,sd=1),
rnorm(20,mean=2,sd=1))
for(i in 1:10){
resp = c(resp[1:20],resp[21:40]+1)
plot(resp~pred,
xlim=c(-1,2),ylim=c(0,14),xaxt="n",xlab="")
axis(side=1,at=c(0,1),labels=c("A","B"))
text(paste("mean B\nequals:",i,sep="\n"),
x=-0.5,y=10,cex=1.5,font=2)
abline(lm(resp~pred))
Sys.sleep(1.25)
}
Deep idea:
A categorical difference
between two groups can be
expressed as a line going from
one group to another
Continuous predictor
… units up
1 unit “to
the right”
Continuous predictor
… units up
1 unit “to
the right”
Categorical predictor
… units up
F
M
1 category “to
the right”
Output: categorical predictor
> summary(lm(RT ~ gender))
Call:
lm(formula = RT ~ gender)
Residuals:
Min
1Q
-231.039 -39.649
Median
2.999
3Q
44.806
Max
155.646
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 349.203
4.334
80.57
<2e-16 ***
genderM
205.885
6.129
33.59
<2e-16 ***
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 61.29 on 398 degrees of freedom
Multiple R-squared: 0.7392, Adjusted R-squared: 0.7386
F-statistic: 1128 on 1 and 398 DF, p-value: < 2.2e-16
Output: categorical predictor
> summary(lm(RT ~ gender))
Call:
lm(formula = RT ~ gender)
Residuals:
Min
1Q
-231.039 -39.649
Median
2.999
3Q
44.806
Max
155.646
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 349.203
4.334
80.57
<2e-16 ***
genderM
205.885
6.129
33.59
<2e-16 ***
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 61.29 on 398 degrees of freedom
Multiple R-squared: 0.7392, Adjusted R-squared: 0.7386
F-statistic: 1128 on 1 and 398 DF, p-value: < 2.2e-16
200
REFERENCE
LEVEL
F
M
300
400
500
600
700
But what happens…
… when I have more than two
groups or categories?
F
M
Group
I
200
300
400
500
Reaction time
600
700
Output: three groups
Females = 349.203 (intercept)
Males = 349.203 + 205.885
Infants = 349.203 + 203.983
> summary(lm(RT ~ gender))
Call:
lm(formula = RT ~ gender)
Residuals:
Min
1Q
-231.039 -41.055
Median
3.404
3Q
38.428
Max
155.646
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 349.203
4.228
82.59
<2e-16 ***
genderM
205.885
5.979
34.43
<2e-16 ***
genderI
203.983
5.979
34.11
<2e-16 ***
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 59.79 on 597 degrees of freedom
Multiple R-squared: 0.724, Adjusted R-squared: 0.7231
F-statistic: 783.1 on 2 and 597 DF, p-value: < 2.2e-16
200
REFERENCE
LEVEL
F
M
Group
I
300
400
500
Reaction time
600
700
Output: changing reference level
Infants = 553.185 (intercept)
Females = 553.185 – 203.983
Males = 553.185 + 1.903
> summary(lm(RT ~ gender))
Call:
lm(formula = RT ~ gender)
Residuals:
Min
1Q
-231.039 -41.055
Median
3.404
3Q
38.428
Max
155.646
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 553.185
4.228 130.835
<2e-16 ***
genderF
-203.983
5.979 -34.114
<2e-16 ***
genderM
1.903
5.979
0.318
0.75
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Notice that nothing has really
Residual standard error: 59.79 on 597
degrees it’s
of freedom
changed…
just a different
Multiple R-squared: 0.724, Adjusted R-squared: 0.7231
perspective
the same data
F-statistic: 783.1 on 2 and 597 DF,
p-value: < on
2.2e-16
F
M
I
Group
REFERENCE
LEVEL
200
300
400
500
Reaction time
600
700
In case you need it:
Releveling: In R
relevel(myvector,
ref="mynew_reference_level”)
More on the linear
model
Centering and standardization
10
8
6
Familiarity Rating
4
2
40000
50000
60000
Word Frequency
70000
80000
Output: weird intercept
> summary(lm(familiarity ~ word_frequency))
Call:
lm(formula = familiarity ~ word_frequency)
Residuals:
Min
1Q Median
-4.5298 -1.2306 -0.0087
3Q
1.1141
Max
4.6988
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)
-2.790e+00 6.232e-01 -4.477 9.37e-06 ***
word_frequency 1.487e-04 1.101e-05 13.513 < 2e-16 ***
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 1.699 on 498 degrees of freedom
Multiple R-squared: 0.2683, Adjusted R-squared: 0.2668
F-statistic: 182.6 on 1 and 498 DF, p-value: < 2.2e-16
10
8
6
4
Familiarity Rating
40
2
50000
60000
70000
80000
0
10
20
Word Frequency
-10
Familiarity Rating
30
40000
0
20000
40000
Word Frequency
60000
80000
40
30
20
Familiarity Rating
10
0
-10
0
20000
40000
Word Frequency
60000
80000
40
30
20
Familiarity Rating
10
0
-10
0
20000
40000
Word Frequency
60000
80000
40
30
20
Familiarity Rating
10
0
-10
-20000
-10000
0
10000
20000
Word Frequency
is now in centered
150
100
0
50
Frequency
-40000
-20000
0
20000
40000
word_frequency
60000
80000
150
100
0
50
Frequency
-40000
-20000
0
20000
40000
word_frequency
60000
80000
Uncentered
> summary(lm(familiarity ~ word_frequency))
Call:
lm(formula = familiarity ~ word_frequency)
Residuals:
Min
1Q Median
-4.5298 -1.2306 -0.0087
3Q
1.1141
Max
4.6988
Coefficients:
(Intercept)
word_frequency
--Signif. codes:
Estimate Std. Error t value Pr(>|t|)
-2.790e+00 6.232e-01 -4.477 9.37e-06 ***
1.487e-04 1.101e-05
13.51
<2e-16 ***
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 1.699 on 498 degrees of freedom
Multiple R-squared: 0.2683, Adjusted R-squared: 0.2668
F-statistic: 182.6 on 1 and 498 DF, p-value: < 2.2e-16
Centered
> summary(lm(familiarity ~ word_frequency.c))
Call:
lm(formula = familiarity ~ word_frequency.c)
Residuals:
Min
1Q Median
-4.5298 -1.2306 -0.0087
3Q
1.1141
Max
4.6988
Coefficients:
Estimate Std. Error t
(Intercept)
5.568e+00 7.598e-02
word_frequency.c 1.487e-04 1.101e-05
--Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01
value Pr(>|t|)
73.28
<2e-16 ***
13.51
<2e-16 ***
‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 1.699 on 498 degrees of freedom
Multiple R-squared: 0.2683, Adjusted R-squared: 0.2668
F-statistic: 182.6 on 1 and 498 DF, p-value: < 2.2e-16
20
10
0
-10
Familiarity Rating
30
40
Centered, not scaled
-20000
-10000
0
Word Frequency
10000
20000
20
10
0
-10
Familiarity Rating
30
40
Centered and scaled
-3
-2
-1
0
1
2
3
Word Frequency
is now in standard
deviations
Centering vs. Standardization
• Centering = subtracting the mean of the data
from the data
mydata = mydata - mean(mydata)
• Standardization = subtracting the mean of the
data from the data and then dividing by the
standard deviation
mydata = (mydata - mean(mydata))/
sd(mydata)
Centering vs. Standardization
• Centering = subtracting the mean of the data
from the data
mydata = mydata - mean(mydata)
• Standardization = subtracting the mean of the
data from the data and then dividing by the
standard deviation
mydata = scale(mydata)
Centering vs. Standardization
• Centering = often leads to more interpretable
coefficients; doesn’t change metric
mydata = mydata - mean(mydata)
• Standardization = gets rid of the metric (is
then in standard units) and then dividing by
the standard deviation
mydata
Standardization is also often called z= (mydata
mean(mydata))/
scoring-and
sometimes normalization
(but you should not call it that way)
sd(mydata)
2
1
0
-1
-2
… which means it doesn’t really
do anything to your results
-3
Word Frequency (SDs)
3
“Standardization” is a linear
transformation
40000
50000
60000
Word Frequency
70000
80000
Linear Transformations
• Seconds  Milliseconds
• Word Frequency  Word Frequency by 1000
• Centering, Standardization
None of these change the
“significance”, only the
metric of the coefficients
More on the linear
model
Interactions
Winter & Bergen (2012)
Jack Vevea,
UC Merced
"Usually (but not
always) the interaction,
if it is present, will be
the most interesting
thing going on."
Main Effects
Interaction
Effects
One main effect
RT (ms)
small
pictures
large
pictures
NearSent
FarSent
Two main effects
RT (ms)
small
pictures
large
pictures
NearSent
FarSent
Interaction #1
RT (ms)
small
pictures
large
pictures
NearSent
FarSent
Interaction #2
RT (ms)
small
pictures
large
pictures
NearSent
FarSent
Interaction #3
RT (ms)
small
pictures
large
pictures
NearSent
FarSent
Interaction #4
RT (ms)
small
pictures
large
pictures
NearSent
FarSent
Visualizing interactions
with continuous variables
respons
e
pre
d ic
tor
2
pr
c to
e di
r1
Visualizing interactions
with continuous variables
e
respons
pre
d ic
tor
2
pr
c to
e di
r1
Visualizing interactions
with continuous variables
respons
e
pre
d ic
tor
2
pr
c to
e di
r1
Visualizing interactions
with continuous variables
respons
e
pre
d ic
tor
2
pr
c to
e di
r1
Visualizing interactions
with continuous variables
respons
e
pre
d ic
tor
2
pr
c to
e di
r1
Interpretation
of Main Effects
If interaction is significant, the
interpretation of main effects is
not straightforward
35
30
25
20
15
Response Latency (ms)
10
0
5
Neutral
Area A
Area B
50
100
Delay of Electrical Stimulation (ms)
150
“The first three rules of
statistics”
1. Draw a
picture!
2. Draw a
picture!
Michael Starbird
3. Draw a
picture!
In R: How to include an
interaction
Main effects only:
lm(RT ~ PrimeType + VowelType)
Main effects and interaction:
lm(RT ~ PrimeType*VowelType)
That’s it
(for now)