Transcript 10-17 lecture +Qx

Review
+14
MA = 14
+14
Flood
-27
-41 MB = -36.3
-41
Find the mean square (MS) based on these two samples.
A.
B.
C.
D.
E.
26.1
32.7
43.6
65.3
792.7
Review
+14
MA = 14
+14
Flood
-27
-41 MB = -36.3
-41
The mean square is MS = 43.6. Now find the standard error, s M A -M B .
A. 6.0
B. 8.5
C. 19.5
D. 36.3
E. 56.0
Review
+14
MA = 14
+14
Flood
-27
-41 MB = -36.3
-41
The standard error is s M A -M B = 6.0. Now calculate t.
A.
B.
C.
D.
3.7
8.4
133.8
301.8
Effect Size
10/17
Effect Size
• If there's an effect, how big is it?
– How different is m from m0, or mA from mB, etc.?
• Separate from reliability
– Inferential statistics measure effect relative to standard error
– Tiny effects can be reliable, with enough power
– Danger of forgetting about practical importance
• Estimation vs. inference
– Inferential statistics convey confidence
– Estimation conveys actual, physical values
• Ways of estimating effect size
– Raw difference in means
– Relative to standard deviation of raw scores
Direct Estimates of Effect Size
• Goal: estimate difference in population means
– One sample: m - m0
– Independent samples: mA – mB
– Paired samples: mdiff
• Solution: use M as estimate of m
– One sample: M – m0
– Independent samples: MA – MB
– Paired samples: Mdiff
• Point vs. interval estimates
– We don't know exact effect size; samples just provide an estimate
– Better to report a range that reflects our uncertainty
• Confidence Interval
– Range of effect sizes that are consistent with the data
– Values that would not be rejected as H0
• CI is range of values for m or mA – mB consistent with data
– Values that, if chosen as null hypothesis, would lead to |t| < tcrit
• One-sample t-test (or paired samples):
– Retain H0 if t =
M - m0
< tcrit i.e.
SE
M - m0 < tcrit × SE
-4
p
p
0.0 0.4 0.8
0.0 0.4 0.8
– Therefore any value of m0 within tcritSE of M would not be rejected
0.0 0.4 0.8
0.0 0.4p 0.8
Computing Confidence Intervals
-4 -2
tcritSE
M M
m
m
m
-2 00 -4 -400 2 -2 -22 4 00
M
z
– tcritSE
z
m
004
M +ztcritSE
z
2
2
4
4
Example
X ={10,6,14,8,12}
Reject null hypothesis if:
M =10
t > tcrit
å( X - M )
s=
n -1
sM =
t=
s
n
2
= 3.16
= 1.41
M -m0 10 -m0
=
sM
1.41
tcrit = 2.78
10 -m0
1.41
> 2.78
10 -m0 > 2.78 ×1.41= 3.93
Confidence Interval:
10 ±3.93
M ± tcrit × SE
Formulas for Confidence Intervals
• Mean of a single population (or of difference scores)
M ± tcritSE
• Difference between two means
(MA – MB) ± tcritSE
• Always use two-tailed critical value
– p(tdf > tcrit) = a/2
– Confidence interval has upper and lower bounds
– Need a/2 probability of falling outside either end
• Effect of sample size
– Increasing n decreases standard error
– Confidence interval becomes narrower
– More data means more precise estimate
Interpretation of Confidence Interval
• Pick any possible value for m (or mA – mB)
• IF this were true population value
– 5% chance of getting data that would lead us to falsely reject that value
– 95% chance we don’t reject that value
• For 95% of experiments, CI will contain true population value
– "95% confidence"
–
–
–
–
0.0 0.4 0.8
p
• Other levels of confidence
Can calculate 90% CI, 99% CI, etc.
Correspond to different alpha levels: confidence = 1 – a
Leads to different tcrit: t.crit = qt(alpha/2,df,low=FALSE)
Higher confidence requires wider intervals (tcrit increases)
tcritSE
• Relationship to hypothesis testing
– If m0 (or 0) is not in the confidence
interval,
M
M then we reject H0
m
-4
-2
0
M – tcritSE
z
2
4
M + tcritSE
Standardized Effect Size
• Interpreting effect size depends on variable being measured
– Improving digit span by 2 more important than for IQ
• Solution: measure effect size relative to variability in raw scores
• Cohen's d
– Effect size divided by standard deviation of raw scores
– Like a z-score for means
Samples
d
dTrue
Estimated
One
Independent
Paired
m - m0
s
m A - mB
s
m diff
s
M - m0
M A - MB
M diff
MS
MS
MS
Meaning of Cohen's d
• How many standard deviations does the mean change by?
• Gives z-score of one mean within the other population
– (negative) z-score of m0 within population
– z-score of mA within Population B
• pnorm(d) tells how many scores are above other mean
(if population is Normal)
-4
m
-2 0
m
0
z
2
4
0.0 0.4 0.8
ds
0.0 0.4 0.8
p
p
0.0 0.4 0.8
– Fraction of scores in population that are greater than m0
– Fraction of scores in Population A that are greater than mB
-4
ds
-4-2
m
m
0 2A
z
z
-20 B
24
4
Cohen's d vs. t
Samples
Statistic
d
t
One
Independent
Paired
M - m0
M A - MB
M diff
MS
M - m0
MS n
MS
M A - MB
æ
ö
MSç n1 + n1 ÷
è A
Bø
MS
M diff
MS n
• t depends on n; d does not
• Bigger n makes you more confident in the effect,
but it doesn't change the size of the effect
Review
The average guppy can swim at 21 mph. On a scuba trip, you discover a
new species and wonder if they go the same speed as normal guppies. You
time 15 fish and calculate a confidence interval for the mean of [21.4, 25.0].
What do you conclude?
A. Null hypothesis: These guppies are the same as average
B. Alternative hypothesis: These guppies are faster than average
C. It depends on your choice of a
Review
The average guppy can swim at 21 mph. On a scuba trip, you discover a
new species and wonder if they go the same speed as normal guppies. You
time 15 fish and calculate a confidence interval for the mean of [21.4, 25.0].
What was the mean of your sample?
A.
B.
C.
D.
E.
21.0
21.4
22.8
23.2
25.0
Review
Ten subjects are assessed for anxiety before and after a session of mindful
meditation training. Calculate the standardized effect size.
Subject
A.
B.
C.
D.
1
2
3
4
5
6
7
8
9
10
Before
27
44
29
32
47
53
24
59
38
17
After
23
40
25
33
41
51
21
50
34
16
Difference
4
4
4
-1
6
2
3
9
4
1
1.33
4.00
4.22
9.72
Mdiff = 3.6
sdiff = 2.7