Transcript Power and Sample Size - Measurement & Statistics Club (MSC)

Concepts
• Probability to reject null when it is false, the
complement of type II error rate
power  1  
• Related factors:
– type I error rate (significance level)
– Sample size
– Effect size
– Design (One tail /two tail test, independent
/dependent Test)
Concepts
• Prospective power: conditional probability of
reject the null given the null hypothesis is
false. (29/(31+29)=.483)
• Retrospective power: conditional probability
of reject the null given the null is rejected.
(29/(2+29=.935)
H0 = T
H0 = F
Total
Fail to reject
38
31
69
Reject
2
29
31
Total
40
60
100
Concepts
• A priori power analysis
the sample size N is computed as a function of
the required power level, the pre-specified
significance level , and the population effect
size to be detected with probability.
• Post hoc power analysis: the power is
computed as a function of significance level,
the population effect size parameter, and the
sample size(s) used in a study.
Concepts
• Compromise power analyses
both alpha and power are computed as functions of the effect
size, N, and an error probability ratio (beta/alpha).
• Sensitivity analyses
the critical population effect size is computed as a function of
alpha, power, and N. Sensitivity analyses is useful for
evaluating published research. what is the minimum effect
size the test was sufficiently sensitive to. before conducting a
study to see whether, given a limited N, the size of the effect
that can be detected is at all realistic .
Concepts
• Criterion analyses compute alpha, as a
function of power, the effect size, and a given
sample size. Criterion analyses are alternatives
to post hoc power analyses after a study has
already been conducted. They may be
reasonable whenever the control of alpha is
less important than the control of beta.
Example1
• A sample was taken from a normal distributed
population, sample mean is 5, population
standard deviation is 2, sample size is 20.
calculate the power if the true mean is 6.5
(alpha = .05) with same sigma.
• Step 1: Impose 95%CI for mean=5
• Step 2: Shift to the mean=6.5, get the beta.
• Step 3: 1-beta = power
Example2
• A priori analysis: ANOVA: mean1=15,
mean2=18, mean3=24;
sqrt(MSE) = 10, alpha = .05, power = 0.8
find N and f. (N = 72, f = .3742)
• Post hoc analysis: ANOVA: mean1=15,
mean2=18, mean3=24;
sqrt(MSE) = 13, alpha = .05,
find power and f. (f = .2878, power = .5809)
Example3
• Random sampling:
Given margin of error with 95% confidence,
find sample size:
2 V (Y )  2
 2 ( N  n)
n( N  1)
 B, n 
N 2
B2
,
whereD

( N  1) D   2
4
• For proportion, if there is no prior information
about p, set p = 0.5
n
NP(1  P)
B2
, whereD 
( N  1) D  P(1  P)
4