Transcript Lecture 1

Lecture 4
Sample size determination
4.1 Criteria for sample size determination
4.2 Finding the sample size
4.3 Some simple variations
4.4 Further considerations
RDP
Statistical Methods in Scientific Research - Lecture 4
1
4.1 Criteria for sample size determination
Suppose that we are to conduct an investigation comparing
populations, PA and PB
Sample A comprises nA units of observation from PA
Sample B comprises nB units of observation from PB
Suppose that nA = nB and that n = nA + nB
The responses will be quantitative, and the analysis will use a
t-test
How should we choose n?
RDP
Statistical Methods in Scientific Research - Lecture 4
2
Let
mA = mean response for PA
mB = mean response for PB
Null hypothesis is H0: mA = mB
From the data, we will obtain the sample means x A and x B
and sample standard deviations SA and SB for groups A and B
Once we have the data, we can:
- Reject H0 and say that mA > mB
- Reject H0 and say that mA < mB
- Not reject H0
RDP
Statistical Methods in Scientific Research - Lecture 4
3
When nA = nB = n/2, the t-statistic is
t
xA - xB
 1
1 
S 


n
n
B 
 A
xA - xB 


n
2S
where
S
 n A - 1 SA2   n B - 1 SB2
nA  nB - 2
S2A  SB2

2
t will tend to be positive if mA > mB, negative if mA < mB and
close to zero if mA = mB
RDP
Statistical Methods in Scientific Research - Lecture 4
4
We will:
- Reject H0 and say that mA > mB if t  k
- Reject H0 and say that mA < mB if t  -k
- Not reject H0 if -k < t < k
Say mA < mB
Say mA > mB
Do not reject H0
-k
0
k
t
Now we need to find both n and k
RDP
Statistical Methods in Scientific Research - Lecture 4
5
Suppose that, in truth, mA = mB
This does not mean that we will observe x A  x B nor t = 0
In fact, we may observe t  k or t  -k, just by chance
This means that we might reject H0 when H0 is true
This is called type I error
RDP
Statistical Methods in Scientific Research - Lecture 4
6
Suppose that, in truth, mA = mB + d
where d > 0, and is of a magnitude that would be
scientifically worth detecting
We may still observe t  k by chance
This means that we might fail to reject H0 when H0 is false
This is called type II error
RDP
Statistical Methods in Scientific Research - Lecture 4
7
The probability that t  k or t  -k, when mA = mB, is called
the risk of type I error, and is denoted by a
(This is for a two-sided alternative: the probability that t  k,
when mA = mB, is the risk of type I error for a one-sided
alternative and is equal to a/2)
The probability that t  k, when mA = mB + d is called
the risk of type II error, and is denoted by b
The probability that t  k, when mA = mB + d is called
the power, and is equal to 1 - b
RDP
Statistical Methods in Scientific Research - Lecture 4
8
Reducing type I error
Increase k – make it difficult to reject H0
Increasing power
Decrease k – make it easy to reject H0
Reducing type I error and increasing power
simultaneously
Increase n – this will make the study more informative, but it
will cost more
RDP
Statistical Methods in Scientific Research - Lecture 4
9
4.2 Finding the sample size
Suppose that the true standard deviation within each of the
populations PA and PB is s
Then t  Z where
Z
 xA - xB 
n
2s
Z follows the normal distribution, with standard deviation 1
When mA = mB, Z has mean 0
When mA = mB + d, Z has mean dn/(2s)
RDP
Statistical Methods in Scientific Research - Lecture 4
10
Specify that the type I risk of error (two-sided) should be a:
P( Z  k or Z  -k : mA = mB) = a
(1)
Under H0, Z is
normally distributed
with mean 0 and
st dev 1
k is the value
exceeded by a
normal (0, 1)
random variable
with prob a/2
RDP
Statistical Methods in Scientific Research - Lecture 4
11
Specify that the type II risk of error should be b:
P( Z  k : mA = mB + d) = b
(2)
Under H0, Z is
normally distributed
with mean dn/(2s)
and st dev 1
k - dn/(2s)
is the value
exceeded by a
normal (0, 1)
random variable
with prob 1 - b
RDP
Statistical Methods in Scientific Research - Lecture 4
12
For a = 0.05 and 1 – b = 0.90, we have
k = 1.960 and k - dn/(2s) = -1.282
Thus
2 1.960  1.282 
n  4s
2
d2
s
n  
d
Type I
error: a
RDP
s
 42.030  
d
2
Power: 1 - b
2
0.8
0.9
0.95
0.1
24.730
34.255
43.289
0.05
31.396
42.030
51.979
0.01
46.716
59.518
71.257
Statistical Methods in Scientific Research - Lecture 4
13
Sample size increases: - as s increases
- as d decreases
- as a decreases
- as 1 - b increases
RDP
Statistical Methods in Scientific Research - Lecture 4
14
4.3 Some simple variations
Unequal randomisation
nEnC
The power of a study depends on
n
which, for equal sample sizes is equal to
 n / 2  n / 2 
n
n

4
For nE = RnC, n = RnC + nC and so
n E n C Rn /  R  1n /  R  1
Rn


2
n
n
 R  1
RDP
Statistical Methods in Scientific Research - Lecture 4
15
Unequal randomisation
So, the overall sample size is multiplied by the factor
2
R

1


4Rn
Fn

2
4R
 R  1
and nE by FE and nC by FC, where
R 1
R 1
FE 
and FC 
2
2R
RDP
R
1
2
3
5
10
F
FE
1
1
1.125
1.500
1.333
2.000
1.800
3.000
3.025
5.500
FC
1
0.750
0.667
0.600
0.550
Statistical Methods in Scientific Research - Lecture 4
16
Unknown standard deviation
The sample size formula depends on guessing s
If this guess is smaller than the truth, the sample size will be
too small and the study underpowered
If this guess is larger than the truth, the sample size will be
too large and the sample size unnecessarily large
A more accurate calculation can be based on the t-distribution
rather than the normal, but this makes little difference and
does not overcome the dependence on s
RDP
Statistical Methods in Scientific Research - Lecture 4
17
Unknown standard deviation
Often, the final analysis will be based on a linear model, not
just a t-test
The formulae given can still be used, but s is now the
residual standard deviation (the SD about the fitted model)
Fitting the right factors will reduce the residual standard
deviation, and so the sample size will also be reduced
- but you have to guess what s will be in advance!
RDP
Statistical Methods in Scientific Research - Lecture 4
18
Sample size for estimation
The sample size can be determined to give a confidence
interval of specified width W
The 95% confidence interval for d = mA - mB is of the form
 1
 1
1 
1 
 x A - x C  - 1.96 s     d   x A - x C   1.96 s   
 nA nB 
 nA nB 
when sample sizes are large (Lecture 1, Slide 24)
When nA = nB = n/2, this has length
s
4
2 1.96 s    7.84
n
n
RDP
Statistical Methods in Scientific Research - Lecture 4
19
Sample size for estimation
We need to set
s
7.84
W
n
which means that
 s
n  61.47  
W
RDP
2
Statistical Methods in Scientific Research - Lecture 4
20
Binary data
For R = 1, a = 0.05 and 1 – b = 0.90, we have
n
4 1.960  1.282 
2 p 1 - p 
2
42.030
 2
 p 1 - p 
where
 pE
  log e 
 1 - pE
 pC 

Rp E  p C
 and p 
 - log e 
R 1

 1 - pC 
pC is the anticipated success rate in PC, and pE the improved
rate in PE to be detected with power 1 - b
RDP
Statistical Methods in Scientific Research - Lecture 4
21
Examples for binary data: R = 1, a = 0.05 and 1 – b = 0.90
RDP
pC
pE

p 1 - p 
0.1
0.2
0.811
0.1275
502
0.1
0.3
1.350
0.1600
144
0.1
0.5
2.197
0.2100
42
0.3
0.4
0.442
0.2275
946
0.3
0.5
0.847
0.2400
244
0.3
0.7
1.695
0.2500
60
0.4
0.5
0.405
0.2475
1034
0.4
0.6
0.811
0.2500
256
0.4
0.8
1.792
0.2400
56
0.5
0.6
0.405
0.2475
1034
0.5
0.7
0.847
0.2400
244
0.5
0.9
2.197
0.2100
42
Statistical Methods in Scientific Research - Lecture 4
n
22
Binary data
This approach is based on the log-odds ratio
 pE
  log e 
 1 - pE
 pC 


 - log e 
1
p
C 


Many other approximate formulae exist
All give similar answers when sample sizes are large: exact
calculations can be made for small sample sizes
RDP
Statistical Methods in Scientific Research - Lecture 4
23
4.4 Further considerations
Setting the values for a and b
The standard scientific convention is to ensure that a will be
small, and allow any risks to be taken with b
For example, if an SD or a control success rate is
underestimated at the design stage, the study will be
underpowered – the analysis maintains the type I error a at
the cost of losing power
a is the community’s risk of being given a false conclusion
b is the scientist’s risk of not proving his/her point
RDP
Statistical Methods in Scientific Research - Lecture 4
24
Exceptions
If the scientist wishes to prove the null hypothesis
(equivalence testing)
- then b should be kept small, while a can be inflated if
necessary
In a pilot study, preliminary to a larger confirmatory study
- type I errors can be rectified in the next study, but
type II errors will mean that the next study is not
conducted at all
RDP
Statistical Methods in Scientific Research - Lecture 4
25
Finally:
 Many more sample size formulae exist – see Machin et al.
(1997)
 Software also exists: nQuery advisor, PASS
 Ensure that the sample size formula used matches the
intended final analysis
 In complicated situations, the whole study can be
simulated on the computer in advance to determine its
power
RDP
Statistical Methods in Scientific Research - Lecture 4
26