Transcript Number of replications, plot size

Power Analysis
 How confident are we that we can detect an important
difference, if it exists?
ES r

 Power = 1 - 
, where ES = Effect Size

 Used to design an experiment (a priori)
– The size of an experiment is often limited by factors other than
statistics
– Nonetheless, it is good experimental technique to try to estimate
the degree of precision that will be attained and to present this
information as part of the proposal for the experiment
 Evaluate a completed experiment (a posteriori )
– May provide justification for publishing nonsignificant results
– Guide for improving experimental technique in the future
Power Analysis
 Sources of input for power analysis
– educated guesses derived from theory
– results of previous studies reported in literature
– pilot data
 Effect Size – several options
– minimal practical significance
– educated guess of the true underlying effect
 Questions of interest
– number of replications
– number of subsamples
– plot size
– detectable difference
Replications needed to estimate a mean
 For a single population, you may want to
determine the sample size needed to obtain a
given level of precision
Y  0
t
 Recall that
s2
r
 Rearrange the formula
r
t
2
2
α,df
2
s
d
r is the number of replications
t is the critical t with r-1 df
d is the limit of the confidence interval
r
Example
t
2
2
α,df
2
s
d
 In a preliminary trial, a sample of 5 replicates has a standard
deviation of 3.4 units
 You would like to conduct an experiment to estimate the mean
within two units of the true mean with a confidence level of 95%
Y2
But we have a problem here...
 We need to know how many reps in order to calculate
degrees of freedom
 So it is:
– pick
– plug
– adjust
2
t α,df s
r
2
d
Calculations
2
Given s = 3.4, d = 2,  = 0.05
Try starting with r = 5
Know your t table!
In Excel, =T.INV.2T(0.05,4)
 2.78 (2-tailed distribution)
In Kuehl, use =0.025  = 4  2.78 (Prob. of t  t,)
r  (2.782 * 3.42)/22 = 22.3
Additional iterations…
r
df
23
22
13
12
14
13
t
2.1
2.2
2.2
calculated r
12.4
13.7
13.5
r = 14
Detecting differences between means
2(t1  t 2 ) 
r
2
d
2
Where
2
may be expressed as
percentage of mean or
on actual scale
r = the number of replications
t1 = t at the significance level for the test
t2 = t at 2(1-P), where P is the selected probability
of obtaining a significant result (power)
(note that 1-P is , the probability of a type II error)
 = standard deviation (or CV%)
d = the difference that it is desired to detect
What is meaningful?
 If it can be established that the new is
superior to the old by at least some
stated amount, say 20%, then we will
have discovered a useful result
 If the experiment shows no significant
difference, we will be discouraged from
further investigation
 The experiment should be large enough
to ensure a meaningful difference.
For example:
r > 2(t1 + t2)2CV2 / d2
 We have two varieties to compare. A previous experiment with
these treatments found a CV of 11%
 How many replications would be needed to detect a difference of
20% with a probability of .85 using a 5% significance level test?
first pick 4 reps (6 df)
then test to see if 8 reps
(14 df) is correct
t1=T.INV.2T(0.05,6)=2.447
t2=T.INV.2T (0.30,6)=1.134
r > 2(2.145+1.076)2(11)2/(20)2
r > 6.28 ~ 7
r > 2(2.447+1.134)2(11)2/(20)2
r > 7.76 ~ 8
then test to see if 7 reps
(12 df) is correct
r > 6.44 ~ 7
Number of reps for the ANOVA
 The ANOVA may have more than two treatments, so we
must consider differences among multiple means
 Power curves are commonly used
– see Kuehl pg 63 for more information
 For this class, we will often get approximate estimates of
the number of reps needed using the formula for two
means
– use the error term from ANOVA (MSE) to estimate s2
– use the appropriate degrees of freedom for the MSE
• df = #treatments*(r-1) for a CRD
• df = (#treatments-1)(r-1) for an RBD
Power curve for ANOVA - example
SS treatments
 
# treatments * MSE
2
SAS Power Calculations
 PROC POWER
proc power;
Title 'determine #reps;
onewayanova test=overall
groupmeans =
50|56.75|58.25
alpha= 0.01
stddev = 7.7
power = 0.80
npergroup = .
;
run;
 PROC GLMPOWER
 SAS power and sample size application
– Stand alone desktop application that utilizes PROC
POWER and PROC GLMPOWER through a user
friendly interface
Determining Plot Size
Factors that affect plot size
 Type of crop
 Type of experiment
 Phase of the research program
 Variability of the experimental site
 Presence and nature of border effects
 Type of machinery to be used
 Number and type of treatments
 Land area available
 Cost
Factors Affecting Plot Size
Increasing Plot Size
Factor
Small plots
Soil variability
Uniform
Crop
Turf--Cereals
Large Plots
Heterogeneous
-- Row crops -- Trees --
Pasture
Late
Research phase Early
Experiment type Breeding -- Fertilizer -- Tillage -- Irrigation
Machinery
None
Research
Farm Scale
Effect on Variability
 Variability per plot decreases as plot size
increases
 But large plots may yield higher experimental
error because of larger more variable area for
the experiment
 Very small plots are highly variable because:
– Losses at harvest and measurement errors
have a greater effect
– Reduced plant numbers
– Competition and border effects are greater
How Plot Size Affects Variability
Variability
Size
Plot Size ‘Rule of Thumb’
 There are some lower limits:
– Should be large enough to permit
removal of borders with enough left over
to harvest and measure adequately
– Should be large enough to handle the
machinery needed
 Once the plots are large enough to be
handled conveniently, precision is
increased faster by increasing the
number of replications
Smith’s Soil Variability Index
V
Vx = xb
Where: V = variance of a unit plot
Vx = variance, on a per unit basis, of
plots formed from x adjacent units
x = plot size in multiples of adjacent
unit plots
b = index of soil variability
Smith’s Soil Variability Index
V
Vx = b
x
The index can vary from 0 to 1
When b=0 then Vx = V
When b=1 then Vx = V
x
This means there is no relation
between variance and plot size.
Adjacent plots are completely
correlated. Nothing can be
gained from larger plots.
This means that the units that
make up the plots are
independent of each other.
Increasing plot size will reduce
variance.
This is the case when the soil is
highly uniform.
This is the case when the soil
is highly heterogeneous.
Effect of plant sample size
However, can only obtain b=0 when plant
sample size is above a minimal level
 For large plants,
Variance of the mean
sample size may
be more critical
than soil
variability in
determining
optimal plot size
Effect of sample size on variance
8.0
7.5
7.0
6.5
6.0
5.5
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
0.0
0
5
10
15
20
25
30
number of plants
35
40
45
50
Variability Index Calculation
V
Vx = b
x
log Vx = log V - b log x
y = a + bx
a (intercept)
Log of variance
per plot
b (slope)
y
x
Log of plot size
Raw data from Uniformity Trial
488
488
457
440
325
347
424
426
448
476
416
387
470
446
410
388
303
315
406
392
359
403
365
462
387
384
454
430
445
382
418
426
393
420
333
328
491
457
436
388
369
393
408
346
347
490
440
319
353
352
385
389
389
365
479
438
396
450
343
390
354
389
374
428
415
393
402
394
411
419
396
426
438
296
503
420
361
368
386
375
321
435
449
438
470
427
446
414
473
455
367
510
337
384
376
437
402
375
403
452
442
455
410
435
441
348
371
375
976
897
672
850
924
803
916
798
618
798
762
827
771
884
827
844
813
661
948
824
762
754
837
759
705
774
754
917
846
733
743
802
808
796
830
822
734
923
729
761
756
887
897
860
928
877
721
813
777
855
897
845
789
746
Combine yields of plots
from adjacent rows
(2x1)
Combination x
1x1
1x2
1x3
1x6
2x1
2x2
2x3
2x6
3x1
3x2
3x3
3x6
4x1
4x2
4x3
4x6
6x1
6x2
6x3
1
2
3
6
2
4
6
12
3
6
9
18
4
8
12
24
6
12
18
Log(x)
0.0000
0.3010
0.4771
0.7782
0.3010
0.6021
0.7782
1.0792
0.4771
0.7782
0.9542
1.2553
0.6021
0.9031
1.0792
1.3802
0.7782
1.0792
1.2553
Vx
2177.18
1244.46
959.05
631.50
1447.50
920.00
706.10
428.81
1090.53
767.66
592.02
432.91
684.23
472.11
335.64
351.01
339.54
194.84
159.58
Log(Vx)
3.3379
3.0950
2.9818
2.8004
3.1606
2.9638
2.8489
2.6323
3.0376
2.8852
2.7723
2.6364
2.8352
2.6740
2.5259
2.5453
2.5309
2.2897
2.2030
Smith’s Index of Variability
Log of variance per plot
4
3.5
log Vx = 3.3261- 0.7026 log x
3
2.5
2
0.0000
0.4771
0.7782
0.9542
Log of plot size
log Vx = log V - b log x
y = a + bx
1.2553
Is there an easier way?
 Examination of a large number of data sets
indicated that a value of b=0.5 may serve as a
reasonable approximation.
“Finagles” constant b=0.5
Optimum Plot Size
 Optimum Size will either minimize cost for a fixed
variance or minimize variance for a fixed cost
 You must know certain costs:
T = K 1 + K 2x
Where: T = total cost per plot ($ or time)
K1= cost per plot ($ or time) that is
independent of plot size
K2= cost per plot ($ or time) that
depends on plot size
x = number of unit plots
Optimum Plot Size

Knowing those costs, Smith found the
optimum plot size to be:
xopt =
bK1
(1-b)K2
Where: T = total cost per plot ($ or time)
K1= cost per plot ($ or time) that is
independent of plot size
K2= cost per plot ($ or time) that
depends on plot size
x = number of unit plots
b = Smith’s index of soil variability
Optimum Plot Size
 So in our example, we found b to be 0.70 and our unit
plot area was 5m x 2m or 10m2
 Assume K1 = $3.00 and K2 = $5.00 for 10 m2
xopt =
2m
=
7m
bK1
(1-b)K2
(0.70)(3.00) = 1.4
(1-0.70)(5.00)
Area = (1.4)(10) = 14.0 m2
Convenient Plot Size
 Smith’s optimum plot size was based on soil
variability and cost.
 Hathaway developed a formula based on soil
variability, the size of difference to be detected,
and significance level of the test.
xb = 2(t1 + t2)2 2
rd2
Convenient Plot Size
xb = 2(t1 + t2)2 2
rd2
Where: x= number of units of plots of size x (plot size)
b= Smith’s coefficient of soil variability
t1= t at the significance level for the test
t2= t in the t table at 2(1-P), where P is the selected
probability of obtaining a significant result
2= variance from a previous experiment (may also use
the CV2)
r=
number of replications
d= difference to be detected (may be an absolute amount
or expressed as percentage of the mean)
For Example . . .
 A variety trial of 50 selections in a
randomized block design with three
blocks
 What size plot do we need to detect
a difference of 25% of the mean
with an 80% probability of obtaining
a significant result using a 5%
significance level test?
 Previous experiment had a CV of
11%
xb = 2(t1 + t2)2 CV2
rd2
error df = (3-1)(50-1) = 98
t1 = t0.05(98)) = 1.984
t2 = t0.40(98) = 0.845 {t 2(1-P) = t 2(1-0.80) = t0.40}
b = 0.70
CV = 11%
Then:
xb = 2(1.984 + 0.845)2(11)2 = 1936.78
(3)(25)2
1875
xb = 1.0330 or x.70 = 1.0330
x = 1.03301/.70 = 1.0475 basic units
Convenient Plot Size
 Since the basic plot has an area of 10 m2, the
required plot size would be:
(10.00)(1.0475) = 10.48 m2
2m
5.24 m
The Detectable Difference
 Using Hathaway’s formula for convenient plot
size and solving for d2, it is possible to
compute the detectable difference possible
when you know the number of reps and the
plot size.
d2 = 2(t1 + t2)2 2
rxb
An Alternative Example
 We want to find the difference that can be
detected 80% of the time at a 5% significance
level using a plot 2 m wide and 7 m long in 4
replications
d2 = 2(t1 + t2)2 CV2
rxb
 As before, the CV=11%, and b=0.70
d2 = 2(t1 + t2)2 CV2
rxb
d2 = 2(1.976 + 0.844)2(11)2
(4)(1.40.70)
= (2)(7.954)(121)
(4)(1.2656)
because 2m*7m/10m2=1.4
=
1924.90
5.0623
= 380.24
d = 19.50
Therefore, with a plot size of 2m x 7m using 4
replications, a difference of 19.50% of the mean
could be detected
35
30
Detectable
difference 25
(% of
20
Mean)
15
10
5
2
4
6
8
10
20
Plot Area (m2)
30
Replication
Using these tools