Transcript Definitions of Statistical Power

Phlebotomy Training for M-III
Students: Statistical Analysis of
Test Results
Richard A. McPherson, M.D., M.S.
Phlebotomy Training 2001-2008
• Exercise offered to third year medical
students as part of orientation every year
since 2001.
• This is the third year that phlebotomy
training was mandatory.
• Other exercises offered in IV/Foley
catheter placement.
Numbers of Students Submitting
Blood Specimens Each year
2001
2002
2003
2004
2005
2006
2007
2008
Total
103
83
102
87
98
150
147
134
904
150
100
50
2001 2002 2003 2004 2005 2006 2007 2008
Count
•
•
•
•
•
•
•
•
•
Phlebotomy Training 2008
• Wednesday, July 23, 2008 in three separate
sessions held in the Medical Sciences Building at 1,
2 and 3 PM.
• A total of 153 students attended the exercise in
which each student collected two tubes of blood on
a partner.
• Specimens were successfully collected from 134
students and submitted to the laboratory for simple
chemical and hematological measurements.
• The students’ own results were provided to them
with a unique identifying number known only to
each individual student.
Age
Frequencies
Level
20
39 Years
37 Years
32 Years
31 Years
30 Years
29 Years
28 Years
27 Years
26 Years
25 Years
24 Years
23 Years
22 Years
10
Count
30
22 Years
23 Years
24 Years
25 Years
26 Years
27 Years
28 Years
29 Years
30 Years
31 Years
32 Years
37 Years
39 Years
Total
13 Levels
Count
Prob
1
15
33
31
21
12
6
6
3
2
2
1
1
134
0.00746
0.11194
0.24627
0.23134
0.15672
0.08955
0.04478
0.04478
0.02239
0.01493
0.01493
0.00746
0.00746
1.00000
Gender
Frequencies
60
20
Count
40
Level
Count
F
M
Total
66 0.49254
68 0.50746
134 1.00000
2 Levels
F
M
Prob
Race
Frequencies
Level
50
Count
75
Asian-Pacific
Black
Hispanic
Other
Unknow n
White
Total
6 Levels
25
Asian-Pacific
Black
Hispanic
Other
Unknow n White
Count
Prob
30
4
1
3
7
89
134
0.22388
0.02985
0.00746
0.02239
0.05224
0.66418
1.00000
Specimens by Gender
Male
Female
Total
68
66
134
Hematology 67
62
129
Chemistry
Reasons to Test Student Specimens
• Courtesy to students for participation
• Teach interpretation of laboratory results (i.e.,
reference ranges) to students
• Evaluate current reference ranges for
appropriateness
• Discover previously unknown medical
condition
– Students could opt out from testing blood.
• Demonstrate statistical applications
Goal 1. Descriptive Statistics
• Measure of Central Tendency
– Mean
– Median
– Mode
• Measure of Dispersion
– Standard deviation
– Interquartile range (25th to 75th percentile range)
Before you get going with the analysis,
LOOK AT YOUR
DATA!!!**#$!@$%
Strategies for Dealing with
Non-normal distributions
1. Check for outliers
– Extreme cases from errors of recording or
entering data
– Individuals that clearly do not belong in the
population sampled.
Example: Checking for Outliers
Four methods evaluated for Erythrocyte
Mean Cell Volume on 131 blood specimens
1100
120
120
110
110
100
90
80
100
1000
900
700
500
400
80
70
70
110
800
600
90
120
100
90
80
300
200
70
100
60
60
0
60
Method 1 vs Method 3
Method 3 clearly has data entry errors of
1000.0 and 18.9
Quantiles
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
Quantiles
123.00
123.00
108.00
98.16
93.40
88.80
83.80
74.64
64.98
58.90
58.90
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
1000.0
1000.0
117.3
99.5
93.6
89.5
84.4
74.0
64.4
18.9
18.9
Method 3 edited to remove incorrect
values; more normal in distribution
130
120
110
100
120
120
110
110
100
100
90
90
80
90
120
110
100
90
80
80
80
70
70
70
60
70
60
50
60
60
Outlier Trimming
• Remove upper and lower percentiles of
data such as 0.5% to use data between
0.5 percentile and 99.5 percentile
• Eliminates what is most likely to be
severely atypical information or data entry
error
Serum ALT values trimmed for
central 99 percent
0 100 300 500 700 900 1100 1300 1500 1700 1900 2100 2300 2500 2700
10 20 30 40 50 60 70 80 90100 120 140 160 180 200 220
Strategies for Dealing with
Non-normal distributions
2. If results are skewed, transform to a scale
that is more nearly normal by logarithm,
square root, etc.
2
.95
.90
1
.75
.50
0
.99
2
.95
.90
1
.75
0
.50
.25
2
.95
.90
1
.75
.50
-1
.10
.05
.10
.05
-2
-2
.01
-2
.01
.01
-3
-3
2.5
3
3.5
4
4.5
0
.25
-1
.10
.05
120
.99
.25
-1
10 20 30 40 50 60 70 80 90 100
3
Normal Quantile Plot
.99
3
Normal Quantile Plot
3
5
-3
3
4
5
6
7
8
9
10 11 12
Normal Quantile Plot
ALT, Log ALT, SQRT ALT
2008 Student Hemoglobin Distribution
Quantiles
20
10
5
11
12
13
14
15
Hemoglobin (g/dL)
16
17
Count
15
100.0% maximum
99.5%
97.5%
90.0%
75.0%
quartile
50.0%
median
25.0%
quartile
10.0%
2.5%
0.5%
0.0%
minimum
17.3
17.3
16.9
16.2
15.1
14.3
13.4
12.6
11.6
10.8
10.8
Moments
Mean
Std Dev
Std Err Mean
upper 95% Mean
low er 95% Mean
N
14.3
1.32
0.12
14.5
14.1
129
.99
2
.95
.90
1
.75
.50
0
Normal Quantile Plot
3
.25
.10
-1
Assessment of Normality of
Distribution:
Normal Quantile Plot
.05
-2
.01
-3
20
10
5
11
12
13
14
15
Hemoglobin (g/dL)
16
17
Count
15
Parameters: Mean
• Formula for mean
n
x1  x 2  x3  ...x n
mean  x 

n
x
i 1
n
i
Parameters: Variance
• Formula for variance (variances are additive)
n
variance  s 
2
 x
i 1
 x
2
i
n 1
Parameters: Standard Deviation
• Formula for Std Dev
n
s s 
2
 x
i 1
i
 x
n 1
2
Goal 2. Comparative Statistics
• Parametric: uses a formula to describe
distribution
– Student t-test
– One-way analysis of variance
• Non-parametric: assumes no particular
distribution
– Wilcoxon rank-order test
Comparison of Hgb in Females vs Males
30
20
15
Count
Count
20
10
10
5
10
11
12
13
14
15
Hemoglobin in Females
16
17
18
10
11
12
13
14
15
Hemoglobin in Males
16
17
18
Assumptions for Use of t test
• Similar numbers in each group
• Similar variances in each group
• Individuals in each group are independent of one
another (random selection, non-biased)
• Values are normally distributed.
You want to make a conclusion (inference) that is
generalizable to a larger population than that which
constitutes your sample. Accordingly the sample
should be representative of the population.
Student’s t Test
• Student was the pseudonym for William
Sealy Gossett [1876-1937], who developed
statistical methods for solving problems in a
brewery where he worked (Guinness in
Dublin). He published his work in 1908 in the
journal Biometrika. He did not publish under
his own name so the nature of his work for
optimizing production conditions could
remain a trade secret.
Student’s t Test
• Principle: Compare the difference between
means to the amount of noise (scatter) in
measurements to judge if the difference in
means could be due to chance alone.
signal Difference between group means
t

noise
Variabilit y of groups

(mean of A) - (mean of B)
variance of A variance of B

number of A number of B
Comparative Statistics:
Student’s t Test
17
16
HGB
15
14
13
12
11
F
M
Gender
• Hemoglobin mean
values: females, 13.3
g/dL, males15.2 g/dL
• Are these mean
values truly different
from one another?
• Student t-value of
10.873, df=127, pvalue <0.0001, or less
than once in 10,000
times by chance
alone.
Confidence Intervals on Means of the
Groups being Compared: no overlap
Level
Number Mean
Std
Error
Lower Upper
95% CI 95% CI
Female 62
13.33
0.1209 13.093 13.572
Male
15.16
0.1163
67
14.927 15.387
Comparison of WBC in Females vs Males
13
12
11
WBC
10
9
8
7
6
5
4
3
F
M
Gender
• WBC mean values:
females, 7.1, males 6.6
• Are these mean values
truly different from one
another?
• Student t-value of
1.787, df=127, p-value
= 0.0764, or about 1 in
13 times by chance
alone.
If we suspect a gender-related difference,
how can we show it to have statistical
significance? Adjustables:
• Distance between means; use a more
discriminating instrument, method, principle
of measurement.
• Noise level: use a more precise method with
less scatter in measurement
• Accept a higher type I error rate.
• Number of observations: increase N
Goal 3. Power Analysis
Do a power analysis to find N at which the
conditions of a pilot study predict
significance (at the level a specified) could
be achieved if your estimates of mean
difference (delta) and variance (noise
level) are accurate.
Definitions of Statistical Power
•
The likelihood of finding a statistically significant
difference when a true difference exists.
Online Learning Center
•
The power of a statistical test is the probability
that the test will reject a false null hypothesis (that
it will not make a Type II error). As power
increases, the chances of a Type II error decrease.
The probability of a Type II error is referred to as
the false negative rate (β). Therefore power is
equal to 1 − β. Wikipedia
Za
Formula for calculating sample size
4Z a  Z b  s
2
2N 
d
2
2
• N = number of subjects in each group
• Za = parameter for chance of finding a difference by
chance alone (usually set to 5 percent) = 1.96
• Zb = parameter indicating power of finding a
difference (usually set to 80 percent) = 0.84
 d = the difference between group means (usually
obtained from a pilot study or by an informed guess
 s = common SD for both groups
Power Analysis for WBC vs Gender
Power
Alpha Sigma
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
0.0500 1.586712
Delta
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
0.249608
Number
129
139
149
159
169
179
189
199
209
219
229
239
249
259
269
279
289
299
309
319
329
339
349
Power
0.4260
0.4530
0.4792
0.5046
0.5293
0.5530
0.5760
0.5981
0.6193
0.6397
0.6593
0.6780
0.6959
0.7130
0.7294
0.7449
0.7597
0.7738
0.7872
0.7999
0.8119
0.8233
0.8342
Power Plot
1.00
0.80
Power
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
0.60
0.40
0.20
0.00
100
150
200
250
300
350
Number
Gender
Alpha=0.05 Sigma=1.58671 Delta=0.24961
Need a total of 320 subjects to show
significance at 0.05 level
with 80% power
Goal 4. How to fit a line
• Least squares regression minimizes
square of (vertical) distances from data
points to line (best fit)
• y = ax + b
50
48
46
HCT
44
42
40
38
36
34
11
12
13
14
HGB
15
16
17
Plot of residuals shows
homoscedasticity (uniformity of data
over entire range)
Residual
3
1
-1
-3
11
12
13
14
HGB
15
16
17
Hct = 8.890 + 2.284xHgb
• R2 = 0.9069, so >90% of variation in Hct is
predicted by Hgb
• Intercept = 8.890
– t = 9.55, p<0.0001
• Slope = 2.284
– t = 35.17, p<0.0001
• Is this a great fit or what?
• What if Hgb = 0? Then Hct should = 0, not
8.890
So force the line through the
origin.
Hct = 0 + 2.901xHgb
50
48
46
HCT
44
42
40
38
36
34
11
12
13
14
HGB
15
16
17
Normal Variants?
100
95
90
MCV
85
80
75
70
65
60
4
4.5
5
5.5
RBC
6
6.5
Super Difference by Gender
11
10
9
Uric Acid
8
7
6
5
4
3
2
1
F
M
Gender
Gender Different Analytes
147
146
32
30
28
Carbon Dioxide
Sodium
145
144
143
142
141
140
139
138
26
24
22
20
18
137
136
16
F
M
Gender
F
M
Gender
Gender Different Analytes
35
1.5
1.4
30
1.3
1.2
1.1
Creatinine
BUN
25
20
15
10
1
0.9
0.8
0.7
0.6
0.5
0.4
F
M
Gender
F
M
Gender
Gender Different Analytes
2.8
2.7
11
Magnesium
Calcium
10.5
10
9.5
2.6
2.5
2.4
2.3
2.2
9
2.1
2
1.9
8.5
1.8
1.7
F
M
Gender
F
M
Gender
Gender Different Analytes
5.5
5
5
4.5
Globulins
Albumin
4.5
4
3.5
4
3.5
3
3
2.5
2.5
2
F
M
Gender
F
M
Gender
Gender Different Analytes
90
120
70
100
60
80
ALT
AST
80
50
40
60
30
40
20
20
10
0
0
F
M
Gender
F
M
Gender
Gender Different Analytes
130
2.5
120
110
2
90
Bili Total
Alk Phos
100
80
70
1.5
1
60
50
0.5
40
30
0
F
M
Gender
F
M
Gender
Variation over Time: Platelets
600
500
PLT
400
300
200
100
2001
2002
2003
2004
2005
Year
2006
2007
2008
WBC
Variation over Time: WBC
10
2001 2002 2003
2004 2005
Year
2006
2007
2008
Glucose 2008: postprandial (1 to 4 PM)
40
20
50
60
70
80
90 100 110 120 130
Glucose (mg/dL)
Count
60
Glucose over the Years
Glucose
300
200
100
2001 2002 2003 2004 2005
Year
2006
2007
2008
Acknowledgements
Pathology faculty
• Roger Riley, MD
• Kim Sanford, MD
• Samuel B. Hunter, MD
Resident
• Saud Rahman, MD
Nurse
• Jennifer Anderson, RN
Phlebotomists
• Linda Walker, MT, Supervisor
• Charity Delacruz, CPT
• Rogelio Inocencio, MLT
• Jean Merritt, CPT
• Shirley White, CPT
Test ordering, set-up, and
processing
• Caroline Greene, MT
• Susan Handwerk
• June Lee, MT, Evening
Supervisor
• Kristina Nilsen, MT
• Millicent Smith, MT
• Karen Tinsley, MT