Transcript apr_ws732ef

EUROMET Project 732 « Towards new Fixed Points »
Workshop
COMPONENTS OF VARIANCE
MODEL FOR THE ANALYSIS OF
REPEATED MEASUREMENTS
Eduarda Filipe
St Denis, 23rd November 2006
Summary
•
•
•
•
2
Introduction
General principles and concepts
The Nested or Hierarchical Design
Application example of a three-stage nested
experiment.
Eduarda Filipe
Workshop Pr. 732
Introduction
• The realization of the International Temperature
Scale of 1990 - ITS90 requires that the
Laboratories usually have more than one cell for
each fixed point.
• The Laboratory may consider one of the cells as
a reference cell and its reference value,
performing the other(s) the role of working
standard(s), or the Laboratory considers its own
reference value as the average of the cells.
3
Eduarda Filipe
Workshop Pr. 732
Introduction
 In both cases, the cells must be regularly
compared and the calculation of the
uncertainty of these comparisons performed.
 A similar situation exists when the laboratory
compares its own reference value with the
value of a travelling standard during an interlaboratory comparison.
4
Eduarda Filipe
Workshop Pr. 732
Introduction
 These comparisons experiments are usually
performed with two or three thermometers to
obtain the differences between the cells.
 The repeatability measurements are performed
each day at the equilibrium plateau and the
experiment is repeated in subsequent days.
This complete procedure may also be repeated
some time after.
5
Eduarda Filipe
Workshop Pr. 732
Introduction
 The uncertainty calculation should take into
account these time-dependent sources of
variability, arising from short-term
repeatability, the day-to-day or ”medium
term” reproducibility and the long-term
random variations in the results;
6
Eduarda Filipe
Workshop Pr. 732
Introduction
 The Type A method of evaluation by the
statistical analysis of the data obtained from
the experiment is performed using the
Analysis of Variance (ANOVA) for designs,
consisting of nested or hierarchical sequences
of measurements.
7
Eduarda Filipe
Workshop Pr. 732
ANOVA definition
ISO 3534-3, Statistics – Vocabulary and Symbols
“Technique, which subdivides the total variation
of a response variable into meaningful
components, associated with specific sources of
variation”.
8
Eduarda Filipe
Workshop Pr. 732
General principles and concepts
 Experimental design is a statistical tool
concerned with planning the experiments to
obtain the maximum amount of information
from the available resources
 This tool is used generally for the improvement
and optimisation of processes
9
Eduarda Filipe
Workshop Pr. 732
General principles and concepts
• it can be used to test the homogeneity of a
sample(s)
• to identify the results that can be considered as
“outliers”
• to evaluate the components of variance
between the “controllable” factors
10
Eduarda Filipe
Workshop Pr. 732
General principles and concepts
• tool applied to Metrology for the analysis of
large amount of repeated measurements
• Measurements in:
– short-term repeatability
– day-to-day reproducibility
– long-term reproducibility
• permitting “mining” the results and to include
this “time-dependent sources of variability”
information at the uncertainty calculation.
11
Eduarda Filipe
Workshop Pr. 732
General principles and concepts
 In the comparison experiment to be described,
the factors are the standard thermometers, the
subsequent days measurements and the run
measurements.
 These factors are considered as random
samples of the population from which we are
interested to draw conclusions.
12
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. Definition
ISO 3534-3, Statistics – Vocabulary and Symbols
“The experimental design in which each level of
a given factor appears in only a single level of
any other factor”.
Purpose of this model
To deduce the values of component
variances that cannot be measured
directly.
13
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
• The factors are hierarchized like a “tree” and
any path from the “trunk” to the “extreme
branches” will find the same number of
nodes.
The analysis of each factor is done with:
“Random Effects One-way ANOVA”
or components-of-variance model, nested in
the subsequent factor.
14
Eduarda Filipe
Workshop Pr. 732
Nested design
1
1
A
1
1
1
A
2
1
1
B
1
1
1
B
2
M=2
Measurements
T=2
Factor T
D = 10
Factor D
P=2
1
2
Factor P
Observer
15
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
For one factor with a different levels taken
randomly from a large population,
yij  M i  e ij  m  t i  e ij
(i  1, 2,..., a
and
j  1, 2,..., n)
where Mi is the expected (random) value of the group of
observations i, m the overall mean, ti the i th group
random effect and eij the random error component.
16
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
• For the hypothesis testing, the errors and the factor-levels
effects are assumed to be normally and independently
distributed, respectively eij ~ N (0, s 2) and ti ~ N (0, st2).
• The variance of any
observation y is composed by
the sum of the variance
components
17
s  st  s
2
y
Eduarda Filipe
2
2
Workshop Pr. 732
The nested or hierarchical design. General model
• The test is unilateral and the
hypotheses are:
H 0 : st  0
2
H1 : s t2  0
• That is, if the null hypothesis is true, all factor-effects are
“equal” and each observation is made up of the overall
mean plus the random error eij ~ N (0, s 2).
18
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
The total sum of squares (SST), a measure of total variability
in the data, may be expressed by:
a
n
 ( y
i 1 j 1
ij
a
n


a
a
n
 y )  ( yi  y )  ( yij  yi )  n ( yi  y )   ( yij  yi ) 2 
2
2
i 1 j 1
a
2
i 1
i 1 j 1
n
 2  ( yi  y )( yij  yi )
i 1 j 1
SSFactor
SSE
=0
SSFactor sum of squares of differences between factor-level averages and
the grand average or a measure of the differences between factor-level
SSE a sum of squares of the differences of observations within a factorlevel from the factor-level average, due to the random error
19
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
Dividing each sum of squares by the respectively degrees of
freedom, we obtain the corresponding mean squares (MS):
MSFactor - mean square between factorlevel is:
n a
2
MS Factor 
(
y

y
)
 i
a  1 i 1
a
MS Error 
•an unbiased estimate of the
variance s 2, if H0 is true
n
2
(
y

y
)
 ij i
i 1 j 1
a (n  1)
s
2
•or a surestimate of s 2, if H0 is false.
MSError - mean square within factor (error)
is always
20
• the unbiased estimate of the
variance s 2.
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
To test the hypotheses, we use the statistic:
MS Factor
F0 
~ F , a 1, a ( n 1)
MS Error
F is the Fisher
sampling distribution
with a and a x (n -1)
degrees of freedom
If F0 > F, a-1, a(n-1) , we reject the null hypothesis.
and conclude that the variance st2 is significantly different
of zero.
21
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
The expected value of the MSFactor is:
 n a
2
2
2
E ( MS Factor )  E 
(
y

y
)

s

n
s

i
t

 a  1 i 1

and the variance component of
the factor is obtained by:
22
2
E
(
MS
)

s
Factor
s t2 
n
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
Considering now the three level nested design of the
figure the mathematical model is:
yrdtm  m  P p  D d  Tt  e rdtm
yrdtm (rdtm) th observation
m
overall mean
Pr
P th random level effect
Dd D th random level effect
Tt T th random level effect
ertdm random error component.
23
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
The errors and the level effects are assumed to be
normally and independently distributed, respectively with
mean zero and variance s 2 and with mean zero and
variances sr2, sd2 and st2.
The variance of any observation is composed by the sum
of the variance components and the total number of
measurements N is obtained by the product of the
dimensions N = P × D × T × M.
24
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
The total variability of the data can be expressed by
2
2
(
y

y
)

DTM
(
y

y
)

 pdtm

p
p
d
t
m
p
  TM ( y pd  y p ) 2   M .( y pdt  y pd ) 2 
p
d
p
  ( y pdtm  y pdt )
p
d
t
d
t
2
m
or
SST  SS P  SS D P  SST DP  SS E
25
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
• Dividing by the respective degrees of freedom
P –1 P  (D -1) P D (T -1) P D  T  (M -1)
• Equating the mean squares to their expected
values
• Solving the resulting equations
• We obtain the estimates of the components of the
variance.
26
Eduarda Filipe
Workshop Pr. 732
The nested or hierarchical design. General model
 SS P 
2
2
2
2
E ( MS P )  E 

s

M
s

TM
s

DTM
s
T
D
P

P

1


 SS D P 
2
2
2
E ( MS D P )  E 

s

M
s

TM
s

T
D
P
(
D

1
)


 SST D P 
2
2
E ( MST D P )  E 

s

M
s

T
PD
(
T

1
)




SS E
2
E ( MS E )  E 

s

PDT
(
M

1
)


27
Eduarda Filipe
Workshop Pr. 732
Example of the comparison of two thermometric Water
triple point cells in a three-stage nested experiment
Two water cells, JA
and HS
Two standard
platinum
resistance
thermometers
(SPRTs) A and B.
A – water vapour
B – water in the liquid phase
C – Ice mantle
D – thermometer (SPRT) well
t = 0,01 ºC
28
Eduarda Filipe
Workshop Pr. 732
Four measurement
differences obtained daily
with the two SPRTs
This set of measurements
repeated during ten
consecutive days.
And two weeks after a
complete run was repeated
Run 2/Plateau 2.
29
Eduarda Filipe
Workshop Pr. 732
Example of the comparison of two thermometric Water
triple point cells in a three-stage nested experiment
 In this nested experiment, are considered the effects
of:
 Factor-P from the Plateaus (P = 2)
 Factor-D from the Days (D = 10) for the same Plateau
 Factor-T from the Thermometers (T = 2) for the same
Day and for the same Run
 the variation between Measurements (M = 2) for the
same Thermometer, the same Day and for the same
Plateau or the residual variation.
30
Eduarda Filipe
Workshop Pr. 732
Example of the comparison of two thermometric Water triple
Measurements
point
cells in a three-stage nested experiment
(mK)
Days SPRTs
7
6
5
4
3
2
1
A21
B21
A22
B22
A23
B23
A24
B24
A25
B25
A26
B26
A27
B27
A28
B28
A29
B29
A20
Eduarda Filipe
B20
8
103
93
78
68
126
96
88
78
97
118
98
80
100
108
80
67
92
84
96
63
9
93
93
73
118
91
130
93
118
117
118
108
80
105
128
110
77
97
104
106
68
Plateau 2
5
6
A11
B11
A12
B12
A13
B13
A14
B14
A15
B15
A16
B16
A17
B17
A18
B18
A19
B19
A10
B10
2
10
31
10
9
8
7
Plateau 1
4
3
2
1
1
107
117
74
54
123
48
103
83
114
64
72
102
74
119
70
105
68
58
104
104
93
83
70
100
77
89
104
84
62
112
91
51
68
63
69
68
95
88
Workshop Pr. 732
113
78
Example of the comparison of two thermometric Water triple
point cells in a three-stage nested experiment
Schematic representation of the observed temperatures
differences
32
Eduarda Filipe
Workshop Pr. 732
Example of the comparison of two thermometric Water triple
point cells in a three-stage nested experiment
Analysis of variance table
Source of variation
Sum of squares
Degrees of
freedom
Mean square
Expected value of mean square
F0
5,7296 s 2 + 2s T 2 + 4 s D 2 + 40 s R 2
Plateaus
2187,96
1
2187,96
Days
6873,64
18
381,87
1,0032 s 2 + 2s T 2 + 4s D 2
Thermometers
7613,19
20
380,66
1,0314 s 2 + 2s T 2
Measurements
Total
14762,50
31437,29
40
79
369,06
s2
F0 values compared with the critical values Fn1,n2 ( = 5%)
F0,05, 1, 18 = 4,4139 for the Plateau/Run effect
F0,05, 18,20 = 2,1515 for the Days effect and
33
F0,05, 20, 40 =1,8389 for the Thermometers effect.
Eduarda Filipe
Workshop Pr. 732
Example of the comparison of two thermometric Water triple
point cells in a three-stage nested experiment
 F0 values are inferior to the F distribution for the
factor-days and factor-thermometers so the null
hypotheses are not rejected.
 At the Plateau factor, the null hypothesis is
rejected for  = 5% so a significant difference
exists between the two Plateaus
34
Eduarda Filipe
Workshop Pr. 732
Example of the comparison of two thermometric Water triple
point cells in a three-stage nested experiment
• equating the mean squares to their expected
values
• we can calculate the variance components
and
• include them in the budget of the components
of uncertainty
35
Eduarda Filipe
Workshop Pr. 732
Example of the comparison of two thermometric Water triple
point cells in a three-level nested experiment
Uncertainty budget for components of uncertainty
evaluated by a Type A method
Components of uncertainty
Type A evaluation
Variance (mK)
Plateaus
Days
Thermometers
Measurements
Total
36
2
45,15
0,30
5,80
369,06
420,32
Eduarda Filipe
Standard deviation (mK)
6,7
0,5
2,4
19,2
20,5
Workshop Pr. 732
Example of the comparison of two thermometric Water
triple point cells in a three-level nested experiment
• These components of uncertainty, evaluated by
Type A method, reflect the random components
of variance due to the factors effects
• This model for uncertainty evaluation that takes
into account the time-dependent sources of
variability it is foreseen by the GUM
37
Eduarda Filipe
Workshop Pr. 732
Example of the comparison of two thermometric Water
triple point cells in a three-level nested experiment
Comparison of different approaches
• Components of uncertainty evaluated by a
Type A method
– Nested structure: uA= 20,5 mK
– Standard deviation of the mean of 80
measurements uA= 2,2 mK
– Standard deviation of the mean 1st Plateau (40
measurements) uA= 2,8 mK (the 2nd plateau
considered as reproducibility and evaluated by a
type B method)
38
Eduarda Filipe
Workshop Pr. 732
Concluding Remarks
 The plateaus values continuously taken where
previously analyzed in terms of its normality.
 The nested-hierarchical design was described as a tool
to identify and evaluate components of uncertainty
arising from random effects.
 Applied to measurement, it is suitable to calculate the
components evaluated by a Type A method standard
uncertainty in time-dependent situations.
39
Eduarda Filipe
Workshop Pr. 732
Concluding Remarks
 An application of the design has been used to
illustrate the variance components analysis in a
three-factor nested model of a short, medium and
long-term comparison of two thermometric fixed
points.
 The same model can be applied to other staged
designs, easily treated in an Excel spreadsheet.
40
Eduarda Filipe
Workshop Pr. 732
REFERENCES
41
1.
BIPM et al, Guide to the Expression of Uncertainty in Measurement (GUM), 2nd ed., International
Organization for Standardization, Genève, 1995, pp 11, 83-87.
2.
ISO 3534-3, Statistics – Vocabulary and Symbols – Part 3: Design of Experiments, 2nd ed.,
Genève, International Organization for Standardization, 1999, pp. 31 (2.6) and 40-42 (3.4)
3.
Milliken, G.A., Johnson D. E., Analysis of Messy Data. Vol. I: Designed Experiments. 1st ed.,
London, Chapmann & Hall, 1997.
4.
Montgomery, D., Introduction to Statistical Quality Control, 3rd ed., New York, John Wiley & Sons,
1996, pp. 496-499.
5.
ISO TS 21749 Measurement uncertainty for metrological applications — Simple replication and
nested experiments Genève, International Organization for Standardization, 2005.
6.
Guimarães R.C., Cabral J.S., Estatística, 1st ed., Amadora: Mc-Graw Hill de Portugal, 1999, pp.
444-480.
7.
Murteira, B., Probabilidades e Estatística. Vol. II, 2nd ed., Amadora, Mc-Graw Hill de Portugal,
1990, pp. 361-364.
8.
Box, G.E.P., Hunter, W.G., Hunter J.S., Statistics for Experimenters. An Introduction to Design,
Data Analysis and Model Building, 1st ed., New York, John Wiley & Sons, 1978, pp. 571-582.
9.
Poirier J., “Analyse de la Variance et de la Régression. Plans d’Experience”, Techniques de
l’Ingenieur, R1, 1993, pp. R260-1 to R260-23.
Eduarda Filipe
Workshop Pr. 732
Thank you!
42
Eduarda Filipe
Workshop Pr. 732