#### Transcript Module Seven: Quantifying and Presenting Uncertainty

```Module Seven : Quantifying uncertainties
In the previous modules, we have discussed a variety of numerical and
graphical methods for handling one and two sample problems in Interlaboratory studies. In this module, we will discuss some general rules to
operate uncertainties.
•How to quantify the uncertainty of a system of operations.
•How to combine uncertainties of the same characteristics measured different
individuals.
•When no information about the relationship of variables
•When variables are independent
•When variables are correlated
•How to determine uncertainty based on the probability distributions.
•Uncertainty from linear least squares calibration.
•Measuring Type B uncertainty
1
A system of components involving the measurement of uncertainty
In many practical cases, we are interested in measuring the
combined uncertainty of the entire system which consists of
several uncertainty measurements.
Input
X1
X2
Xk
Output
The system consists of k components. Each component, we measure its uncertainty:
Where
xi
xi  Uxi
is the best estimate for the component i. Uxi is the measurement uncertainty.
The system is a function of
x1 , x2 ,..., xk
,denoted by
f ( x1 , x2 ,..., xk )
2
Four Questions involving quantifying the
uncertainty of the system
Q1: When measuring the uncertainty of one variable, x. It is
often that f(x) is the interest of measurement. How do we extend
the measurement uncertainty to f(x), if the measurement of x
with uncertainty is x  U x ?
Q2: How to quantify the uncertainty of f ( x1 , x2 ,..., xk )
Our goal is to quantify f ( x1  Ux1 , x2  Ux2 ,..., xk  Uxk )
in terms of
f ( x1 , x2 ,..., xk )  U f ( x , x ,..., x )
1
2
k
Q3: If there are n individuals measure the same component.
Each obtains a measurement x j  Ux j
How do we combine these measurements and uncertainties ?
3
For each measurand, the measurement of the measured property
usually obeys certain probability distribution. For example, a
normal curve has been used for a variety of continuous variables.
These distributional characteristics allow us to make a confidence
interval estimate of the measurement with an intended level of
confidence.
Q4: When the distribution characteristic is applied to
measuring the uncertainty, how to quantify the uncertainty
based on the distribution characteristic of the measured
property?
4
Q1: When measuring the uncertainty of one variable, x. It is
often that f(x) is the interest of measurement. How do we extend
the measurement uncertainty to f(x), if the measurement of x
with uncertainty is x  U x ?
Some common functions of x variable are
Case 1: f(x) = axn
Case 2: exp(x)
We consider the general form of f(x) and apply to these special cases.
f(x+Ux)
f(x)
f(x-Ux)
f(x)
Uf
f(x)
Uf
x-Ux
f(x-Ux)
f(x+Ux)
x x+Ux
x-Ux x x+Ux
5
A fundamental calculus approximation asserts the following fact , given f(x) is
monotone increasing, for Ux sufficiently small:
f ( x  U x )  f ( x) 
df
Ux
dx
df
This gives the uncertainty of f(x): U f  U x , when f(x) is an increasing
dx
function of x.
Similarly, when f(x) is decreasing, the uncertainty of f(x) is
Uf 
df
Ux
dx
Combining these two situations, the measurement of f(x) with uncertainty is
given by:
df
f ( x) |
|Ux
dx
Case 1: f(x) = xa : f(x) with uncertainty is x a  nx a 1U x  x a (1 | a | U x )
| x|
Case 2: f(x) = exp(x): f(x) with uncertainty isexp( x)  exp( x)U x  exp( x)(1  U x )
ln( x ) 
Case 3: f(x) = ln(x): f(x) with uncertainty is
Case 4: f(x) = cos(x): f(x) with uncertainty is
Case 5: f(x) = ax: f(x) with uncertainty is
Ux
x
cos( x) | sin x | U x
ax  | a | U x
6
Examples:
1.
One measured the thickness of 100 sheets of papers many times and obtain
the thickness:
2.50  .12cm
What is the thick ness of one sheet of paper?
Ans: f(x) =x/100. The thickness of one sheet is (2.5 /100)  (.12 /100)  .025  .0012cm
2.
One measured the radius of a circle and obtain the measurement: 4.5  .1cm
What is the measurement of the area of the circle?
Ans: f(x) = pr2. The area is p r 2 (1  2 U r )  p (4.5) 2 [1  2(.1) / 4.5]  20.25p  .9p
r
Class Hands-on activity
Using the hands-on activity of ‘drawing 2 cm of line segment’ data to compute the
uncertainty of your draws using the 10 data points, that is compute the sample
s.d. Then, estimate the uncertainty for drawing one cm line segment.
7
Q2: How to quantify the uncertainty of
Our goal is to quantify
f ( x1 , x2 ,..., xk )
f ( x1  Ux1 , x2  Ux2 ,..., xk  Uxk )
f ( x1 , x2 ,..., xk )  Uf ( x1 , x2 ,..., xk )
in terms of
We begin with a simple system of two components, x and y
The measurements and uncertainties are
x  Ux and y  Uy
The basic function of f(x,y) for describing the system are
Case 1: x + y
Case 2: x-y
Case 3: xy
Case 4: x/y
Any function that is a combination of these operations can be
propagated.
8
Conditions of measuring uncertainty when combining variables:
(1) No prior information about the components themselves or their relationship:
For this situation, the maximum combined uncertainty may be more
appropriate.
(2) When variables are independent, that is the process of measuring one
component is not related to the process of measuring other processes: For this
situation, the independence property can be applied to reduce the combined
uncertainty.
(3) When distribution of the variables are known or can be approximated
reasonably by some probability distributions, the measurement uncertainty of
the component is estimated by standard error of the best estimate, and a
probability of confidence can be obtained to describe the measurement
uncertainty. Type A and Type B uncertainty are defined accordingly.
(4) It often happens in a practical situation, uncertainty occurs not only randomly,
but also systematically. One needs to be careful about the existence of
systematic error, and make an effort to estimate this component of error, when
ever possible. When reporting uncertainty involving both random error and
systematic error, it is a good practice to provide separate presentation as well
9
as combined uncertainty.
We have introduced Youden Plots and some numerical measures for measuring
systematic and random errors. The estimation of systematic error requires some
additional effort by conducting appropriate experiments that are specifically
designed for estimating a suspected systematic error component. The analysis
often depends on the design of the experiment. Some commonly used designs and
their analysis will be discussed later. In this following we will focus on measuring
measurement uncertainty for condition (1) and (2).
Case 1: f(x,y) = x+y
Given the uncertainty for each variable: x  Ux and y  Uy
We are looking for ( x  y )  U ( x  y )
Under Condition 1: Maximum possible measurement for (x+y) is
x+Ux + (y+Uy) = (x+y) +(Ux+Uy)
The minimum possible measurement is (x+y) –(Ux+Uy)
Therefore, the uncertainty for the sum is :
( x  y )  (Ux  Uy )
10
Case 1: Under Condition 2 : x and y are independent
Independence of x and y has the following geometric property for the uncertainty:
Uy
Ux
The uncertainty of x+y, due to independence, is given by
U(x+y) =
U 2x U 2 y
Case 1: Condition 3: x and y follow a certain probability distribution. The
measurement of property x is the best estimate from the sample information (Type
A) or from the physical property or prior knowledge (Type B). At first, step, we
determine the best estimate for the characteristic by observing n measurements.
The uncertainty of the variable x is estimated by sample standard deviation. In
many cases, we are interested in estimating the unknown average of the
characteristic. The best estimate is the sample mean,
x
The uncertainty of
And

x is the Standard Error
of
x , which is  / n
is estimated by the sample standard deviation, sx.
11
The estimate of the average characteristic for variable x and its uncertainty is
x
sx
n
Similarly, the best estimate of the average for variable y and its uncertainty is
y
sy
n
Assuming that the two samples are chosen at random , and x
and y are independent.
Therefore, the uncertainty of the average of (x+y) is
( x  y )  ( sx2 / n)  ( s y2 / n)
12
Case 2: f(x,y) = x-y:
Under Condition 1: the uncertainty is
Under Condition 2: the uncertainty is
( x  y )  (Ux  Uy )
( x  y )  U x2  U y2
Under Condition3: the best estimate and the uncertainty
is:
( x  y )  ( s 2 / n )  ( s 2 / n)
x
y
13
Case 3: f(x,y) =xy
Under Condition 1: x is measured by x  U x = x(1+
Ux
)
|x|
Therefore, the measurement of f(x,y) = xy is
x (1  U x ) * y (1  U y ) = xy (1  U x ) (1  U y )
| x|
| y|
| x|
Uy
Uy
U
U
x
= xy(1+
+ | y| + x
)
|x| | y|
|x|
U f Ux U y


Therefore, | f | | x | | y |
| y|

Uy
Ux
(xy)(1+
+
) = xy (1+ U xy )
|x| | y|
| xy |
Case 3, Under Condition 2: X and Y are independent:
The fractional uncertainty of f(x,y) = xy is given by
2
 Ux   U y 
The measurement of x/y with uncertainty is (xy) [1  
 ]
 
x
y

 

2
14
(1  U x / | x |)
Case 4: f(x,y) = x/y: Under Condition x1:
y(1  U y / | y |)
The largest measurement of f(x,y)= x/y =
x(1  U x / | x |)
y(1  U y / | y |)
The smallest measurement of f(x,y)= x/y =
Using the Binomial Expression: 1/(1-a) = 1+a+a2+a3+ …. When a is small, we
can approximate
 1/(1-a) = 1+a. The largest
 measurement of x/y has the form of
(1+b)/(1-a)
x
U
U
y
[1  x  1+a+b
]
(1+b)(1+a) = 1+b+a+ab
(since ab is very small).
y
| x| | y|
The largest measurement of x/y =
U U
x
[1  x  y ]
y
| x| | y|
U U
x
[1  ( x  y )]
y
| x| | y|
Similarly, smallest measurement is
Therefore, the measurement of x/y with uncertainty is
Case 4: f(x,y)=x/y: Under Condition 2:
2
U  U 
[1   x    y  ]
y
 x   y 
The measurement of x/y with uncertainty is x
2
15
A General form of combining k variables, x1,x2, … xk.
Case 1 to 4 are cases of combining two variables of x and y. Their
measurement uncertainties are investigated under three different
conditions for case 1 and 2,and under two conditions for case 3 and 4.
In this section, we will consider a general function, f(x1,x2, …,xk)
Under Condition1: the measurement of f(x1,x2, …, xk) with
uncertainty is given by:
f ( x1 , x2 ,
, xk ) | f / x1 | U x1  | f / x2 | U x2 
 | f / xk | U xk
Under Condition 2: xi’s are independent. The measurement of
f(x1,x2, …, xk) with uncertainty is given by:
2
f ( x1 , x2 ,
2
 f
  f

, xk )  
U x1   
U x2  
 x1
  x2

 f


U xk 
 xk

2
16
Examples:
1.
To find the area of a triangle, the base is measured to be
The height is measured to be
5.0  .3cm
3.0  .2cm
What is the measurement of the area?
Ans: f(b,h) = bh/2. Under condition 1: the fractional uncertainty of f(x) =
Ub U h

 (.3 / 5)  (.2 / 3)  .1266
b
h
Hence, the area is (5)(3) / 2[1  .1266]  7.5  .94
Under Condition 2: The measuring process of base and height are independent.
The fractional uncertainty of f(b.h) =
2
2
 Ub   U h 
2
2
      (.3 / 5)  (.2 / 3)  .0897
 b   h 
Hence, the area is 7.5(1  .0897) = 7.5  .67
17
Hands-on Activity
Consider a system consists of four components: x,y,z,w. The
measurements are
x  80  4, y  100  6, z=70  2, w=150  5
(a) Suppose the system is
f ( x, y, z, w)  3x 2  yz
Find the measurement of f under condition 1 and 2.
(b) Suppose the system is
f ( x, y, z , w)  wz  ln( y )
Find the measurement of f under condition 1 and 2.
3x 2  yz
( c) Suppose the system is f ( x, y, z, w) 
exp( wz )  2 xy z
Find the measurement of f under condition 1 and 2.
(d) Suppose the system is f ( x, y, z , w)  xy / wz
Find the measurement of f under condition 1 and 2.
(e) Suppose the system isf ( x, y, z, w)  ( x  y  2 z )
Find the measurement of f under condition 1 and 2.
18
A General form of combining k variables, x1,x2, … xk- Continued
We discuss how to determine the uncertainty of a general function, f(x1,x2, …,xk)
under Condition 1: no information is known, and (2) Variables are independent. There
are situations where some variables may not be independent. For example, when
measuring the water pressure of water in a container, it clearly related to the volume
of the container. In some cases, the physical or chemical properties give us functional
relations between variables, and we can take the advantages of the function
relationship. In many, we do not have the functional relation. However, we can use
data to estimate their relation and take into account the relation into the computation
of combined uncertainty. Recall the results under conditions (1) and (2).
Under Condition1: the measurement of f(x1,x2, …, xk) with uncertainty is given by:
f ( x1 , x2 ,
, xk ) | f / x1 | U x1  | f / x2 | U x2 
 | f / xk | U xk
Under Condition 2: xi’s are independent. The measurement of f(x1,x2, …, xk) with
uncertainty is given by:
2
f ( x1 , x2 ,
, xk ) 
2
 f

 f

U x1   
U x2  

 x1

 x2

 f


U xk 
 xk

2
When Xi’s are correlated, we need to estimate the correlation structure
and take it into the computation of the uncertainty.
19
We shall consider several simple cases before discussing the
general form for cases when variables are correlated.
Case 1: f(x,y) = x +y. X and y are random variables follows a certain
probability distribution. Then a typical measure of uncertainty is the
variance.
Result 1: V(x+y) = v(x) +v(y) + cov(x,y), where cov(x,y) is called the
covariance of x and y. When samples are uses to estimate these components,
v(x) is estimated by
sx2  [ xi2  n( x )2 ]/(n  1)
V(y) is estimated by s y2  [ yi2  n( y )2 ]/(n  1)
And cov(x,y) is estimated by
sxy  [ xi yi  n( xy )]/(n 1)
The standardize form of cov(x,y) gives the correlation coefficient,
is estimated by the Pearson’s correlation:
sxy
rxy 
20
sx2 s y2
For computational simplicity, we usually compute Sum of Squares and apply
them to compute variances , covariance and correlation coefficient.
SSx  [ xi2  n( x )2 ] , and sx2  SSx /(n  1)
SS y  [ yi2  n( y )2 ] , and s y2  SS y /(n  1)
SSxy  [ xi yi  n( xy )] and sxy  SSxy /(n 1)
rxy 
sxy
2 2
x y
s s

SS xy
SS x SS y
NOTE: -1  r  1
From the correlation, the estimated covariance can be computed from correlation and
sample variances:
sxy  rxy ( sx2 s y2 )  rxy (sx )(s y )  rxy ( SS x SS y )
These formulae look pretty complicated. However, they all
come from three values: SSx , SSy, and SSxy. The following
figures demonstrate some cases of correlation.
21
Relationship between X and Y
r > 0 and about .7
r is positive, about .7, and the
relationship is nonlonear
r < 0 and about - .7
15
15
10
15
10
5
Y
Y
Y
10
5
0
1
2
3
4
5
6
7
8
9
10
5
0
X
1
2
3
4
5
6
7
8
9
10
10
5
X
4
5
6
7
8
9
6
7
8
9
10
15
10
5
4
3
Y
Y
5
3
2
X
Y
15
2
1
r is approximately zero
15
1
0
r is almost one
r is approximately zero, and the
relationship is nonlonear
0
10
X
10
5
0
1
2
3
4
5
X
6
7
8
9
10
0
1
2
3
4
5
X
6
7
8
9
10
22
Result 2: f ( x1 , x2 ,..., xk )   ai xi
with xi’s being random variables
V ( ai xi )   ai2V ( xi )  2 ai a j Cov( xi , x j )
i
j
In terms of sample estimates:
s 2 ( ai xi )   ai2 sx2i  2 ai a j [rij ( sxi )( sx j )]
i
j
The uncertainty is just the sample standard deviation of the corresponding estimate
of the function when using sample information. Therefore, in terms of the
uncertainty notation, result 2 becomes to:
Result 3:
U ( ai xi ) 
=
f ( x1 , x2 ,..., xk )   ai xi
a U
2
i
a U
2
i
2
( xi )  2 ai a jU ( xi , x j )
i
2
j
( xi )  2 ai a j [ rij (U xi )(U x j )]
i
j
23
Measurement Uncertainty of a general function f(x1,x2, …,xk) with
the xi’s being correlated.
U ( f ( x1 ,x2 ,
=
,xk )) =
a U
2
i
a U
2
i
2
2
( xi )  2 ai a jU ( xi , x j )
i
j
( xi )  2 ai a j [rij (U xi )(U x j )]
i
j
where ai  f xi , which is called the sensitivity coefficient.
This measures the change of the function f on the coordinate x i
when x i is incresed by one unit.
NOTE:
When sample information is used to estimate the uncertainty,
U(x) = sx , and U(x,y) = cov(x,y) = rxyU(x)U(y) = rxy(sx)(sy)
24
Hands-on activity
A system consists of three components, x,y,z. These components are
correlated. Based on the sample information, the sample variancecovariance matrix is
 s 2x
 16



Var-Cov matirx of (x,y,z) =  sxy s y2  
 sxz s yz sz2 


 8 25 


 4 1036 
The measurement for each component is:
x  U x  50  4,
y  U y  44  5,
z  U z  74  6
(a) Suppose the system is f(x,y,z) = 2x-y/z.
Determine the uncertainty for the system.
(b) Suppose the system is f(x,y,z) =(x/2 – y2)z
Determine the uncertainty for the system.
25
Q3: If there are n individuals measure the same component.
Each obtains a measurement x j  Ux j
How do we combine these measurements and uncertainties ?
Consider the situation: Three standardized labs tested the same
material using the same procedure and assume the environmental
conditions are uniform. The tested results from the three labs are:
Lab1: 25  1.5,
Lab2: 24.5  .8, Lab3: 26.2  1.8
The intend is to combine the testing results as the reference for other
labs. This type of problem is different from what we discussed
before. In this case, we measured one variables by three participants,
and obtain different uncertainties.
It is most likely that each result is from a repetitions of several tests.
In this case, xj is the sample mean, and Uj is the standard deviation of
26
mean, s / n .
In combining these results to obtain the best estimate, we should make sure that the
lab testing results are consistent, and the lab systematic error is negligible. That is the
uncertainty is from the random error only. Therefore, before combining the test
results, we should make a quick check :
•If there are any unusually large discrepancy |xi – xj| between each lab.If this
discrepancy is much larger than Ui and Uj, we should suspect that at least one
measurement has gone wrong, and a close examination of the process of testing is
needed.
•If there is any unusually large uncertainty from a lab. If Ui/Uj is over three, we
should suspect some systematic errors exist, and a close examine of the process of
testing is needed.
One way to combine the testing results is by an average:
f(x1,x2,x3) = (x1+x2+x3)/3
However, the results have different precision. We would assume that the more
precise result (smaller uncertainty) should be given a larger weight. In stead
of using the un-weighted average, a weighted average gives a better estimate.
27
Weighted Average for combining measurements that measuring the
same variable.
Different weighting scheme can be developed mathematically. In the following
we apply the one that is developed based on ‘Maximum Likelihood Principle’,
one of the most important estimation principle in statistical estimation.
Consider k labs measured the same variable and obtained
x1  U1 , x 2  U 2 ,
, xk  Uk
The weighted average is given by:
 1/ U12 
 1/ U 22 
 1/ U k2 
xcomb  
x 
x  
x
  (1/ U 2 )  1   (1/ U 2 )  2
  (1/ U 2 )  k
i 
i 
i 



x
=  i2  (1/ U i2 ) =  w i xi  wi
Ui
where, w i 
1
U i2
The measurement uncertainty of x comb is given by
1
w
=
i
1
 (1/U
2
i
28
)
Example: Consider now the case of three labs. The measurements are
Lab1: 25  1.5,
Lab2: 24.5  .8, Lab3: 26.2  1.8
The best estimate of the measurement for the variable X is
 25.0
24.5
26.2   1
1
1 
xcomb  


/


 57.48 / 2.3156  24.82
2
2
2  
2
2
2 
 (1.5) (0.8) (1.8)   (1.5) (0.8) (1.8) 
The combined uncertainty is
1
1 
 1
 1/  2 

 1/1.5217  .657
2
2 
1.5
0.8
1.8


 1 
U 2 
 i 
1
The best combined estimate for the variable x is
24.82  .657
29
Hands-on Activity
1.
Three individuals measured the same component and obtain the following
results:
A :80  5
B: 78  4
C: 83  5
Obtain the weighted combined measurement and its uncertainty.
2. Four labs conducted the same testing procedure to test the same material in order
to set up a reference uncertainty for the material. Each lab repeated the test for
eight times. The testing results are:
Lab A
12.4
12.6
12.3
11.9
12.0
12.4
12.1
12.0
Lab B
11.9
11.9
12.2
12.1
12.4
12.0
11.8
12.2
Lab C
12.0
12.3
12.6
12.5
12.0
12.1
12.0
12.3
Lab D
11.8
12.3
12.5
11.9
12.2
12.3
12.3
12.2
(a) Use this data to estimate the lab average and within-lab uncertainty for each lab.
(b) Is there any unusual lab averages or within-lab uncertainties?
(c) Determine the best estimate of lab average with uncertainty of the best estimate
for each lab.
(d) Obtain the best estimate of combined lab average and uncertainty using the
30
weighted method.
Q4: When the distribution characteristic is applied to
measuring the uncertainty, how to quantify the uncertainty
based on the distribution characteristic of the measured
property?
In the previous sections, we discussed how to quantify
uncertainties without imposing the concept of probability
distribution to the variables of interest. The uncertainty has been
presented as one unit uncertainty.
An important question in statistical estimation and and quantifying
‘How much confidence’ we can claim that the actual unknown
characteristic falls between
xbestEst  U x
31
In the following section, we will discuss how a confidence level is formed, and the
role that probability distribution plays in making the level of confidence.
We will focus on
1.
Estimating population mean. Our purpose is to be able to make statements such
as we are 95% confidence (sure) that the true lab average for testing Material A
is between 32.3 to 35.4, and so on.
The key difference between ‘be able to make a confidence level statement’ and
‘presenting uncertainty only’ is that the occurrence of variable of interest follows a
certain probability distribution. Therefore, we can characterize the variable using an
appropriate distribution.
For example, based on our common experience, adult weights usually follows a
normal curve. While, distribution of salary is usually skewed-to-right. B
By imposing adequate distribution, we can find out how much chance that each of
these intervals will cover the truth of the measurement:
xBestEst  U x , xBestEst  2U x , or more generally, xBestEst  kU x
32
D.F. for the combined measurement uncertainty
When combining several Type A uncertainties, due to the fact that different
components may be measured using different number of observations. Each
uncertainty has different degrees of freedom. As a consequence, a combined
degrees of freedom would be necessary for expanded uncertainty when
distribution property is assumed. A simple approach of combining d.f.’s is by
using a weighted average.
(This is what is called Welch-Satterthwaite method)
Consider uncertainty for component i is obtained based on vi d.f.. Then, a
weighted combined d.f. can be found by:
 comb
U 4 ( f ( xi , x2 ,..., xk )

(f / xi )4 U 4 ( xi )

vi
NOTE:  comb   vi
An expanded uncertainty based on t-distribution is now
possible:
f  t( / 2, df )U ( f )
33
Determine the uncertainty of Sample Mean
Recall the activity of ‘drawing two centimeters of line segment ten time’.
•Our goal is to estimate how well we can draw two cm line. The best single estimate
would be the average. Since there is always uncertainty in our drawing, we usually
report our estimate as an interval:
Best Estimate  k(Uncertainty of the Best Estimate)
In the case of using sample average to estimate the unknow population mean:
an appropriate report is
X  k[ SD( X )]
•The quantity k is determined by the level of confidence we intend to report.
•In the drawing activity, there are two possible averages: Individual’s average of ten
draws for estimating individual’s measurement and the average of all draws from
everyone for estimating the measurement of the target population of interest.
34
•Depending on the purpose, we use the corresponding sample average to
estimate the unknown nature of the true average. Accordingly, we will need
to estimate the uncertainty of using sample mean, x to estimate the
 .
population unknown truth mean, we usually use the notation
How much uncertainty is it when using sample mean to estimate
the population mean? How to estimate this uncertainty?
•Using our common experience, we can conclude that if we increase sample
size, then the sample mean is closer to the population mean.
•How close is it? Can we measure the degree of closeness? The answer can
be understood from the following scenario:
Imagine in a laboratory, we test a characteristic of a material. Assuming
the testing procedure is standardized and the testing process is under
statistical control. Let X represent the measurement of the characteristic.
Each day, five sub-samples of the material are tested for 400 days.
35
Recording the data in a spreadsheet:
Sample
1
2
3
4
5
Day1
X11
X12
X13
X14
X15
Day2
X21
X22
X23
X24
Day3
X31
X32
X33
Day400
X400,1
X400,2
X400,3
Sample
mean, x
Standard
deviation, s
s1
X25
x1
x2
X34
X35
x3
s3
X400,4
X400,5
Average of all 400 days
x400
x
s2
s400
s
We notice that
•The individual measurements are different, and we can observe the average,
variability and the histogram of these 2000 individual measurements.
•The daily averages are also different, but they are closer to each other than
individual measurements. We can also observe the average, variability and
histogram of these 400 sample means.
36
Computer Simulation Activity to Demonstrate Sampling
Distribution of Sample Mean
Simulate 400 days of laboratory tests.
Consider (a) n =5 sub-samples per day
(b) n = 20 sub-samples per day
(c) n = 40 sub-samples per day.
Compute daily averages, and demonstrate the relation ship
among the distributions of individual test measurements,
sample means from n = 5, n = 20 and n = 40.
Summarize the pattern of these relationships.
37
Patterns from the computer simulation
38
•Consider n = 5, the 2000 measurements resemble the unknown population. And
therefore, the distribution of these 2000 observations, the average and s.d. should
also be very close to the population. We use the notation: the population mean is
, and population s.d. is .
•The distribution of the 400 sample means reflects the uncertainty of sample means of size
n = 5. The smaller the s.d. of the sample mean, the more precise the sample mean is for
estimating the population mean. We use the notation: the distribution mean of sample
averages is  x , the distribution s.d. of the sample averages is  x .
•The relationships are:
x
= , and
x
=
/ n
Distribution means are the same. This is the property of UNBIAS: The averages of all
possible sample means = population mean.
Distribution s.d. of Sample Mean is smaller when sample size increase. This measures how
close sample mean is to the population mean for a sample of size n. Therefore, s.d .( X )   / n
measures the uncertainty of using sample mean to estimate population mean.
When we obtain a sample, and compute the sample mean, s.d .( X )   / n says one unit
of error between the population mean,  and the observed x is  / n .
When sample size increases, the error of using sample mean to estimate population mean
decreases. Therefore, we also call s.d .( X )   / n the Standard Error of Sample Mean, SE ( X )
39
Since population s.d., , is usually unknown, we use sample s.d., s to estimate .
Therefore, SE ( X )  s / n.
To report the sample mean with the uncertainty measurement is given by:
x  k  SE ( x ) or equivalently, x  k  ( s / n )
The multiple, k, is determined based on the level of confidence we make the claim. Recall the
Empirical Rule suggests that if k = 2 and the distribution shape of the sample mean is normal,
then, this provides about a 95% level of confidence.
How can we be sure and when the shape of X distribution will follow a normal
distribution?
• Fortunately, it is guaranteed that, if the sample size is large, the shape of X distribution is
approximately normal (common experience suggests n > 30 is large enough to guarantee the
normal shape. Putting the above discussion together, we have:
•The distribution of
X is approximately normal with mean  and s.d. =  /
replace  by s if  is not known, when sample size is large enough (n > 30).
40
n
Using the above 400 days of testing data, sample size n = 5 each day. Suppose these 2000
measurements give us the best estimate of the population characteristics:
x  15, and s = 3. Since our sample size , n = 5, we would can determine the distribution
of sample means has the center = 15 and
SE ( X )  s.d .( X )  s / n  3 / 5  1.342
By assuming the population characteristic follows normal, we can make a 95%
confidence level of estimation for the population mean:
95% of chance that the average of the characteristic is between
15  2(1.342)
Equivalently, we are 95% confident (sure) that the the average of the characteristic
is between 15  2(1.342)
More precisely, in stead of using k =2 , we can apply the normal distribution and use
k = 1.96.
Extending this, we can obtain a general pattern for constructing a confidence
interval for estimating population mean at any level of confidence.
41
A General Pattern for constructing 100(1-)% confidence interval for
the unknown population mean when sample size is considered large
X ~ N (,  / n), For this example, X ~ N (15,3/ 5)
.95 = 1-
.025 = /2
.025 = /2
X
15
-1.96=-Z/2
0
Z = ( X -15)/1.342
1.96=Z/2
100(1-)% confidence interval for population mean is
x  ( z / 2 ) SE ( X )
95% confidence interval is x  (1.96) SE ( X )
90% confidence interval is x  (1.645)SE ( X )
99% confidence interval is x  (2.576) SE ( X )
42
How can we determine a confidence interval of population
mean using the information of a small sample?
When reporting the uncertainty of a sample mean, we report:
x  k  SE ( x ) or equivalently, x  k  ( s / n )
The multiple, k, is related to the level of confidence. For large sample cases, we take the advantage
of the normality property of X , and use its standardized distribution, the Z-distribution to
determine the multiple, k, for any given level of confidence.
When sample is drawn from a normal population with unknown variability, we are not able to enjoy
the same nice normality property for the distribution of X
A somewhat more complicated distribution, called t-distribution is used as the standardized
distribution for X
T-distribution is similar to the Z-distribution: It has the center at 0 and distribution is also symmetric
about the center, 0. However, it depends on what we called degrees of freedom (d.f.). In one sample
confidence interval cases, the d.f. = (n-1).
.95 = 1-
Example: n = 10, d.f. = n-1=9.
T-values depends on d.f.. Some
commonly used t-values can be
found in every statistical book.
.025 = /2
.025 = /2
t 
-2.262=-t/2,9
0
2.262=t/2,9
( X  )
s/ n
43
A General Pattern for constructing 100(1-)% confidence interval for
the unknown population mean when sample size, n, is small
x  (t / 2 ,df )
100(1-)% confidence interval for population mean is
95% confidence interval, when n = 10, is
s
s n
x  (2.262)
90% confidence interval, when n = 10, is x  (1.833)
s
n
n
s
99% confidence interval, when n = 10, is x  (3.25)
n
An example: In a lab testing, 16 samples are tested. Data are summarized.
Sample mean, x = 32.5 and s = 4.8
A 95% confidence interval is
s
4.8
x  (t / 2 ,df )
 32.5  (t.025,161 )
 32.5  2.1311.2  [29.94,35.06]
n
16
More exercises for applying t-distribution, if needed.
44
Some commonly used distributions
1.
Normal distribution: is the most common one for describing many
continuous variables. It is symmetric about mean . The shape is bellshaped. The spread is characterized by the scale parameter, .
2.
Uniform distribution: This distribution describes the random occurrence
with equal chance in a given interval. This is applied for describing the
Type B uncertainty.
3.
Triangular distribution: This distribution is applied for describing the Type
B uncertainty as well.
(NOTE (2) and (3) are easy to operate. They uncertainty of uniform distribution
is usually large, therefore, more conservative. This is applied when no prior
knowledge for Type B uncertainty. Triangular distribution is symmetric
about and easy to operate.When the variables are not measured empirically,
this is applied for a less conservative Type B uncertainty.)
4.
Weibull distribution is common for characterizing physical properties, such
as life time, strength, hardness, and so on. It is usually skewed to the right.
45
5. Gamma and Exponential distributions: These are another
common distributions for characterizing physical properties.
6. Binomial distribution: X = # of successes in n identical and
independent trials. This is for discrete random variables. It is
most commonly used in describing # of defectives in a sample.
It is a very useful tool for quality control when the # of
defectives are to be monitored in a process.
7. Poisson distribution: X = # of occurrences of an event in a
short time period. This is common for describing # of mutations
in cells in a given short time period when treated with a different
dosages of radiation. It is also common in quality control, where
the number of defects in a finish product is to be monitored.
46
8. Several important distributions for sampling statistics: tdistribution, F-distribution, and Chi-Square distribution.
T-distribution describes the standardized distribution of
sample mean when sample size is small. It is commonly used for
testing the difference of two population means.
F-distribution describes the distribution of some function of
the ratio of sample variances. It is commonly used in analysis of
variance, and in comparing two population variances.
Chi-square distribution describes the sum of squared
standardized difference between observed and expected
frequencies. It is commonly use as a goodness of fit test for
testing how well a distribution fits a real world data. It is also
used for comparing a sample variance with a reference variance.
47
Each distribution describes a certain real world phenomena. It
is important to learn the property and assumptions when
applying them to meet your needs.
48
Uncertainty from Linear Least Squares Calibration
Calibration is an important tool for adjusting an instrument so
that the systematic bias due to the inaccurate instrument read
out will be reduced. Calibration involves with
1.
Determine response variable and the level of analyte x for the instrument
to be calibrated.
2.
Observing the responses y at different levels of the analyte x.
3.
Modeling the relationship between x and y.
In most cases, such a relationship is linear: y = a+bx
4.
Use an observed response y to determine the anlyte level x through the
regression line. This level is then compared with a standard or a reference
level that supposes to result the given y response.
5.
The uncertainty of the predicted level of x from the response y can be
estimated using the regression line.
49
Consider the following simple situation:
When a machine is used to measure the diameter of a bearing. The
A standard bearing with known diameter,  is measured by the
machine for n times, it is often the case that the measurements are
varied, xi, and worse, the average of the measurements may be
much far away from the ‘known’ true diameter, .
That is, B  x   is not zero. This is the the system bias,
which is often due to instrunment wearing along the time.
The approach of calibration is used to adjust the instrument
back to its original condition.
In statistical term, it is to make the Bias = 0.
50
Example, suppose we would like to calibrate a weighing device. A
key adjustment knob is the key to the weighing system.
An object with a known ‘correct’ weight is used for calibrating the
system. The knob can be set up at a continuous position.
For the calibration purpose, 10 different positions are tested. These
positions are measured at degree of angle from the zero position.
And suppose the true correct weight of this object is 46.0 mg. How
can we use the data and the correct weight to calibrate the knob?
X
y
30
45.6 mg
31
45.8
32
45.8
33
46.0
34
46.2
35
46.2
36
46.5
37
46.7
51
Procedure of conducting a calibration
1.
Fit a linear regression line: y = a+bx by the least square method:
b
SS xy
SS x

sxy2
s
2
x
 rxy
sy
sx
 rxy
SS y
SS x
and a  y  bx
2.
For a given yobs, compute the predicted x-value is : xpred = (yob – a)/b
3.
Compute the uncertainty involves with xpred by : s(xpred) = s(yobs)/b
2
1 ( x pred  x )
where s( yobs )  s 1  
n
SS x
4. For the given y response, the predicted x measurement is presented by
x pred  s ( x pred )
5. A 100(1-)% confidence interval for the predicted x is (the expanded
uncertainty) is given by
x pred  t( / 2,n 2) s ( x pred )
52
Hands-on Activity
Consider the weighing device calibration example
X
y
Variable
N
Mean
StDev
SE Mean
30
45.6 mg
X
8
33.5
2.449
0.866
31
45.8
Y
8
46.1
0.374
0.132
32
45.8
33
46.0
34
46.2
35
46.2
36
46.5
37
46.7
Regression Plot for Calibration Study
Y = 41.075 + 0.15 X
S = 0.0763763
R-Sq = 96.4 %
Y
46.5
46.0
Covariances: X, Y
X
X
Y
Y
6.00
0.90
Regression
95% CI
45.5
30
0.14
31
32
33
34
35
36
37
X
53
At y = 46.0, the ‘correct’ weight, xpred = (46-41.075)/.15 = 32.83
Row
x-pred-y
y-pred
sdyfit
Several x-values are used to
predict the y-responses and the
results are in the table.
1
32.80
45.9950
0.0282351
2
32.81
45.9965
0.0282009
3
32.82
45.9980
0.0281672
For y = 46, we obtain x = 38.33.
The knob should be calibrated to
38.33. The uncertainty of xpred is
given by
4
32.83
45.9995
0.0281339
5
32.84
46.0010
0.0281010
6
32.85
46.0025
0.0280686
s( x pred )  s( yobs ) / b  MSE  s 2 ( y fit ) / b  .07637632  .02813392 / .15  .5426
NOTE: The Minitab output provides s(yfit) not s(ypred). The relationship between
s2(ypred) and s2(yfit) is
s2(ypred) = MSE + s2(yfit)
54
Some important activities for calibration study
It is important to conduct a diagnosis of the residuals
1.
to make sure that the linear fit is adequate, and
2.
to make sure there is no unusual observations that due to some special
causes.
3.
If the fit line is not adequate,
•
It most likely because the range of x-values is too wide. One can narrow
the range of the x-values, observe additional responses in the smaller
range of x-values, then re-conduct the calibration study.
•
If the small range of x-values does not work, a more complicated model
such as quadratic model may be needed.
•
It may be possible that the x-variable is not the key factor that affect the
‘shift’ of y-responses. Further investigation starting from brain-storming
and setting up cause-effect diagram may be needed.
55
Type B Uncertainty
Type B uncertainty rises when the uncertainty is due to the physical
property, the prior information, previous experience, or specifications
of measurements predetermined for the instruments, or sometimes,
the environment does not allow or is very costly to conduct empirical
data analysis to determine Type A uncertainty.
Because of the limitations of observing the actual data to estimate
Type A uncertainty, we usually are more conservative in determining
the Type B uncertainty when no information about the distribution of
possible range of measurement. In addition, in estimating Type B
uncertainty, we would like to simplify the estimation process with a
relatively conservative, but, ‘good enough’ estimate. For these
reasons, simple distributions such as such as Rectangle or Triangle
distribution are typically applied.
56
Variable follows Uniform Distribution
When there is no information about the variable to be measured, the Type
B uncertainty assumes the distribution to be rectangular distribution with
the x-range of 2a. From statistical probability point of view, this is a
uniform distribution, and x-values have equal chance to occur. The
distribution shape:
2a
The uncertainty is x.
Based on a uniform distribution,
2 = ((c+a)-(c-a))2/12 = a2/3.
Therefore, U(x) =  = a / 3
.5/a
c+a
c-a
X
57
Variable follows a Triangular distribution
In some cases, we are able to assume the variable follows a symmetric
distribution with smaller variability than Uniform, and at the same time
keep the simplicity for computation.
A Triangular distribution is a good approximation for these situations.
The uncertainty of the Triangular
distribution is . The variance of X
is given by 2 = a2/6. Therefore,
U ( x)  a
6
When we measure Type B uncertainty, we
need to determine which distribution is more
appropriate.
2a
1/a
c-a
X
c+a
1
x

for x  [c-a,0]
a a (a  c)
1
x
= 
for x  [0,c+a]
a a (a  c)
f ( x) 
58
Type B uncertainty based on Normal Distribution
Normal distribution is commonly used in Type A. It can also be
applied for Type B. Since Type B does not require observing
data to estimate the standard deviation, we need a quick estimate
of uncertainty if the variable is assumed normal.
For Normal distribution,
2 from the mean is about 95% of coverage probability, and
3 from the mean is about 99% of converage porbability.
Therefore, if we have some good idea about the range of the variable, R (=largest - smallest),
we can approximate the uncertainty,  , by
  R / 6 (since R  6 co vers almost 100% of the data)
Or a somewhat more conservative estimate by   R / 4(s ince R  4 covers95% of data.)
59
Degrees of Freedom for Type B uncertainty is another activity we
need to estimate.
When a system consists of uncertainty components of both
Type A and Type B, there is a need to determine appropriate
degrees of freedom for the Type B uncertainty, which can then
be used to combine with Type A uncertainty. This may be
needed for the expanded uncertainty as discussed before.
In most cases, the degrees of freedom for Type B uncertainty is
assumed infinite, since the uncertainty is considered the
population parameter, not a sample estimate.
60
61
```