Transcript ppt - Extras Springer

Ch 9: 111
National Research
Council Canada
Conseil national
de recherches
Sept 5, 2003
User’s Guide to the ‘QDE Toolkit Pro’
Excel Tools for Presenting Metrological Comparisons
by
B.M. Wood, R.J. Douglas & A.G. Steele
Chapter 9.
Graphing and Pooling
Measurement Distributions (ii)
In this chapter, we present the use of the QDE Toolkit Pro’s facilities for
graphing distributions and pooled distributions. We also present some
summary statistics, calculated by the QDE Toolkit Pro, that are useful
even before a KCRV is chosen.
Ch 9: 112
QDE Toolkit Pro Graphs - working with these Pooled Distributions
After running the macro tk_pool_PlotBuilder, the graphs can be adjusted
with Excel. Here’s a quick review of some things you will want to do:
Point at a graph, left mouse click to select it
and pop it to the top of the stacked graphs.
Point at a corner or edge of a selected graph
and drag (left mouse button) to resize.
Tip: We find Excel’s
Double-click on the selected chart’s y axis (the vertical line at x=0) to
get a dialog box, select the scale tab, enter the new y maximum...
undo capability is very
useful in reversing
unwanted formatting of
graphs. Select
Edit|Undo with the
mouse, or ‘control Z’
from the keyboard.
Similarly, the x-axis range, etc. can be adjusted. There is a very wide range in x of data points
available (~1000 uniformly spaced points across the default graph range and ~1000 more on an
expanding mesh to cover ~200 x the breadth (for integrating Student distribution tails).
Ch 9: 113
QDE Toolkit Pro Graphs - working with these Pooled Distributions
After running the macro tk_pool_PlotBuilder, the graphs’ data can be
edited or recalculated with Excel (for example to convolute with either a
uniform or a “goalpost” distribution).
The data (~2000 rows x ~10-20
columns) are extensive enough so
that some Excel tricks (shift
Edit|Paste Picture, or using F9 to
convert a graph series into an
explicit formula array) for taking a
frozen picture of a graph no longer
always work well.
It is often very helpful to take a
frozen snapshot of a graph, and
going beyond screen resolution
can be a challenge!
After getting the Excel graph about
right, one trick seems to give the
best for-printing conversion to
Prior to Excel 97 SR2, Excel had a memory leak when creating
lots of charts that could give a “not enough memory” error and
Windows metafile form…
required restarting Windows occasionally.
(continued on next page)
Ch 9: 114
QDE Toolkit Pro Graphs – freezing a graph (as a picture)
For Office 97, or
for Office 2000,
we copy the
selected graph to
the clipboard, and
do a “Edit |Paste
Special” into
Microsoft Word
as a “Picture
(Enhanced
Metafile)”.
Then, copying
from this graph in
Word, it can be
pasted (not Paste
Special this time)
back into Excel
as a “frozen
picture”.
For Office XP,
we copy the selected graph to the clipboard – Control-C
Paste it into Microsoft Word – Control-V,
then click on the clipboard icon beside the
pasted graph to select the “Paste Options”
select “Picture of Chart
(smaller file size)”
Press the Esc key
to get rid of the icon
Select and copy the graph in Word
Paste it back into Excel
as a “frozen picture”.
Ch 9: 115
QDE Toolkit Pro Graphs - working with Reference-Value Distributions
The macro tk_pool_PlotBuilder can also graph the PDFs of the candidate
KCRVs. The RVs are graphed with thick lines.
In column A of the input table, the block of reference values starts beneath the block
of Lab comparison data, with the first name containing the character pair RV (case
insensitive). It ends with the first blank in column A.
Here, two RVs are graphed. One
is labeled the KCRV, with no
uncertainty given. (This is
sometimes done to avoid
profound difficulties, for example
by the CCT.)
This first RV (its value happens
to be the simple mean of the 8
pooled Labs) is plotted as a
delta-function (really .001 of the
initial graph width).
The second RV is an inversevariance weighted mean of the
8 pooled labs, with its formal
standard uncertainty (the same
as the product of PDFs, since
they are all normal) and
correlation coefficients with the
contributor labs (which are not
used in plotting the RV’s PDF).
Ch 9: 116
QDE Toolkit Pro Graphs - working with the “supplementary information”
After running the macro tk_pool_PlotBuilder,
there is a block of supplementary information
in the 16 columns to the right of the correlation
coefficient matrix. We will discuss them in turn,
but here’s the overview:
Tip: the Toolkit Pro’s output italic
numbers are unitless, and
regular font numbers have units.
Col after rij
1 Lab names: comments hold
reduced chi-square of differences
with all other pooled Labs.
2-4 Input Comparison Data for Labs,
similar statistics for pools
5-6 Du/u for degrees of freedom:
variance-based and tail-based.
7-8 Coverage Factors for 68.0% and
95.0% confidence (from trapezoidal
integration of the distributions).
Mostly of interest for the pooled
distributions.
9-12 Error in the symmetric vs
rigorous confidence intervals. Mostly
of interest for quantifying the unimportance of this effect for the
asymmetric pooled distributions.
13-15 Mean, Median and (first) Mode
of each distribution. Mostly of interest
for the pooled distributions,
16 Lab Names, just for convenience
Ch 9: 117
QDE Toolkit Pro Graphs - col 1 of the “supplementary information”
1
Col after rij
1 The comment in the top row of the column
contains the date and time of creation of the block
of supplementary information.
The column is of Lab names for each row, but
some really useful information is in the comments
of n “in-pool” labs, which give the reduced chisquare statistical information about their rms En
with all other (n-1) “in-pool” Labs: (n-1) terms,
between 1 and (n-1) degrees of freedom.
The bottom (“Pair Difference”) name has a
comment that gives the all-pairs chi-squared for
the pool: there are n(n-1) terms to sum, with an
obvious 2:1 redundancy: Despite having n(n-1)
terms, the degrees of freedom is still (n-1).
Note that this chi-square is independent of any
choice of KCRV. Its use is illustrated in: Hill, Steele
and Douglas, Metrologia 39, 269 (2002). It is also
discussed in a bit more detail on the following
page. It uses the Excel statistical function ChiDist
to convert from an all-pairs-variance to a
probability that it could be exceeded by chance.
Ch 9: 118
QDE Toolkit Pro Graphs - VERY USEFUL INFORMATION about Lab j
The chi-squared of each of N in-pool Labs, with respect to the N-1 other Labs, is
calculated and put into this column as comment boxes. This is the Lab’s “rms En”:
j2 = (N-1)-1 i=1Ni≠j(xi – xj)2 / (ui2 + uj2 - 2rijuiuj)
If the differences are independent and normally distributed about zero, then this is EXACTLY a reduced chisquared statistic with N-1 degrees of freedom. It is perhaps the best test as to whether Lab j agrees with the
“rest of the world”, and is independent of any choice of KCRV. In practice, the differences are only
independent when the other Labs’ uncertainties dominate, and the degrees of freedom for j2 may be as
small as 1 when uj dominates.
If the differences were
independent and there was
perfect agreement, within the
stated uncertainties, for Lab j,
this value of j2 is expected to be
exceeded by chance with only
this probability: a minimum
probability that may be a
substantial underestimate if the
differences’ lack of
independence is considered.
In the Toolkit Version 2.07, the
probability, between 0 and 1, is
given in scientific number format.
Ch 9: 119
QDE Toolkit Pro Graphs - VERY USEFUL INFORMATION about comparison
The reduced chi-squared Labs j with respect to the N-1 other Labs, is:
j2 = (N-1)-1 i=1Ni≠j(xi – xj)2 / (ui2 + uj2 - 2rijuiuj)
By averaging the N j2’s,
2 = (N)-1 i=1N j2
the all-pairs variance. It is perhaps the best test as to whether these Labs agree
with each other within their uncertainties, and is independent of any choice of
KCRV. The degrees of freedom of this reduced chi-squared is N-1.
If there were perfect agreement,
within the stated uncertainties,
for all Labs, this value of 2 is
expected to be exceeded by
chance with only
this probability.
If this probability is very small,
then the measured differences
are not described very well by
the stated uncertainties.
In the Toolkit Version 2.07, the
probability, between 0 and 1, is
given in scientific number format.
Ch 9: 120
QDE Toolkit Pro Graphs - VERY USEFUL INFORMATION – a TIP
There are so many comment boxes that the workbook needs to have its comments
hidden unless mouse is pointing to a commented cell.
(Tools | Options |View | and select “Comment indicator only”. )
To keep one or more comment
boxes on display,
Right-mouse-click on the cell
Select “Show comment”
To hide one or more comment
boxes that is on display,
Right-mouse-click on the cell
Select “Hide comment”
Ch 9: 121
QDE Toolkit Pro - Using Pair Differences
The bilateral pair differences of laboratories can incorporate all
knowledge about the pairs, including correlations, so the reduced chisquare of all pair differences is an appropriate tool to consider for
characterizing a comparison model without having to choose a specific
reference value. The “all-pairs variance” or APV of a comparison with N
laboratories can be calculated
APV =
[ i =1N j =1N (xj - xi)2 / (ui2 + uj2 - 2 rij ui uj) ] / (N(N-1))
for normal PDF’s for the xi’s , the APV is distributed as a reduced chisquare with N-1 degrees of freedom.
Aside: Here we use the name APV here since it sometimes may be
appropriate to interpret the APV in terms of its model distribution, which is
not necessarily a reduced chi-squared if we include the effects of the
stated effective degrees of freedom for each laboratory. In the QDE
Toolkit Pro Versions 2.04…2.07, the analysis is all in terms of the normal
distribution limit and the reduced chi-squared.
Ch 9: 122
QDE Toolkit Pro - Using Pair Differences
The “all-pairs variance” or APV of a comparison with N laboratories is
APV = [i =1N j =1N (xj - xi)2 / (ui2 + uj2 - 2 rij ui uj) ] / (N(N-1))
for normal PDF’s for the xi’s , the APV is distributed as a reduced chisquare with N-1 degrees of freedom.
At the left, the Monte Carlo simulation of
the distribution of the APV of a 12-Lab
comparison is compared with the reduced
chi-squared curves having 10, 11 and 12
degrees of freedom.
Although the APV sum is over 132 nonzero terms, with 66 terms that look
distinct, the distribution of APVs is just the
reduced chi-squared distribution with 11
degrees of freedom.
The APV has been constructed to be
independent of any change of reference
value. Although we may have not yet
chosen a specific KCRV, one degree of
freedom is used by this possibility.
Ch 9: 123
QDE Toolkit Pro - Using Pair Differences
The “all-pairs variance” or APV of a comparison with N laboratories is
the master chi-squared for the comparison. This is true in the sense that
… if a comparison has failed the APV chi-squared test,
then NO choice of reference value can, by itself, rescue
the comparison…
This can be demonstrated by considering what happens to the pair
differences when the reference value is changed: the pair differences are
invariant, and so the APV chi squared statistic is also invariant.
Ch 9: 124
QDE Toolkit Pro Graphs - col 2-4 of the “supplementary information”
2
3
4
Col after rij
2-4 Input Comparison Data for Labs: value,
uncertainty, degrees of freedom. In columns 2 and
3, similar stastistical information is calculated for
the pools, and explained in comments.
Degrees of freedom is just as input, with a default
to “normal.
3
4
Ch 9: 125
QDE Toolkit Pro Graphs - col 5-6 of the “supplementary information”
5
6
Col after rij
5 Du/u for degrees of freedom: estimate based on the chi distribution’s varianceabout-the-mean based and tail-based. The overly broad tails of the Student
distribution at low degrees of freedom arise from the small arguments (and large
value) of the chi distribution.
Approximate fractional uncertainty in the standard uncertainty from variance of
the chi-square family of distributions that are highly asymmetric for degrees of
freedom less than 10.
See ISO Guide Eq.E-7 & Table E-1.
6 Du/u for degrees of freedom
Improved estimate Du for the fractional uncertainty in the standard uncertainty,
derived from the correct inverse-chi distribution's interval [0,u+Du] with 84%
confidence (the same Du as Col 5 in the limit of large degrees of freedom).
See discussion on pages 49-52 of this Guide, in the context of discussing
Tables of Equivalence.
Two definitions for degrees of freedom?
No. The variance-based method is simply a bad approximation for degrees of
freedom < 10, if the degrees of freedom is to be used for evaluating coverage
factors from Student distributions. Nonetheless, we suggest using the symbol nS
for a degrees of freedom aimed at describing the tails of the Student distribution.
Fortunately, in precision metrology, usually n > 10 and there is no difficulty.
Ch 9: 126
QDE Toolkit Pro Graphs - col 7-8 of the “supplementary information”
7 8
Col after rij
7-8 Coverage Factors for 68.0% and 95.0%
confidence (from trapezoidal integration of the
distributions). Mostly of interest for the pooled
distributions.
Because the product PDF is narrow, there may be
only a few tens of samples within ±s. An easy way
of monitoring this effect is to include a normal RV
of about the same value and width. Any variation of
the coverage factor from the expected norm (here
+0.1%) is an indication of the accuracy if the
product were over Student distributions instead of
normal distributions.
If the accuracy is not sufficient, in the Visual Basic
code in module QDE_Toolkit_PlotBuilder, in
subroutine tk_pool_PlotBuilder_With_Anchor,
near comment line ‘C140, change
poolpoints = 1002 to a larger number, such as
poolpoints = 10002 (Excel will limit you to
~60000)...
near comment line ‘C110 in the same subprogram
Confidence_k1 = 0.68
Confidence_k2 = 0.95
can be edited to the values of your choice. These
do not affect the MRA value of 95% confidence.
Ch 9: 127
QDE Toolkit Pro Graphs - col 9-12 of the “supplementary information”
9
10
11
12
Col after rij
9-12 Error in the symmetric vs rigorous
confidence intervals. For the symmetric input
data, you see the effects of round-off in the
trapezoidal integration.
These columns are mostly of interest for
quantifying the un-importance of asymmetry
for the asymmetric pooled distributions:
“How close are the intervals {mean-u,
mean+u] and [mean-U, mean+U] to the true
asymmetric confidence intervals, starting for
example at X where CDF(X)=0.025 and
running to X’ where CDF(X’)=0.975.
Ch 9: 128
QDE Toolkit Pro Graphs - col 13-15 of the “supplementary information”
13
14
15
Col after rij
13-15 Mean, Median and (rightmost) Mode
(the zero-slope peak) determined from the
table of each distribution.
Now this is mostly of interest for the pooled
distributions,
So as you see, there’s lots of information included in the
“supplementary information” columns that is eminently
ignore-able, most of the time. If and when it is wanted, it
will be waiting!