Transcript Document

Content – Annex
Content
Checklists
Glossary of terms
References
Statistics & graphics for the laboratory
105
Annex – Checklists
Checklists
IQC software
Administrative capabilities
-Easy set-up and modification
-Online (real time) connection with LIS
-Full sample & IQC traceability
-“Accreditation-conform” documentation
-Up-to-date data safety
“IQC”-capabilities
-Transparent & efficient data presentation
-Great variety of rules
-Rule selection logic
-Automatic release
-Automatic “flags” and remedial action
(but with open decision logic)
IQC-sample
• Nature
– Correspond with patient sample
– Compatible with the test
• Concentration: medically relevant
– Number of levels
• Stability (liquid versus lyophilized)
• Lyophilized
– Variation in fill content
– Accuracy of reconstitution
• Target mean
– Sufficient digits
– System specific
– Uncertainty
• Target SD/CV
– Representative for system
– Uncertainty
Statistics & graphics for the laboratory
106
Annex – Checklists
Checklists
“Stable” imprecision
Instrument stability (general system robustness), e.g.:
• Pipetting
• Temperature
• Photometer (wavelength/intensity/sensor)
Test stability & reproducibility (individual test robustness)
• Total/within-day CV (CVa,tot/CVa,w ratio)
• Calibration tolerance (within/between lot), -function
• Reagent (within/between lot)
• Test robustness
Note: Ideally, a GUM-type variance analysis of all elements should be available.
 Decide about approaching the variation by IQC and/or quality assurance
Statistics & graphics for the laboratory
107
Annex – Checklists
Checklists
IQC-measurement-frequency and location
• Minimum: 2 samples per run
• Desirable: ~1-2% of patient samples
• -Make a cost/benefit calculation
• Frequency should be related to test stability: requires knowledge of instrument
and test
• Consider “dummy” measurements before 1st IQC
• Frequency may depend on the control rule
• Block: Maximizes chance of assignable cause of variability between subgroups
• Continuous: Maximizes chance of assignable cause of variability within
subgroups
Basic statistics
Calculations
• Mean
• SD
• CV
Gaussian (normal) distribution
• Graphic of the usual & the cumulated distribution
• Probabilities within certain distances (s) of the mean
• Probabilities outside certain distances (s) of the mean
• 1-sided and 2-sided probabilities
• Important values for s
• Convention on “rare events”
• Possibility of wrong decisions
• The 1st IQC principle: monitoring the outsides
Metrology, knowledge of
• The total error concept (TE, SE, RE)
• Concept of maximum allowable total error (TEa) (process specifications)
• Critical error
Statistics & graphics for the laboratory
108
Annex – Checklists
Checklists
TEa
Apply TEa values from the following hierarchy
• 1. Clinical models (e.g., cholesterol; glucose)
• 2. Biological variation (obtain the database)
• -Bottom-line values
• No numbers from 1-2: 3. Expert models
• No numbers from 1-3: 4. Regulation
• No numbers from 1-4: 5. Better state-of-the-art
Note: Critically review the proposed numbers.
Rule selection
• Statistical basis: A rule is chosen based on Pfr and Ped. SD-limits are taken from
stable performance.
• TEa basis: From a specification for TEa, critical error values can be calculated.
From the critical error values, adequate IQC rules can be selected, naturally, on
statistical basis.
• Selection tools: power functions, OPSpecs, the TEa/CVa,tot ratio; the IQC
decision tool
Power of control rules
• Ped should be 90%
• Realistic errors to be detected by IQC are
• D SE = >2
• D RE = >3 • REstable
• Ped AND Pfr increase with
• lower s-limits (2s > 3s)
• n (note, some rules are connected to the number of materials: multiples of 2,
3 with 2 or 3 materials)
• Generally, do not consider rules with Pfr >1%
• Ped increases by combination of rules
• Ped of mean and variance rules > than single or combined rules
• Ped for SE > RE
• Pfr at non-zero can be minimized by movement of the power curve
• The power curve should have a good steepness
Statistics & graphics for the laboratory
109
Annex – Checklists
Checklists
Input elements for a successful IQC policy
• The guiding rule (regulation)
• Knowledge
– Basic statistics
– Power functions
– TE error concept (metrology)
– TEa (critical errors, specifications)
– Selection of TEa values (and problems)
• IQC Software
– Additional: rule selection tools
• Adequate samples
• Adequate frequency (and location) policy
– Cost/benefit calculations
• Instrument stability data
• Test stability and reproducibility data
• Integrated rule selection (TEa; statistics; costs)
– Release of patient data
– Process control
• Remedial actions policy
– Release of patient data
– Process control
• Integration of IQC in the overall quality management
• Dedicated personnel
Review your current practice
Which~
• Tests
– Stable performance~
– Known?
– Compared to manufacturer claim/Peers?
– Compared to medical requirements?
• Materials~
– Sort and concentration
– Targets (mean & CV)
– Frequency (run-length)
• Rules (& software)
• Remedial actions policy~
– Responsible for interpretation
– % of non-billable tests (calibration, IQC, repeat)
• Remedial actions: success-rate
Statistics & graphics for the laboratory
110
Annex – Glossary of terms
Glossary (and abbreviations) of terms – IQC
• Ped: probability of error detection
• Pfr: probability of false rejection
• TEa: total allowable error
• DSEcrit, critical systematic error: the amount of systematic error that places 5% of
results outside TEa
• DREcrit, critical random error: the amount of random error that places 5% of
results outside TEa
• ARL, average run length: the average number of runs that occur before a run is
rejected by the IQC procedure
Glossary of terms – IQC rules
Control rules with a fixed limit
a) Control rules in the form of Nz•s (N is the number of observations, z•s is a
certain number of standard deviations of the Gaussian distribution, commonly
used are 2s, 2.5s, or 3s).
13s: refers to a control rule where action is taken when one measurement falls
outside the ±3s range around the target value.
22s: refers to the situation where action is taken when two consecutive
measurements exceed the same limit, either the +2s or -2s range.
b) Range rules
R4s: refers to a situation where action is taken when the absolute difference
between the highest and the lowest result exceeds 4 • s (Westgard interpretation:
one >+2s, the other <-2s).
c) Others
7X(mean): 7 consecutive measurements lie on one side of the target
7t
: 7 consecutive measurements show a trend (increasing/decreasing)
Statistics & graphics for the laboratory
111
Annex – Glossary of terms
Glossary of terms – IQC rules
Control rules with variable limit (depending on the number of observations N),
however, a fixed probability of false rejections (Pfr): XPfr
10.05: 1 control measurement in a group of N measurements exceeds a limit based
on a Pfr of 0.05. The limit widens when N increases (tables are available).
X0.05 : The mean of a group of N measurements exceeds a limit based on a P fr of
0.05 (also called "mean" rules). The limit decreases when N increases (tables are
available).
R0.05: The range in a group of N measurements exceeds a limit based on a P fr of
0.05. The limit increases when N increases (tables are available).
Chi20.05: The ratio s2obs(N-1)/s2 exceeds the critical chi-square value at a Pfr of 0.05
(sobs is the standard deviation calculated from the control measurements, s is the
intrinsic standard deviation) (also called "variance" rules). The limit increases
when N increases (tables are available).
CS ("CUSUM"): The difference between individual results and the mean is
summed up and plotted. Interpretation is either graphical ("V" mask) or with a
numerical limit.
Statistics & graphics for the laboratory
112
Annex – References
References
Statistical textbooks
Basic
• Miller JC, Miller JN. Statistics for analytical chemistry. 3rd ed. Chichester (UK):
Ellis Horwood, 1993.
Advanced
• Sokal RR, Rohlf FJ. Biometry. 3rd ed. New York: W. H. Freeman and Company,
1995.
• Altman DG. Practical Statistics for medical research. Chapman & Hall, London,
1991.
Electronic textbook
• www.statsoft.com/textbook/stathome.htm
• http://faculty.vassar.edu/lowry/VassarStats.html
Software for laboratory statistics
• www.cbstat.com
• www.medcalc.be
• www.marquis-soft.com
IQC-websites
• www.westgard.com
• www.marquis-soft.com
IQC-software
• QC-today: www.ilww.com
• Unity: Bio-Rad
• EZ Runs: Westgard
• QC Validator. EZ-rules. Westgard Quality Corporation. www.westgard.com.
Useful links
Regulation
• www.iph.fgov.be
• www.baek.de
• www.cms.hhs.gov/clia/, and www.phppo.cdc.gov/clia/regs/toc.asp
• www.cenorm.be
• www.iso.ch
Statistics & graphics for the laboratory
113
Annex – References
References
IQC-general
• National Committee for Clinical Laboratory Standards (NCCLS).C24-A2.
Statistical quality control for quantitative measurements: Principles and definitions;
Approved Guideline – 2nd ed (1999).
• Westgard JO, Barry PL. Cost-effective quality control. AACC Press, 1995
• Shewart WA. Economic control of manufactured products. Van Nostrand: 1931.
• Levey S, Jennings ER. The use of control charts in the clinical laboratory. Am J
Clin Pathol 1950;20:1059-66.
• Westgard JO, Groth T, Aronsson T, Falk H, de Verdier C-H. Performance
characteristics of rules for internal quality control: probabilities for false rejection
and error detection. Clin Chem 1977;23:1857-67.
• Hyltoft Petersen P, Ricós C, Stöckl D, Libeer J-C, Baadenhuijsen H, Fraser CG,
Thienpont LM. Proposed guidelines for the internal quality control of analytical
results in the medical laboratory. Eur J Clin Chem Clin Biochem 1996;34:983-99.
• Linnet K. Mean and variance rules are more powerful or selective than quality
control rules based on individual values. Eur J Clin Chem Clin Biochem
1991;29:417-24.
IQC-practice surveys
• Steindel SJ, Tetrault G. Quality control practices for calcium, cholesterol, digoxin,
and hemoglobin. A College of American Pathologists Q-probes study in 505
hospital laboratories. Arch Pathol Lab Med 1998;122:401-8: Recommend 2.5 –
2.7s rule
• Tetrault GA. QC in the clinical lab: 6 questions for the pathologist. CAP Today
1995 (April):60-1: Recommends 3.5 s rule
• Steindel SJ. Quality control systems in the clinical laboratory. A survey on
implementation in 505 hospital laboratories. Labmedica Int 1999;16:8-12:
Recommends 2.5 – 2.7s rule
• Krishnan S, Webb S, Henderson AR, Cheung CM, Nazir DJ, Richardson H. An
overview of quality control practices in Ontario with particular reference to
cholesterol analysis. Clin Biochem 1999;32:93-9.
Statistics & graphics for the laboratory
114
Annex – References
References
Quality management
• ISO 8402: 1994. Quality management and quality assurance - Vocabulary
• National Committee for Clinical Laboratory Standards (NCCLS). HS1-A (replaces
GP26-A). A quality system model for health care; Approved guideline (2002).
• ISO/DIS 15189.2 Medical laboratories - Particular requirements for quality and
competence.
• Burnett D. Understanding accreditation in laboratory medicine. ACB Venture
Publications. London: Association of Clinical Biochemists, 1996.
• Stewart CE, Koepke JA. Basic quality assurance practices for clinical
laboratories. Philadelphia (USA): J. B. Lippincott Company, 1987.
• Garfield FM. Quality assurance principles for analytical laboratories. AOAC
International: 1994.
• St John A. Critical care testing. Quality assurance. Mannheim: Roche
Diagnostics, 2001.
• Nilsen CL. Managing the analytical laboratory: plain and simple. Buffalo Grove
(IL): Interpharm Press, 1996.
Metrology
• ISO VIM. Vocabulaire international des terms fondamentaux et généraux de
métrologie.
• ISO GUM. Guide to the expression of uncertainty in measurement
• ISO 5725-1. Accuracy (trueness and precision) of measurement methods and
results.
• Stöckl D. Metrology and analysis in laboratory medicine: a criticism from the
workbench. Scand J Clin Lab Invest 1996;56:193-7.
Statistics & graphics for the laboratory
115
Annex – References
References
Regulation
• Praktijkrichtlijn voor het opzetten van een kwaliteitshandboek in erkende
klinische laboratoria werkzaam binnen het kader van het RIZIV.
• Royal Decree from December 3 1999 regarding the authorization of clinical
chemical laboratories. Moniteur Belge. December 30, 1999. Implementation
document: Praktijkrichtlijn (Practice guideline):
www.iph.fgov.be/Clinbiol/NL/index.htm.
• Guidelines of the German Medical Association (Bundesärztekammer) regarding
the quality assurance in medical laboratories (RILIBÄK). Deutsches Ärzteblatt
2001;98, 42:A 2747-59 + 2002;99, 17: A 1187. www.bundesaerztekammer.de/30/Richtlinien/Richtidx/Labor2002/index.html.
• QC - THE REGULATIONS. Sharon S. Ehrmeyer, Ph.D.
www.westgard.com/guest8.htm.
• Directive 98/79/EC of the European Parliament and of the Council of 27 October
1998 on in vitro diagnostic medical devices. Official Journal of the European
Communities 1998 (Dec 7):L 331/1-L 331/37.
• CEN prEN ISO 17511 (draft Dec 2000) in vitro diagnostic medical devices Measurement of quantities in samples of biological origin - Metrological traceability
of values assigned to calibrators and control materials (ISO/DIS 17511:2000).
• CEN prEN 13612 (final draft September 2001) Performance evaluation of in vitro
diagnostic medical devices.
• ISO/DIS 15198 (April 2001). Clinical laboratory medicine - Validation of
manufacturer´s recommendations for user quality control.
• CEN EN 591 (2001) Instructions for use for in vitro diagnostic instruments for
professional use.
• CEN EN 375 (2001) Information supplied by the manufacturer with in vitro
diagnostic reagents for professional use.
• CEN prEN 14136 (draft March 2001) Use of external quality assessment
schemes in the assessment of the performance of in vitro diagnostic procedures.
• ISO/IEC 17025: 1999 - General requirements for the competence of testing and
calibration laboratories (former EN 45001).
• ISO/DIS 15189.2 Medical laboratories – particular requirements for quality and
competence
• ISO/AWI 22869 Clinical laboratory testing -- Guidance on application of ISO
15189.
• NCCLS EP11-P Uniformity of claims for in vitro diagnostic tests; Proposed
guideline (1996).
• NCCLS EP15-P User demonstration of performance for precision and accuracy;
Proposed guideline (1998).
Statistics & graphics for the laboratory
116
Annex – References
References
Dignostic value of tests
• Büttner J. Diagnostic validity as a theoretical concept and as measurable
quantity. Clin Chim Acta 1998;260:131-43.
• Linnet K. A review on the methodology for assessing diagnostic tests. Clin Chem
1988;34:1379-86.
• National Committee for Clinical Laboratory Standards. Assessment of clinical
accuracy of laboratory tests using receiver operating characteristics (ROC) plots;
approved guideline. NCCLS publication GP10-A. Wayne, PA: NCCLS 1995.
• The Bayes Library of Diagnostic Studies and Reviews. 2nd edition 2002.
http://www.ispm.unibe.ch/files/file/261.Bayes_library_handbook.pdf
• Henderson AR. Assessing test accuracy and its clinical consequences: a primer
for receiver operating characteristic curve analysis [Review]. Ann Clin Biochem
1993;30:521-39.
Performance specifications
• Stöckl D, Baadenhuijsen H, Fraser CG, Libeer J-C, Hyltoft Petersen P, Ricós C.
Desirable routine analytical goals for quantities assayed in serum. Eur J Clin
Chem Clin Biochem 1995;33:157-69.
• Stöckl D. Desirable Performance criteria for quantitative measurements in
medical laboratories based on biological analyte variation - hindrances to reaching
some and reasons to surpass some. Clin Chem 1993;39:913-4.
• www.westgard.com
• Strategies to set global analytical quality specifications in laboratory medicine.
Consensus Statement, Stockholm 1999. Scand J Clin Lab Invest 1999;59:585.
• Hyltoft Petersen, P. Quality specifications based on analysis of effects of
performance on clinical decision making. Scand J Clin Lab Invest 1999;59:517-22.
• ISO 15196/CD Identification and determination of analytical and clinical
performance goals for laboratory methodologies
Statistics & graphics for the laboratory
117
Annex
STT Consulting
Dietmar Stöckl, PhD
Abraham Hansstraat 11
B-9667 Horebeke, Belgium
e-mail: [email protected]
Tel + FAX: +32/5549 8671
Statistics & graphics for the laboratory
118