Transcript Day to day Management of Quality Control problems

Day to Day Management of
Quality Control
Donna Walsh M.S. MT(ASCP)
Deaconess Hospital
Boston, MA
Why We Do Quality Control

Clients of laboratories services see the test
results we produce as information and
expect that information to be flawless, a
very tall order. Statistical Quality Control
practices is one of the tools of the trade
utilized to approach reasonable levels of
flawlessness.
Why We Do Quality Control

We are all familiar with classical approaches
to quality control and the frustrations
involved when these approaches fail and we
need to troubleshoot methodology before
releasing results.
Goal and Objective
This talk will review the basics of statistical
quality control and some 'nuts and bolts'
solutions to common QC problems.
 Case studies will be used to illustrate
effective strategies.

Goals of those
Attending Today

Improving Day to Day
QC Decisions &
Documentation and Review
of QC
Trying to get the days
work reported out
Health Care Today
Emphasis today is on cost containment
 Laboratories have become cost centers as
opposed to revenue centers.

» QC costs involve cost of QC material (special
controls can be very costly)
» QC costs involve rework of out of control runs.
(labor, reagent and control cost)
How Did We Get Here?

Levy-Jennings Plots were adapted from
industrial use and have been in use to this
very day since the 1960’s
120
110
100
5/1/95
90
80
5/6/95
5/11/95 5/16/95 5/21/95 5/26/95 5/31/95
Classic Quality Control

Principles of Statistical QC
» Control Solutions are used to make Control
Measurements
» Control Measurements validate Unknown
Measurements
» Statistical Measurements Provide Guidelines for
Acceptable Control Measurements
Classic Quality Control

Example:
120
110
100
5/1/95
90
80
5/6/95
5/11/95 5/16/95 5/21/95 5/26/95 5/31/95
trend
shift
Youden Plots
Looking at two controls at once
220
210
high control

200
190
180
80
90
100
low control
110
120
Ever Seen a
QC Chart Like This?
Dot
chart
of the
120
110
90’s
100
5/1/95
90
80
5/6/95
5/11/95 5/16/95 5/21/95 5/26/95 5/31/95
Multi-rule Quality Control
Based on the concept that improved error
detection is provided by selecting multirules over single-rule control procedures.
 Widely used multi-rule control procedure is
the one recommended by Westgard

Westgard Multi-Rule
Quality Control Scheme
control
data
12s
No
IN-CONTROL
ACCEPT RUN
No
No
22s
13s
Yes
No
Yes
No
R4s
Yes
OUT-OF-CONTROL
41s
Yes
REJECT RUN
No
10X
Yes
Available Options

Classic QC
» very visual
» tedious manual record
keeping
» often available as an
on-line QC package for
many automated
systems

Multi-Rule
» better error detection
» more cumbersome for
operator
» easily adaptable to
computer analysis
Practical “Nuts & Bolts” Options

Include Analytical Performance in Control Parameter
(mean & SD) Decisions
» Use NCCLS EP-5 to evaluate precision.
» Adapt Control Measurement Frequency to Testing Needs

Set Medically Useful Control Procedures
» Controls Levels at Medical Decision Points

Consider Control Measurements Quality Checks
» Keep track of lot numbers, calibrations & maintenance
procedures
» Use this procedural information to assign causes to failures of
quality checks
Case Studies .... practical QC

Incorporating NCCLS EP-5 data into QC decisions
NCCLS EP- 5 data
within run % of total daily % of total
Method
% CV
variance % CV variance
Carotene @ 97
12.8%
23.0% 14.7% 30.0%
Carotene @ 228
7.6%
24.0% 8.5% 29.0%
IgA @ 121
1.1%
6.0%
2.9% 45.0%
IgA @ 359
1.1%
7.0%
2.7% 44.0%
total
% CV
18.4%
10.8%
3.0%
2.8%
QC
% of total % CV
variance set @
47.0% 15.0%
47.0% 10.1%
45.0% 5.5%
48.0% 5.8%
Case Studies .... practical QC

Adapt Control Measurement Frequency to Testing
Needs
» NCCLS EP-5 calls for 2 replicates per run, 2 runs per
day for 20 day. Adapt to testing situation - Example:
Stat Lab has three shifts and three controls for Oximetry
» Adopt CLIA definition of 24 hour = one run and assay 3
reps, for assay of new lot of control
» Implement different QC levels on each shift for periods
of stable operation.
Practical ‘Nut & Bolts’ HINT

Many of today's testing
procedures have much
improved analytical
performance. You can take
advantage of this improved
accuracy and precision in
the design of your QC
protocol.
Case Studies .... practical QC

Set Medically Useful Control Procedures
» Example: Digoxin
YTD
n=
1430
level 1
level 2
level 3
target target target YTD YTD YTD
mean sd % CV mean sd % CV
0.51
1.51
3.30
0.15 29.4% 0.70 0.20 28.6%
0.20 13.2% 1.69 0.20 11.8%
0.25 7.6% 3.33 0.27 8.1%
Case Studies .... practical QC

Set Medically Useful Control Procedures
» Example: pH Reference Range is 7.37-7.44
(range
= 0.07)
» Control Material A;
standard deviation = 0.014;
+/- 2 sd = 0.056
» Control Material B;
But it is costly to walk
standard deviation = 0.005;
too fine a line...
+/- 2 sd = 0.020
» Control Material B is more
costly than Control material A
Practical ‘Nut & Bolts’ HINT


Set the % CV tight or
use a control protocol
that has improved
precision at Clinically
Significant Decision
Levels
Or put differently,
make sure you have an
appropriate target for
the situation.
Case Studies .... practical QC

Consider Control Measurements
Quality Checks
» Keep track of lot numbers, calibrations
& maintenance procedures
» Use this procedural information to assign
causes to failures of quality checks
» Best way to document assignable
causes of mean & sd changes
lot numbers
Case Studies .... practical QC

Example: Preventative Maintenance
Appropriate for Workload
» Drugs by Fluorescent Polarization
120
110
100
5/1/95
90
80
5/6/95 5/11/95 5/16/95 5/21/95 5/26/95 5/31/95
Case Studies .... practical QC

Increase PM frequency
120
110
100
5/1/95
90
80
5/6/95 5/11/95 5/16/95 5/21/95 5/26/95 5/31/95
Case Studies .... practical QC

Example: new lot number of reagent
120
110
100
5/1/95
90
80
5/6/95
5/11/95 5/16/95 5/21/95 5/26/95 5/31/95
Case Studies .... practical QC

Example: new lot number of reagent
» shift in QC values correlated with
change in lot of reagent.
» check to see if there is a shift in
unknowns (samples) or standards
» matrix effect if shift is only in control
samples
» adjust QC mean to account for shift and
avoid unnecessary repeat work.
Lot numbers
5-1-95 xx0226
5-15-95 xx0228
Case Studies .... practical QC

Long standing shifts may not be obvious until a
lot change causes a shift out of control
78.0
75.0
72.0
5/1/95
69.0
66.0
5/11/95
5/21/95
lot change
5/31/95
Check Year
to Date and
or lot to
date QC for
any shifts or
trends.
Case Studies .... practical QC

Example: Glucose - shift in current month away
from year to date when new electrode used.
YTD target target YTD YTD month month
n= 398 mean sd
mean sd mean
sd
level 1
level 2
72
261
3.0
7.0
74.5 2.20 76.3
257.5 5.80 260.1
2.2
6.7
Case Studies .... practical QC

Example: Change in QC post calibration of
new lot of Calcium
9.30
9.15
9.00
5/1/95
8.85
8.70
5/11/95 5/21/95 5/31/95
calibration done
Lot related
shift post cal
on 5-10.
Recal on 5-20
due to bias in
pt. checks
Case Studies .... practical QC

Example of shift with no assignable cause
» Lithium by ISE
0.83
0.80
0.77
5/1/95
0.74
0.71
0.68
5/11/95
5/21/95
5/31/95
Trouble shooting
- new control
- recalibration
- new electrode
- fresh reagents
NOTHING
WORKED!!!
Case Studies .... practical QC

Lithium Statistical QC information
YTD
n = 274
target
YTD
monthly
mean sd % CV
0.78 0.03 3.8%
0.77 0.02 2.6%
0.74 0.06 8.1%
Practical ‘Nut & Bolts’ HINT

Running control samples is a check on quality;
they do not control quality. You must do that
yourself.
REPORT
RESULTS!
There is
NO such
thing as a
QC Crystal
Ball
Recommendations

View the test results we produce as information
» Include Analytical Performance in Control Protocols
» Set Medically Useful Control Procedures

Make use of on-line QC packages
» facilitates statistical quality control calculations
» provides visual dot charts for operators
Recommendations

Consider Control Measurements Quality Checks
» Keep track of information that could validate a
change in the statistical QC parameters
» Change QC parameters promptly when the situation
warrents
» Use Patient Check Data to help in QC decisions
Benefits
Accurate and Reliable Reporting of Patient Test
Data
 Well Documented Quality Control Program
 Day to Day and Cumulative QC data will be more
useful.
