Transcript Time line for Bristol

The Use of Control Charts in
Health Care Monitoring and Public
Health Surveillance
William H. Woodall
Department of Statistics
Virginia Tech
Blacksburg, VA 24061-0439
[email protected]
1
Topics to be covered:
 Some
general issues
 Risk adjustment
 Examples of some of the plots used for
monitoring
 Detection of clusters of chronic disease
 League tables, control charts, and funnel
charts for cross-sectional data
 Conclusions
2
In 1999 the Institute of Medicine reported the
number of deaths due to medical errors in
U.S. hospitals to be 44,000 to 98,000 per year.
Some prefer the term “preventable adverse
events.”
3
Examples of health care variables










Lab turnaround time
Days from positive mammogram to definitive
biopsy
Patient satisfaction scores
Medication error counts
Emergency service response times
Infection rates
Mortality rates
Number of patient falls
Post-operative length of stay
“Door-to-needle” time ……and many others… 4
Control charts are used to
understand variation over time and
to detect unusual events and trends.
They are most effective in a hospital
when used as a part of its organized
quality improvement program, such
as Six-Sigma.
5
I Chart of Transformed Time
3.0
UCL=2.744
Individual Value
2.5
2.0
1.5
_
X=1.334
1.0
0.5
0.0
LCL=-0.077
5
10
15
20
25
30
Observation
35
40
45
50
6
P Chart of Percentage
0.40
1
0.35
Proportion
0.30
UCL=0.2999
0.25
_
P=0.2016
0.20
0.15
LCL=0.1033
0.10
5
10
15
20
25
30
Sample
35
40
45
50
7
Xbar Chart of QC Variable
102.5
UCL=102.137
102.0
Sample Mean
101.5
101.0
__
X=100.492
100.5
100.0
99.5
99.0
LCL=98.847
2
4
6
8
10
12 14
Sample
16
18
20
22
24
8
CUSUM Chart of QC Variable
UCL=2.194
Cumulative Sum
2
1
0
0
-1
-2
LCL=-2.194
2
4
6
8
10
12 14
Sample
16
18
20
22
24
9
Some Types of Control Charts
 Shewhart
X-bar and R charts (n>1)

Normality
 Shewhart I and MR charts (n=1)

Normality
 Shewhart p-charts for proportions
Binomial
 Shewhart c- and u-charts for counts
Poisson
 Cumulative sum (CUSUM) charts
10
Our focus will be on the
monitoring of chronic
diseases, congenital
malformations, and mortality
rates over time.
11
Health care quality data are often
 attribute
(yes/no) data with 100%
inspection.
 counts
or times to a “failure” with an
assumed underlying Bernoulli,
geometric or exponential distribution.
12
Suppose one counts the number of
births between successive cases of a
specific type of congenital malformation.
The sets method of Chen (1978, JASA)
signals an increase in the rate if a
specified number of consecutive counts
are all less than a specified value.
For example, signal if 5 consecutive
values are less than 1000.
13
Risk-adjustment is often essential
in health care applications, where a
logistic or other model is used to
predict the probability of “failure.”
…oooooooooooooooooooo
…oD89pkej589Cv0238&*&%#$
14
Examples of Risk Factors
Down’s Syndrome: Age of mother
 Heart Surgery: Age, gender, hypertension,
diabetic status, renal function, left ventricular
mass. (Parsonnet score)
 Heart Surgery (Europe): Age, gender, chronic
pulmonary disease, extracardiac arteriopathy,
neurological dysfunction, previous cardiac
surgery, creatinine > 200 µmol/ L, active
endocarditis, critical preoperative state.
(euroSCORE)

15
Much of the focus and work on
mortality rate monitoring for
physicians is being done in the
UK and Canada.
16
17
18
Shipman
Inquiry July
2002:
215 definite
victims,
45 probable
19
200
180
160
140
120
100
80
60
40
20
0
-20
Male
1997
1995
1993
1991
1989
1987
1985
1983
1981
1979
Female
1977
Cumulative excess mortality
Cumulative excess death certificates signed by Shipman: age
>64 and death in home/practice
Year
(Shipman Inquiry: total of definite or probable victims:
189 female > 65, 55 male over 65)
20
Sequential probability ratio test (SPRT) for detection of a
doubling in mortality risk: age >64 years and death in
home/practice for Dr. Harold Shipman. (Spiegelhalter et
21
al. (2003))
Resetting sequential probability ratio test (RSPRT)
for detection of a doubling in mortality risk, age
22
>64. (From Spiegelhalter et al. (2003).
RSPRT charts have a problem with
building up “credit”.
An increase in the mortality rate can
occur when the SPRT value is below
zero.
This phenomenon is referred to as
“inertia” in the industrial SPC
literature.
23
Cumulative risk adjusted mortality (CRAM) chart
with 99% control limits for change in mortality in
last 16 expected deaths. (From Poloniecki et al.
(1998))
24
0
500
1000
1500
2000
2500
3000
3500
0
500
1000
1500
2000
2500
3000
3500
CUSUM X
t
+
6
4
0
0
-
CUSUM X
t
2
-2
-4
-6
Number of Patients
Example of a two-sided risk-adjusted CUSUM
chart (provided by Stefan H. Steiner)
25
The CUSUM chart is the best option.
 It
can be risk-adjusted.
 It has optimality properties in detecting
sustained shifts in the process.
 It has good inertial properties.
 It can be designed based on
meaningful performance measures
such as average run length (ARL).
 It can be used in the background with
CRAM charts.
26
Control charts can be used to identify
physicians or hospitals with unusually high
(or low) mortality rates.
The Society of Cardiothoracic Surgeons of
Great Britain and Ireland interprets giving the
benefit of the doubt to physicians as 9999:1
odds of adverse outcomes being due to
chance alone before any alarm.
27
The Centers for Disease Control and
Prevention use CUSUM and other
control charting methods in their
Early Aberration Reporting System
(EARS).
www.bt.cdc.gov/surveillance/ears/index.asp
28
Virtually all methods for the
detection of clusters of disease are
retrospective, based on historical
spatial data.
There are some new methods for
detecting clusters prospectively, i.e.,
as they are forming.
29
30
Detection of clusters of chronic disease
 Aggregation
of data by time and location
Raubertas (1989, Statistics in Medicine)
Rogerson and Yamada (2004, Statistics in Medicine)
 Aggregation
of data by location
Rogerson (1997, Statistics in Medicine)
 No
aggregation
Rogerson (2001, JRSS-A)
31
It is often useful to compare
units, e.g., institutions or
physicians, using cross-sectional
data.
32
33
Example of a League Table from Adab et al. (2002).
34
Example of Proposed “Control Chart” by Adab et al. (2002)
35
"Funnel plot" of emergency re-admission rates following
treatment for a stroke in large acute or multi-service
hospitals in England and Wales in 2000–1. Exact 95% and
99.9% binomial limits are used. (From Spiegelhalter
36
(2002))
Harold Shipman





Killed his patients using morphine overdoses.
Was caught after carelessly revising a patient’s
will, leaving all her assets to himself.
His office typewriter was used to type the
revised will.
His computer records were doctored to show
his patients had needed morphine just after the
patients had been killed. The computer
software, however, recorded the dates of these
modifications.
He hung himself in prison, never confessing to
his crimes.
37
Baker, R. et al. British Medical Journal 2003;326: pp. 274-276
38
Highly recommended reference:
Michael L. Millenson (1999). Demanding
Medical Excellence: Doctors and
Accountability in the Information Age,
The University of Chicago Press.
39
Recommended References




Sonesson, C. and Bock, D. (2003). “A Review and
Discussion of Prospective Statistical Surveillance in Public
Health”. Journal of the Royal Statistical Society A 166, pp.
5-21.
Grigg, O. A.; Farewell, V. T.; and Spiegelhalter, D. J. (2003).
“Use of Risk-adjusted CUSUM and RSPRT Charts for
Monitoring in Medical Contexts”. Statistical Methods in
Medical Research 12, pp. 147-170.
Grigg, O. and Farewell, V. (2004a). “An Overview of RiskAdjusted Charts”. Journal of the Royal Statistical Society A
167, pp. 523-539.
Steiner, S. H.; Cook, R. J.; Farewell, V. T.; and Treasure, T.
(2000). “Monitoring Surgical Performance Using RiskAdjusted Cumulative Sum Charts”. Biostatistics 1, pp. 44140
452.
My paper is available at
http://filebox.vt.edu/users/bwoodall/
41
Conclusions
 There
are many important applications of
control charts in health care.
 Improvement of health care is a life-or-death
matter.
 There are many interesting SPC research
opportunities in public health surveillance.
 There needs to be a greater transfer of
knowledge between the medical and
industrial application areas.
42