Transcript Diapositiva 1

Risk adjustment using
administrative and clinical data:
model comparison
Paola Colais
Workshop
“Challenges for epidemiology in the context of the National Health Service”
Rome, October 15th – 16th, 2012
Background



The assessment of hospital care quality has become
increasingly important in many European countries,
and worldwide, in response to requests for greater
transparency and accountability and for quality
improvement.
However, observational studies comparing groups or
populations to evaluate services or interventions
strongly require severity and comorbidity adjustment to
account for differences between the groups being
compared.
Little is known about the relative performance of
available information systems in predicting outcomes
and control for confounding.
Objective
To compare the performance of diagnosis,
drug prescription and “RAD-esito”-based
models in predicting outcomes and control
for confounding in hospital quality of care
comparative analysis.
Data sources
Data sources are:

Hospital Information System

pharmacy dispensing database

RAD-esito (AMI, CABG, Hip Fracture)
Study population
Hospital discharges in Lazio region
between January 2010 and November 2010
with a diagnosis of

Acute Myocardial Infarction (AMI)

Hip Fracture
Outcomes

Thirty-days mortality from hospital
admission was evaluated for AMI
patients

Proportion of interventions performed
within 48 hours from admission was
evaluated for hip fracture patients
Methods (1)

A multivariate regression analysis was used to
calculate adjusted hospital-specific risks
using diagnosis, drug prescription and RADesito-based predictive models, both
separately and jointly.

Factors associated with the outcomes were
selected by a bootstrap stepwise procedure to
assign an importance rank for predictors in
logistic regression
Methods (2)

Performances were measured by using the ‘c’
statistic, ranging from 0.5 for chance
prediction of outcome to 1.0 for perfect
prediction.

change-in-estimate methods were applied to
improve parsimony and gain estimates’
precision, by eliminating variables that are not
actual confounders.
Results Hip Fracture (1)
Predictive models
Risk Factors
Age
Gender (females vs males)
n
Crude
(admissions)
OR
.
1.00
4925
1.42
Diagnosisbased model
Diagnosisbased
model+RADesito-based
model
Adj OR p value
Adj OR p value
Diagnosis-based
model+RAD-esitobased model +drug
prescription-based
model
Adj OR
p value
1.00
1.40
0.420
0.000
1.00
1.37
0.440
0.000
1.00
1.37
0.450
0.000
Diabetes
Obesity (index admission)
406
41
0.66
2.36
0.72
2.21
0.036
0.016
0.71
2.29
0.021
0.013
0.69
2.33
0.011
0.011
Obesity
Hypertension
23
877
1.13
0.76
1.22
0.80
0.705
0.028
1.06
0.914
1.04
0.937
Cerebrovascular disease
561
0.73
0.77
0.036
57
4834
2.05
1.00
2.11
1.00
0.009
-
1.98
1.00
0.017
-
INR out of range
INR missing
855
659
0.57
0.83
0.57
0.78
0.000
0.024
0.57
0.78
0.000
0.024
Antiplatelet (3 month)
Anticoagulants (3 month)
851
238
0.78
0.79
0.79
0.91
0.020
0.622
Osteoporosis
INR 0.9-1.2
2.13
0.008
c statistic: 0.555
c statistic: 0.573
c statistic: 0.576
Results Hip Fracture (2)
Adjusted proportion of interventions performed within 48 hours by hospitals
Adjusted proportion
Diagnosis-based model+RAD-esito-based model
70
60
50
40
30
20
10
0
0
10
20
30
40
Diagnosis-based model
50
60
70
Results Hip Fracture (3)
Adjusted proportion of interventions performed within 48 hours by hospitals
Diagnosis-based model+RAD-esito-based model+drug
prescription-based model
Adjusted proportion
70
60
50
40
30
20
10
0
0
10
20
30
40
Diagnosis-based model
50
60
70
Results Hip Fracture (4)
Adjusted proportion of interventions performed within 48 hours by hospitals
Diagnosis-based model+RAD-esito-based model+drug
prescription-based model
Adjusted proportion
70
60
50
40
30
20
10
0
0
10
20
30
40
Diagnosis-based model+RAD-esito-based model
50
60
70
Results AMI (1)
Diagnosisbased model
Predictive models
Risk Factors
Age
Gender (females vs males)
Cancer
Diabetes
Disorders of lipoid metabolism
Blood disorders (index admission)
Blood disorders
Previous AMI
Heart failure
Other forms of ischemic heart disease (index
admission)
Other forms of ischemic heart disease
Chronic renal disease
Other chronic disease (liver, pancreas, intestine)
Previous CABG
Previous PCI
Blood pressure>100 mmHg
Blood pressure<=100 mmHg
Blood pressure missing mmHg
Diuretics (3 month)
ACE inhibitors (3 month)
Angiotensin II antagonists (AIIA) (3 month)
n
Crude
(admissions) OR
. 1.08
2690 1.66
437 1.87
829 1.54
344 0.42
393 1.39
338 2.34
1137 0.85
554 2.55
Adj OR p value
Diagnosis-based
model+RAD-esitobased model
Adj OR
p value
1.08
0.97
1.44
1.34
0.42
0.000
0.738
0.008
0.013
0.001
1.08
0.97
1.44
1.28
0.44
0.000
0.711
0.010
0.045
0.002
0.76
1.66
0.033
0.000
0.73
1.69
Diagnosis-based
model+RAD-esito-based
model +drug prescriptionbased model
Adj OR
p value
1.08
0.96
1.42
0.000
0.589
0.013
0.017
0.000
0.46
0.69
1.28
0.71
1.51
0.004
0.027
0.127
0.010
0.003
175
0.88
0.48
0.009
0.49
0.012
0.47
0.008
103
419
85
310
854
6592
765
256
1577
1977
1823
1.41
2.68
2.08
0.50
0.47
1.00
4.07
1.27
2.54
1.12
1.01
1.33
1.65
0.384
0.000
0.50
0.61
0.007
0.004
1.27
1.59
1.85
0.53
0.62
1.00
4.55
1.33
0.471
0.001
0.047
0.014
0.006
.
0.000
0.188
1.26
1.48
1.92
0.52
0.62
1.00
4.60
1.33
1.69
0.77
0.82
0.478
0.007
0.035
0.012
0.006
.
0.000
0.195
0.000
0.005
0.039
c statistic: 0.761 c statistic: 0.793
c statistic: 0.797
Results AMI (2)
Adjusted 30-days mortality after AMI by hospitals
Adjusted proportion
Diagnosis-based model+RAD-esito-based model
30
25
20
15
10
5
0
0
5
10
15
Diagnosis-based model
20
25
Results AMI (3)
Adjusted 30-days mortality after AMI by hospitals
Diagnosis-based model+RAD-esito-based model+drug
prescription-based model
Adjusted proportion
30
25
20
15
10
5
0
0
5
10
15
Diagnosis-based model
20
25
Results AMI (4)
Adjusted 30-days mortality after AMI by hospitals
Diagnosis-based model+RAD-esito-based model+drug
prescription-based model
Adjusted proportion
30
25
20
15
10
5
0
0
5
10
15
20
Diagnosis-based model+RAD-esito-based model
25
30
Results Change-in-estimate methods
Change-in-estimate methods:
Hip Fracture
No actual confounders
AMI
Age
Hospital
Information
System
Blood pressure
“RAD-esito”
system
Diuretics
drug
prescription
Critical aspects
The use of routinely collected administrative data
in comparative outcome evaluations presents
some limitations:

different coding practices across hospitals and
misclassification of comorbidity;

absence of some clinical information needed to
adjust for patients’ conditions;

some chronic comorbidities, such as
hypertension and diabetes, are known to be
currently under-reported at admission, mainly in
more severely affected patients.
Conclusions

Hip fracture no differences were found
using risk adjustment models with
different information systems because
there are no factors that are actual
confounders.

AMI small differences. Some actual
confounders from the three different
information systems were found.