Transcript Propensity Score Balancing score

Advanced Statistics
for Interventional
Cardiologists
What you will learn
•
•
•
•
•
•
•
•
•
•
•
•
Introduction
Basics of multivariable statistical modeling
Advanced linear regression methods
Logistic regression and generalized linear model
Multifactor analysis of variance
Cox proportional hazards analysis
Propensity analysis
Bayesian methods
Resampling methods
Meta-analysis
Most popular statistical packages
Conclusions and take home messages
Year
2006
2005
2004
2003
2002
2001
2000
1999
1998
1997
1996
1995
1994
1993
1992
1991
1990
1989
1988
1987
1986
1985
1984
1983
Number of publications
Background
Publications in Pub Med with phrase "Propensity Score"
180
160
140
120
100
80
60
40
20
0
In randomized trials…
• All patients have a specified chance of
receiving each treatment
• Treatments are concurrent
• Data collection is concurrent, uniform,
and high quality
• All patient covariates, measured or
unmeasured, are expected to be
balanced between the two treatment
groups
In randomized trials…
• Statistical assumptions underlying
comparison tests are met
• Data collection is concurrent, uniform,
and high quality
• The two groups are comparable and
observed treatment difference is an
unbiased estimate of true treatment
difference
However…
• The above advantages are not
guaranteed for small, poorly designed or
poorly conducted randomized trials
• Large randomized trials take a long time
and great cost to generate answers
(analysis of existing data may be more
timely, yet acceptably accurate)
• Randomized trials are not always
feasible, e.g. when variables cannot be
manipulated (smoke…)
Non-randomized studies
• None of advantages provided by
randomized trials is available in nonrandomized studies
• A potential problem: two treatment
groups are not comparable before the
start of treatment, i.e. not comparable
due to imbalanced covariates between
two treatment groups
• So, direct treatment comparisons are
invalid
Adjustments for covariates
• Three common methods of adjusting
for confounding covariates:
– Matching
– Stratification
– Regression adjustment
Question: When there are many
confounding covariates needed to adjust for:
– Matching: based on many covariates is not
practical
– Stratification: is difficult, as the number of
covariates increases, the number of strata
grows exponentially:
• 1 covariate: 2 strata  5 covariates: 32 (25) strata
– Regression adjustment: may not be
possible: potential problem: over-fitting
Propensity score
• Replace the collection of confounding
covariates with one scalar function of
these covariates
Age
Gender
Ejection fraction
Risk factors
Lesion characteristics
…
1 composite covariate:
Propensity Score
Balancing score
Propensity score
• Propensity score: conditional
probability of receiving Treatment A
rather than Treatment B, given a
collection of observed covariates
• Purpose: simultaneously balance
many covariates in the two treatment
groups and thus reduce the bias
What you will learn
•
Propensity analysis
– Building a propensity score
– Exploiting a propensity score
Propensity score construction
Statistical modeling of the relationship
between treatment membership and covariates
– Statistical method: multiple logistic regression
– Outcome: actual treatment membership
– Predictor variables: all measured covariates,
and even some interaction terms: e.g. age,
gender, ejection fraction, risk factors, previous
history, lesion characteristics…
Propensity score construction
– Outcome variable of interest (e.g. MACE or
TLR or restenosis) is NOT involved in the
modeling
– No concerns regarding over-fitting
– A propensity score model is obtained: it is a
mathematical equation:
PS = f (age, gender, risk factors, …)
– Calculate, through this equation, estimated
propensity scores for all patients
Propensity score properties
• A group of patients with the same
propensity score are equally likely to have
been assigned to treatment A
• Within a group of patients with the same
propensity score, some patients actually
got treatment A and some got treatment B,
just as they had been “randomly” allocated
to whichever treatment they actually
received
Propensity score properties
(Sub-)Group with the
same propensity score
Treatment A
Treatment B
– When the propensity scores are balanced
across two treatment groups, the distribution of
all the covariates are balanced in expectation
across the two groups
– Use the propensity scores as a diagnostic tool
to measure treatment group comparability
– If the two treatment groups overlap well
enough in terms of the propensity scores, we
compare the two treatment groups adjusting
for the PS
Comparability
Estimated Propensity Score
1.0
0.8
0.6
0.4
0.2
0.0
Ctl
Trt
No comparison possible…
Cosgrave et al, AJC 2005
Cosgrave et al, AJC 2005
Variables in the Equation
a
SEX
AGE
PRE MI
PRE BYPASS
PRE PTCA
BP
CHOL
DIABETES
UNSTABLE
VESSEL
IABP
IIIBIIA
CALCIUM
THROMBUS
ECCENTRIC
BIFURCATION
TANDEM LESION
RVD PRE
LESION LENGHT
FINAL BALLOON
DIAMETER
FINAL
PRESSURE
FINAL STENT
LENGTH
N. STENT X
PATIENT
TOTAL STENT
LENGTH
IVUS
Constant
a.
B
-,206
,001
-,008
,802
-,566
,252
,207
-,109
-,169
,007
,558
-1,002
-,069
,833
,137
,123
1,666
-,112
,004
S.E.
,318
,009
,202
,257
,222
,219
,215
,246
,235
,035
,582
,302
,240
,531
,208
,236
,681
,232
,011
Wald
,420
,014
,002
9,707
6,474
1,318
,928
,195
,515
,044
,919
10,995
,082
2,460
,435
,271
5,983
,232
,110
,657
,352
-,023
df
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
Sig.
,517
,907
,968
,002
,011
,251
,335
,659
,473
,833
,338
,001
,775
,117
,510
,603
,014
,630
,740
Exp(B)
,813
1,001
,992
2,230
,568
1,287
1,230
,897
,845
1,007
1,747
,367
,934
2,299
1,147
1,131
5,291
,894
1,004
3,481
1
,062
1,929
,033
,464
1
,496
,978
-,019
,017
1,227
1
,268
,981
-,069
,255
,074
1
,785
,933
,005
,010
,297
1
,585
1,005
,618
-1,242
,298
1,363
4,319
,831
1
1
,038
,362
1,856
,289
Beta is a
regression
coefficient
Y = C + B1*X1 + B2*X2 + B3*X3 + …
PS = -1.242 + (-0.206*sex [0,1]) +
(0.01*age) +… + (0.618*IVUS [0,1])
We obtain a parameter, specific for each patient,
and expressed in logit units
To transform it in probability (included between 0 and 1)
p = 1 / (1 + e-PS)
This is the probability (propensity)
to receive stent A vs. stent B
What you will learn
•
Propensity analysis
– Building a propensity score
– Exploiting a propensity score
Propensity score methods
Fit propensity score model
using all measured covariates
Estimate propensity score for all
patients using propensity model
Compare treatments
adjusting for propensity scores
Compare treatments with
propensity score
• Three common methods of using the
propensity score to adjust results:
– Matching
– Stratification
– Regression adjustment
Matching
PS
A
vs.
B
PS1
PS2
PSm
• Compare treatments based on matched pairs
• Problem: may exclude unmatched patients
Mauri et al, NEJM 2008
Mauri et al, NEJM 2008
Stratification
– All patients are sorted by propensity scores
– Divide into equal-sized subclasses
1
2
……...
5
PS
– Compare two treatments within each subclass,
as in a randomized trial; then estimate overall
treatment effect as weighted average
– It is intended to use all patients
– But, if trial size is small, some subclass may
contain patients from only one treatment group
Brener et al, Circulation 2004
Brener et al, Circulation 2004
Brener et al, Circulation 2004
Regression adjustment
Treatment effect estimation model fitting:
the relationship of clinical outcome and
treatment
Outcome: Clinical outcome, e.g., MACE, TLR, …
Predictor variables: treatment received, propensity
score, (a subset of important covariates)
Statistical method: e.g., logistical regression, Cox
proportional hazards analysis
Brener et al, Circulation 2004
Brener et al, Circulation 2004
Practical Issues
•
Issues in propensity score estimation
– How to handle missing baseline covariate values
– What terms of covariates should be included
– Evaluation of treatment group comparability
•
Issues in treatment comparison
– Which method: matching, stratification, regression
•
Issues in study design with propensity analysis
– Pre-specified vs. post hoc propensity analysis
– Pre-specify the covariates needed to collect in the
study and then included in propensity score estimation
– Sample size estimation adjusting for the propensity
scores
Limitations
• Propensity score methods can only adjust
for observed confounding covariates and
not for unobserved ones
• Propensity score is seriously degraded
when important variables influencing
selection have not been collected
• Propensity score may not eliminate all
selection bias
Limitations
• Propensity score methods work better in
larger samples
• Propensity score methods lack the discipline
and rigor of randomized trials
• Randomized trials remain the highest level
of evidence for comparison
Conclusions
• Propensity score is a technique that allows the
creation of a single confounding covariate that
permits simultaneous adjustment for many
covariates thus reducing bias
• Propensity score methodology is an addition to,
not a substitute of traditional covariate adjustment
methods
• Randomized studies are still preferred and
strongly encouraged whenever possible!
References
• Blackstone, EH, Comparing apples and oranges.
Journal of Thoracic and Cardiovascular Surgery
2002;1:8-15.
• Rubin, DB, Estimating casual effects from large
data sets using propensity scores. Annals of
Internal Medicine 1997;127:757-763.
• D’agostino, RB, Jr., Propensity score methods for
bias reduction in the comparison of a treatment
to a non-randomized control group. Statistics in
medicine 1998;7:2265-2281.
Questions?
For further slides on these topics
please feel free to visit the
metcardio.org website:
http://www.metcardio.org/slides.html