Transcript Use of Whole Population Registers:

Beware of Registries
for their Biases
Hasan Yazici
University of Istanbul
Disclosures
• Pfizer (Turkey) – travel support & speaker’s
fees
• Merck (Turkey) – travel support & speaker’s
fees
Beware of Observational
Studies based on Registries
for their Biases
Hasan Yazici
University of Istanbul
Plan
• For and against observational studies
• Few historical notes
• A summary of important biases in
observational studies based on registries and
administrative data bases
• The bias of determining cancer incidence in
registries - a recent example
• Some naive arithmetic
• What to do?
• In brief
Observational Studies
Advantages
•
•
•
•
•
Real life data
Relatively cheap
Potential to observe multiple outcomes
Ability to assess cause – effect relations
Long term observations
Observational Studies
Problems
• Many are cross sectional or retrospective.
• Selection bias including completeness of
recruitment
• Confounders
• Uniformity of assessment
• Control groups
RCTs vs Observational studies
• Efficacy : RCT superior
• Harm: Observational study superior
• Biomarkers: Observational study superior
JP Vandenbroucke BMJ, 2011
RCTs vs Observational studies
• Efficacy : RCT superior ?
• Harm: Observational study superior
• Biomarkers: Observational study superior
JP Vandenbroucke BMJ, 2011
The DES* Drama
• DES began to be used for threatened abortion (1949)
• Efficacy could not be shown in a double blind, placebo controlled study at
University of Chicago (1958)
• An “epidemic” of vaginal cancer among girls in Boston (1971)
• Among 8 patients with vaginal cancer 7 were daughters of mothers who
had used DES during pregnancy. This contrasted with mothers of 32
healthy girls (born in the same hospital within a day or two) among whom
there were no DES users.
* diethylstilbestrol
The DES* Drama
• DES began to be used for threatened abortion (1949)
• Efficacy could not be shown in a double blind, placebo controlled study at
University of Chicago (1958)
• An “Epidemic” of vaginal cancer among girls in Boston (1971)
• Among 8 patients with vaginal cancer 7 were daughters of mothers who
had used DES during pregnancy. This contrasted with mothers of 32
healthy girls (born in the same hospital within a day or two) among whom
there were no DES users.
• FDA banned DES in pregnancy (1971).
• A study (Mayo Clinic) in mid 70’s among 800 young women with a mother
who had used DES during pregnancy did not reveal any cases of vaginal
cancer.
• No surprise. Since frequency of vaginal cancer: <1/1000 users.
• A further case control study from NY State Tumor Registry confirmed the
association.
* diethylstilbestrol
The Estrone Drama
• “Like a gallant knight (the author of Feminine
Forever) has come to rescue his fair lady not at the
time of her bloom and flowering but in her desparing
years; at a time of life when the preservation and
prolongation of her femaleness are so
paramount…By throwing down his gauntlet, he
challanges the reluctant physician to follow him in
providing the hormones that may allow for a
smoother transition to the menopausal years ahead.
Women will be emancipated only when the shackles
of hormonal deprevation are loosed.”
in Investigating Disease Patterns, Stolley&Lasky, 1995
in Investigating Disease Patterns, Stolley&Lasky, 1995
....I think this is rather misleading. Of the 26
lymphomas they refer to in the main text, 14
occurred within 2 months of TNF antagonist
use. Thus, a more realistic comparator would
be a deduced (from the annual rates)
2-month incidence of lymphoma in the
general population....
H Yazici Arthritis Rheum 2003
Time to Neoplasia
Based on T Bongartz et al. JAMA 2006
The Wandering Comparison of Risk
(Lymphoma)
M. Hudson & S. Suissa Arthritis Care and Res, 2010
The Wandering Comparison of Risk
(Lymphoma)
M. Hudson & S. Suissa Arthritis Care and Res, 2010
The Wandering Comparison of Risk
(Infections)
The Wandering Comparison of Risk
(Infections)
Channeling Bias
(in an administrative data base)
Channeling Bias
(in an administrative data base)
Channeling Bias
(in an administrative data base)
The Immortal Time Bias
• ..arises in a cohort study where an outcome
can hinder, totally or partially, the realization
of the exposure.
The Immortal Time Bias
• Hydroxychloroquine decreases cancer in SLE
by 85%
• G. Ruiz Irastorza et al. Ann Rheum Dis, 2007
• Statins decrease lung cancer by 45%.
• V. Khuarana et al. Chest, 2007
• LE Levesque et al. Br Med J, 2010
References
•
•
•
•
•
•
•
•
•
•
•
•
Eur Respir J (4)
Arch Intern Med (2)
Am J Med (2)
Am J Respir Crit Care Med (2)
Lancet (2)
JAMA (1)
Am J Respir Med (1)
J Allergy Clinical Immunol (1)
Ann Allergy Asthma Immunol (1)
Thorax (1)
Pediatrics (1)
Ann Pharmacother
(1)
• Diabet Med (1)
Misclassified Immortal Time
• The authors study a complete registry of SLE
patients between 2 time points.
• The exposed group consists of patients who
had ever used hydroxychlorquine.
• The non – exposed group consists of patients
who have never used hydroxychloroquine.
• The authors find significantly less cancers in
the “exposed = ever used” group.
OR for death =
Odds of death in the exposed
Odds of death in the non-exposed
What is wrong?
• In the numerator those patients who had died due to
malignancy could not have been prescribed
hydroxychloroqine.
• Thus the duration of follow up time that can lead to
malignancy in the numerator is actually shorter and
this decreases the odds for a malignancy in the
numerator making it lower than what is said.
The RDPR Study
•
•
•
•
•
We hypothesize that topical antifungals (TA) decrease mortality.
In the Registryville Drug Prescriptions Registry (RDPR) between January 1, 2009 and
ending on March 31st, 2009 we identify 1000 patients with an ever prescription for TA.
We follow all patients fom the time of prescription to December 31st 2010, for
emerging death.
We use the RMR (Registryville Mortality Registry) to confirm the deaths.
From RDPR we also randomize a 1000 sample of age, gender and practically everything
else matched individuals AND also follow them for the same outcome up to December
31st. 2010.
• Exposed group: patients with a prescription for a TA
• Non- exposed group: patients with other prescriptions
• Outcome: death
The RDPR Study
• At the end of the study we compare:
OR (exposed)/OR (non-exposed)
• 30 people (30/732 pt. yrs.) in the exposed
group;
• 60 people (60/950 pt. yrs.) in the nonexposed group have dies.
• TA significantly lessens mortality
• OR=0.50; p= 0.009
Excluded Immortal Time
• We correctly exclude from the numerator the
follow up of those patients before they were
prescribed the exposure drug. This is the
immortal time and, again, deaths can only
happen after the exposure (the prescription).
Excluded Immortal Time
• We correctly exclude from the numerator the
follow up of those patients before they were
prescribed the exposure drug. This is the
immortal time and, again, deaths can only
happen after the exposure (the prescription).
• However this is not enough. This time period
should be added to the denominator.
Exposed
Exposed
Entry
Exposed
Entry
Calendar
Age
Birth date
Disease onset
Diagnosis
Registration in a clinic
Registration in a database
Exposed
Exposure
ie prescription
Entry
Calendar
Age
Birth date
Disease onset
Diagnosis
Registration in a clinic
Registration in a database
Exposed
Exposure
ie prescription
Entry
Calendar
Age
Birth date
Disease onset
Diagnosis
Registration in a clinic
Registration in a database
Observation ends
Exposed
Unexposed
Exposure
ie prescription
Entry
Calendar
Age
Birth date
Disease onset
Diagnosis
Registration in a clinic
Registration in a database
Observation ends
Exposed
Unexposed
Exposure
ie prescription
Observation ends
Entry
Calendar
Age
Birth date
Disease onset
Diagnosis
Registration in a clinic
Registration in a database
Entry
Observation ends
Exposed
Unexposed
Exposure
ie prescription
Observation ends
Entry
immortal time
Calendar
Age
Birth date
Disease onset
Diagnosis
Registration in a clinic
Registration in a database
Entry
Observation ends
Exposed
Unexposed
Exposure
ie prescription
Observation ends
Entry
immortal time
Calendar
Age
Birth date
Disease onset
Diagnosis
Registration in a clinic
Registration in a database
Entry
Observation ends
Exposed
Unexposed
Exposure
ie prescription
Observation ends
Entry
immortal time
Calendar
Age
Birth date
Disease onset
Diagnosis
Registration in a clinic
Registration in a database
Entry
Observation should end
Exposed
Unexposed
Exposure
ie prescription
Observation ends
Entry
immortal time
Calendar
Age
Birth date
Disease onset
Diagnosis
Registration in a clinic
Registration in a database
Entry
Observation should end
outcome free!
Remedy (s)
• There might be none especially when we are
unclear about the entry, hence the
observation time.
• Always include the “immortal time” to the
duration of follow up in the non-exposed.
The RDPR Study
• At the end of the study we compare:
OR (exposed)/OR (non-exposed)
• 30 people (30/732 pt. yrs.) in the exposed
group;
• 60 people (60/950 pt. yrs.) in the nonexposed group have died.
• TA significantly lessens mortality
• OR=0.50; p= 0.009
The RDPR Study
• At the end of the study we compare:
OR (exposed)/OR (non-exposed)
• 30 people (30/732 pt. yrs.) in the exposed
group;
• 60 people (60/1200 pt. yrs.) in the nonexposed group have died.
• TA does nothing to the mortality, OR=0.98
S Suissa Pharmacoepidemiol Drug Saf, 2007
… which includes adding the immortal
time to the none- exposed arms in either
type of immortal time bias - HY
S Suissa Pharmacoepidemiol Drug Saf, 2007
H Yazici et al Arthritis Rheum , 2011
A Possible Source of Error
in the Method of Cancer Risk Estimation
in Patients with Rheumatoid Arthritis
• We surveyed the PubMed between years 20012011
• Search terms “registry” “cancer” “rheumatoid
arthritis”
• Papers that reported comparative incidence in at
least 2 time points were included
• Change over time was assessed
H Yazici et al. ACR, 2011
A Possible Source of Error
in the Method of Cancer Risk Estimation
in Patients with Rheumatoid Arthritis
• 1274 articles
• 36 articles retrieved
• 6 reported incidence comparison at
multiple timepoints
• The last SIR was less than the initial by 40
to 99%
H Yazici et al. ACR, 2011
A Possible Source of Error
in the Method of Cancer Risk Estimation
in Patients with Rheumatoid Arthritis
* initial year excluded
H Yazici et al. ACR, 2011
Some naive arithmetic
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
Given:
•
•
•
•
•
•
•
•
There are no biologic associations between RA and cancer
Incidence of RA: 1/1000/year
Incidence of cancer: 5/1000/year
These incidences remain same over time.
1000 new cases of RA enter the registry every year.
2/5 of all cancer patients die at the end 1 year.
Of the 2/5 who die 1 dies before and the other dies after developing RA.
Another 1/10 of the cancer patients die, in a logarithmically decreasing
fashion, at the end of 5 years. *
• RA does not kill if not associated with cancer.
• All other causes of death are ignored.
* This group is ignored for the purposes of this presentation.
Its inclusion would only support our hypothesis.
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
• Expected: 5/1000
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
• Expected: 5/1000
• Observed: 4/1000
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
• Expected: 5/1000
• Observed: 4/1000
1/5 cancer patients die before
having the chance to develop RA.
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
• Expected: 5/1000
• Observed: 4/1000
• O/E: 0.80
1/5 cancer patients die before
having the chance to develop RA.
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
•
•
•
•
•
Expected: 5/1000
Observed: 4/1000
O/E: 0.80
2. year:
Expected: 15/2000
1/5 cancer patients die before
having the chance to develop RA.
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
•
•
•
•
•
Expected: 5/1000
Observed: 4/1000
O/E: 0.80
2. year:
Expected: 15/2000
1/5 cancer patients die before
having the chance to develop RA.
5 cancer patients are from the 1.year &
10 from the 2. year, from the total of 2000
patients
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
•
•
•
•
•
•
Expected: 5/1000
Observed: 4/1000
O/E: 0.80
2. year:
Expected: 15/2000
Observed: 11/2000
1/5 cancer patients die before
having the chance to develop RA.
5 cancer patients are from the 1.year &
10 from the total of 2000 patients
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
•
•
•
•
•
•
Expected: 5/1000
Observed: 4/1000
O/E: 0.80
2. year:
Expected: 15/2000
Observed: 11/2000
1/5 cancer patients die before
having the chance to develop RA.
5 cancer patients are from the 1.year &
10 from the total of 2000 patients
3 cancer patients cases are from the 1. year.
We expect 10 incident cancer cases in the 2.
year. 2/10 of these die before they get the
chance to develop RA. The remaining 8 new
cases plus the 3 from the 1. year make up 11
prevalent cases at the end of 2 years.
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
•
•
•
•
•
•
•
Expected: 5/1000
Observed: 4/1000
O/E: 0.80
2. year:
Expected: 15/2000
Observed: 11/2000
O/E: 0.73
1/5 cancer patients die before
having the chance to develop RA.
5 prevalent cases are from the 1.year &
10 from the total of 2000 patients
3 cancer patients cases are from the 1. year.
We expect 10 incident cancer cases in the 2.
year. 2/10 of these die before they get the
chance to develop RA. The remaining 8 new
cases plus the 3 from the 1. year make up 11
prevalent cases at the end of 2 years.
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
•
•
•
•
•
•
•
•
•
Expected: 5/1000
Observed: 4/1000
O/E: 0.80
2. year:
Expected: 15/2000
Observed: 11/2000
O/E: 0.73
3. year:
Expected: 30/2000
1/5 cancer patients die before
having the chance to develop RA.
5 cancer patients cases are from the 1.year &
10 from the total of 2000 patients
3 cancer patients cases are from the 1. year.
We expect 10 incident cancer cases in the 2.
year. 2/10 of these die before they get the
chance to develop RA. The remaining 8 new
cases plus the 3 from the 1. year make up 11
prevalent cases at the end of 2 years.
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
•
•
•
•
•
•
•
•
•
•
Expected: 5/1000
Observed: 4/1000
O/E: 0.80
2. year:
Expected: 15/2000
Observed: 11/2000
O/E: 0.73
3. year:
Expected: 30/2000
Observed: 15/3000
1/5 cancer patients die before
having the chance to develop RA.
5 cancer patients are from the 1.year &
10 from the total of 2000 patients
3 cancer patients cases are from the 1. year.
We expect 10 incident cancer cases in the 2.
year. 2/10 of these die before they get the
chance to develop RA. The remaining 8 new
cases plus the 3 from the 1. year make up 11
prevalent cases at the end of 2 years.
Cumulative Cancer Prevalence
in a Hypothetical RA Registry of Incident Cases of RA
• 1. year:
• Expected: 5/1000
• Observed: 4/1000
• O/E: 0.80
• 2. year:
1/5 cancer patients die before
having the chance to develop RA.
5 cancer patients are from the 1.year &
10 from the total of 2000 patients
• Expected: 15/2000
• Observed: 11/2000
• O/E: 0.73
• 3. year:
• Expected: 30/2000
• Observed: 15/3000
• O/E: 0.50
3 cancer patients are from the 1. year. We
expect 10 incident cancer cases in the 2. year.
2/10 of these die before they get the chance to
develop RA. The remaining 8 new cases plus
the 3 from the 1. year make up 11 prevalent
cases at the end of 2 years.
What to do?
• Be careful when comparing the frequency of a condition
in a registry with the frequency of the same condition in
the population from which that registry comes from;
because of the selection bias.
• Give more importance to comparing like with likeattention to observation times, propensity scoring etc.
• Also give importance to comparing like with alike as we
try to falcify our hypothesis – control groups with other
diseases/conditions
• Proper case control studies starting from the
hypothesized outcome – i.e looking for RA among
neoplasms
CR Meier et al. JAMA, 2000
In brief
• Caveat lector !
In brief
• Caveat lector !
• “Aye, there is the rub !”
In brief
• Caveat lector !
• “Aye, there is the rub!”
• Do not go for a grand slam with 10 points
at hand !