Transcript Slides

LEVELS OF EVIDENCE FROM
DIABETES REGISTRIES
Registry-based Epidemiology?
John M. Lachin
Professor of Biostatistics, Epidemiology and
Statistics
The Biostatistics Center
The George Washington University
EuBIRO-D vs. USA
Ciao Fabrizio e Massimo
No regional or national healthcare program
No national or regional registries
HMO network
Translating Research into Action for Diabetes
Comparative Effectiveness Research
Agency for Healthcare Quality and Research:
patient satisfaction, quality of life
National Institutes of Health: Clinical outcomes
GRADE study
Science and Uncertainty
Jacob Bronowsky:
All information is imperfect. We have to
treat it with humility... Errors are
inextricably bound up with the nature of
human knowledge…
The degree of uncertainty is controlled
through the application of the scientific
method,
and is quantified through statistics.
Statistical Test of an Hypothesis
Null Hypothesis (H0):
The hypothesis to be disproven
The hypothesis of no difference.
Alternative Hypothesis (H1):
The hypothesis to be proven
The hypothesis that a difference exists.
Two types of errors:
Type I: False positive, probability 
Type II: False negative, probability 
Power = 1 - 
Factors that Affect  and Power
Selection and Observational/Experimental
Bias
Poor study design or execution
Missing data
Reproducibility (precision) of assessments
Missing Data
The Fundamental Issue - BIAS
Numerators and denominators may be
biased
Estimates of population parameters,
differences between treatments or
exposure groups may be biased.
Statistical analyses, p–values and
confidence limits may be biased.
p = 0.05 may mean a false positive error rate
() much greater than 0.05;
N=800, 20% missing in treated/exposed,
true  ≈ 0.40.
Can’t Statistics Handle This?
Not definitively.
The magnitude of the bias can not be
estimated, no correction possible.
Analyses can be conducted under certain
assumptions.
But there is no way to prove that the
assumptions apply.
Best way to deal with missing data is to
prevent it.
Sample Size Adjustments
Can adjust sample size to allow for lossesto-follow-up and missing data, e.g.
increase N by 10% if expect 10% losses
BUT, this adjusts only for the loss of
information,
NOT for any bias introduced by missing
data.
Precision or Reliability of Measures
Reliability coefficient  = proportion of
total variation between subjects due
to variation in the true values.
1 -  = proportion of variation due to
random errors of collection,
processing and measurement.
Impact of Reliability
Power
Power decreases as  decreases.
Reliability ()
Impact of Reliability
If N is the sample size needed for a
precise measure then N/ is needed
for an imprecise measure.

1.0
0.9
0.8
0.7
0.6
0.5
1/
1.0
1.11
1.25
1.43
1.67
2.0
Impact of Reliability
Maximum possible correlation between Y
and X is a function of the respective
reliabilities: Max(R2) = x y
x
y
Max(R2)
1.0
0.9
0.90
0.9
0.9
0.81
0.9
0.7
0.63
0.9
0.5
0.45
0.7
0.7
0.49
0.7
0.5
0.35
Impact of Misclassifications
m = fraction of treatment or exposure
misclassifications, or fraction of
outcomes misclassified
N/(1-2m)2 is needed
m
1/(1-2m)2
0
1.0
0.1
0.8
0.7
0.6
0.5
1.56 2.78 6.25 25.0
∞
Randomized Clinical Trial
Randomization:
• Subjects assigned to each treatment
independently of patient characteristics
• No selection bias. Treatment groups
expected to be similar for all variables
measured and unmeasured.
• No confounding of the experimental
treatment with other uncontrolled factors
• May infer a cause – effect relationship
between treatment and the outcome,
provided the trial is of good quality.
Randomized Clinical Trial
Precisely defined population
Precisely defined exposure (the treatments)
Precisely defined outcome measure
Results clearly interpretable
Observational Study
Many types, e.g. case-control study
Prospective cohort study
No randomized controls
Maybe a precisely defined population
Maybe a precisely defined exposure (the
treatments)
Maybe a precisely defined outcome
measure
Observational Study
Many potential biases
Selection bias – composition of groups
Confounding with other factors
Statistical adjustments substituted for
randomization
Observational Study
Necessary in settings where a randomized
study is impossible
Smoking and lung cancer
Generally describe an association between
the exposure factor and an outcome that
may not represent a causal relationship.
Difficult to establish causality, though
possible with replication of a highly specific
association.
Observational Evidence
The essential issues with observational
evidence is the degree to which an
observed relationship can or can not be
explained by
• other variables,
• other mechanisms, or
• biases
– even after statistical adjustment
Confounding
When the study factor (groups) are
associated with another (confounding)
factor that is a direct cause of the
outcome.
Coffee consumption and cancer.
Coffee consumption confounded with
smoking.
Higher fraction of smokers among coffee
drinkers.
Statistical Adjustment for
Confounding
Regression or stratification model including
the study factor and the possible
confounding factor(s)
Assumes that the operating confounding
factors have been identified and
measured.
Assumes that the regression model
specifications are correct.
Statistical Adjustment for
Confounding
Estimates the association of the factor with
the outcome IF the confounding factor
were equally distributed among the
groups.
Difference in cancer risk between coffee
drinkers and non-drinkers IF the fraction
of smokers was the same among drinkers
and non-drinkers.
Coffee drinking and smoking are alterable.
Thus, the results would have a population
interpretation.
Statistical Adjustments
NOT all covariate imbalances introduce
bias, in which case adjustment itself
introduces bias.
Gender inherently confounded with body
weight
Gender adjusted for body weight estimates
the gender difference if males and females
had the same weight distribution.
Statistical Adjustments
Adjustment for weight provides a biased
estimate of the overall male:female
difference in risk in the population
But weight-adjusted estimate describes the
additional male:female difference in risk, if
any, that is associated with gender
differences other than weight
Of mechanistic interest.
Omitted Covariates
Observational study can only adjust for
what has been measured.
Adjustment for observed factors can not
eliminate bias due to imbalances in
unmeasured covariates.
Inappropriate Covariates
Analysis should follow the prospective
history of covariates
Statistically invalid to define a covariate
over a period of exposure that goes
beyond the observation of an event.
Example, mean HbA1c over 5 years as a
predictor of outcomes observed during
the 5 years.
Rather, use the mean HbA1c up to the time
of each successive event.
Confounding by Indication
In some cases, however, exposure to a
factor (e.g. drug) may be confounded with
the indications leading to the exposure.
Example: statins indicated in the presence
of hyperlipidemia.
Recent data suggests that statin use may
also increase risk of T2D in IFG/IGT.
But is the increased risk due to the statin
use or the prior history of hyperlipidemia?
Confounding by Indication
In other cases an adjusting factor (e.g.
dose) may likewise be confounded with an
indication.
Example: Hemkens et al. analysis of the
association of insulin glargine vs. human
insulin with cancer in a German claims
database.
14% decrease in age, gender adjusted risk.
But substantial dose imbalance.
14% increase in risk when also adjusted for
dose.
Reasons for Dose Imbalance
Confounding by indication, or allocation
bias.
High or low glargine (or human insulin)
dose may be determined by unmeasured
patient factors that are differentially
distributed within groups.
e.g. high glargine dose only administered to
severely ill patients.
Impossible to statistically adjust for such
confounding
Adjusted analysis results are biased.
Registries
Many types:
100% population captured, e.g. public
health care system
Non-random subsample, e.g. insurance
provider or hospital based
In latter case, registry population may not
represent the full population of interest
Inherently prospective
But no standardized follow-up schedule
Registries
Relies on data capture in conjunction with
the administration of medical care
No specific exposure of interest when
established, in epidemiological sense
No specific outcome measure of interest.
Rather medical status and treatment
recorded (possible exposures) and other
major morbidities and mortality recorded
(possible outcomes).
Registries
Epidemiologic analyses may be attempted.
But, difficult to precisely define exposure to
a factor:
When is a subject
First at risk of being exposed (e.g. when
is a drug introduced to the market?)
Actually first exposed (e.g. starts drug)
Removed from exposure (e.g. off drug)
Confounding by indication often an issue
Registries
Coding, classification of events may not be
standardized
Often no adjudication
May be difficult to determine whether or
exactly when an outcome event occurred,
e.g. macroalbuminuria is “intervalcensored”
May be difficult to determine when subject
no longer at risk (right censored)
Incidence may be difficult to assess reliably.
Registries - Uses
Prevalence
Distribution of patient status or
conditions in the population
Cross-sectional associations
If “representative” but not proportionally,
weighted analyses can provide
estimates in the broader population.
Disadvantaged populations (poverty,
uninsured) may not be represented
Registries - Epidemiology
Exposure to a factor and outcomes
Open to many biases.
Statistical adjustments may be
inadequate.
But, a registry can be the foundation for
first-rate epidemiologic studies.
Registries - Epidemiology
Nested case-control studies
Sub-sample of possible cases that is
carefully adjudicated
Sub-sample of possible controls
(matched by follow-up time) also
verified.
Exposure (risk) and confounding factors
also verified.
Registries - Epidemiology
Prospective cohort studies
Identify eligible subjects -- representative
of the registry (general) population
Formally enroll subjects (consent) with a
systematic follow-up schedule
Careful characterization of exposure
(risk) and confounding factors
Specific outcome reporting
(assessments) with adjudication.
Registries - Epidemiology
Embedded cohort study
Identify eligible subjects
Enroll subjects (consent)
Establish a schedule of assessments to
be conducted as part of routine care
Send notices to patients when visits due
Capture exposure (risk) and confounding
factors
Identify possible outcomes through
medical reports, with subsequent
adjudication.
Registries - Epidemiology
A hybrid
Establish an embedded cohort study.
Also implement a formal prospective
study in a sub-sample.
The latter can serve as a quality check on
the former.
Registries - Epidemiology
LARGE Sample Size
N needed to detect a rare outcome (e.g.
fulminant hepatotoxicity, or angioedema)
If risk is 1 in 10,000, need N = 29,956 to be
95% confident that at least one case will
be observed.
If wished to have 85% power to detect a 50%
increased risk, at least 75 events
required.
N = 836,000 followed for 1 year!!
Conclusions
Registry can provide superior descriptions
of quality of care and distribution of
factors in broad population of interest.
Not as rigorous as a formal prospective
epidemiologic study, but can form the
basis for such studies.
Affords opportunities for large sample sizes
needed to detect rare outcomes.