Transcript Document

A Statistician’s Perspective on
Biomarkers in Drug Development
PSI Biomarkers Special Interest Group
Martin Jenkins and Chris Harbron, AstraZeneca
Co-authors: Aiden Flynn, Trevor Smart, Chris Harbron, Tony Sabin, Jayantha
Ratnayake, Paul Delmar, Athula Herath, Philip Jarvis and James Matcham;
Volume 10, Issue 6, Pages 494-507, November/December 2011
What is a biomarker?
Biological marker (biomarker): A characteristic that is objectively
measured and evaluated as an indicator of normal biological
processes, pathogenic processes, or pharmacologic responses to a
therapeutic intervention.
NIH biomarker definitions working group, 2001
• Many endpoints could be considered as biomarkers
• Commonly thought of in terms of biological / tissue samples (eg
blood), imaging techniques or even examinations
• Many technology platforms (proteomics, histopathology etc)
• Does not define one purpose and so a clear objective and
terminology can be helpful
• Appear at many stages throughout the development program
27/09/2012
PSI Biomarker SIG
2
Types of biomarker
27/09/2012
PSI Biomarker SIG
3
Biomarker Examples
Biomarker
Current Use
Classification
HER2, EGFR, K-RAS
mutations
Directing treatment in
oncology
Predictive biomarker Defines indication in label,
diagnostic development
UGT1A1, TMPT
polymorphisms
Predisposition to certain
toxicities
Predictive biomarker Appear in label as risk factor
suggesting dose adjustment
AB1-42
Diagnose prodromal
Alzheimer’s Disease
Prognostic biomarker Enrich trial populations. Example
of qualification
Gene signature
chips
Prognosis prediction in
oncology
Prognostic biomarker Diagnostic qualification process
applies
CRP, IL-6, TNFa in
blood samples
Proof of principle in
inflammatory diseases
Pharmacodynamic
biomarker
Fit for purpose assay validation
FDG-PET imaging
Proof of concept (eg
Tumour metabolism)
Pharmacodynamic
biomarker
Collaborative opportunities for
qualification
HbA1c
Represents glycemic
control in diabetics
Surrogate Endpoint
Primary evidence for label.
Qualification burden
27/09/2012
PSI Biomarker SIG
Qualification
4
What characteristics are we interested in?
• It is generally advisable to learn as much as possible about a
biomarker prior to it’s application for decision-making
– The science behind the choice of biomarker should be supported prior
to initiating studies
– The practical feasibility of using the marker should be examined
– The statistical properties of the biomarker are of interest
• Statistical properties of interest include
– Variability estimates (and components of variability, within and
between subjects)
– Effect sizes (using a positive control, challenge model or other data
sources) and dynamic range
– Distributions and scaling approaches required
– Appropriate analysis methods considered
27/09/2012
PSI Biomarker SIG
5
Methodology studies
• Feasibility / Methodology studies can be used to learn the
characteristics of an endpoint, but also the practicalities
– Feasibility of multiple assessments
– Acceptability to the patient, degree of dropout
• The same design and analysis methodology should be used as is
envisaged for the subsequent clinical trial
• Information learned can aid in designing or sizing this trial
• Work carried out should be fit-for-purpose enough to ensure that
there is sufficient confidence that decisions can be made based
upon the biomarker, given it’s intended use. Further studies,
reviews or meta-analyses can aid in this if required.
27/09/2012
PSI Biomarker SIG
6
Sources of variation
Much biomarker work is accompanied by similar practical
challenges around controlling sources of variation:
•Patient-related factors
• Individual biological factors: genetics, race, age, comorbidities
• Social and habitual factors: smoking, physical fitness, diet
• Natural biological variation: eg diurnal effects
•External factors:
PCA Scores Plot
Coloured According to Analysis Batch
5
4
3
2
Principal Component 2
• Pre-analytical variation: sample collection
and fixation practices
• Analytical variation: between laboratories,
readers and batches, technical precision of
the assay
1
0
-1
-2
-3
-4
-5
-6
-8
-7
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10 11
Principal Component 1
Ellipse:
Hotelling
T2 (0.95)
SIMCA-P+
12.0.1 - 2010-04-21
16:23:07 (UTC+0)
27/09/2012
PSI Biomarker SIG
7
Sources of bias
Bias can arise, especially in open-label studies, for several reasons:
• Verification bias because of the choice of locally available methods
• Patient selection bias, for example, patients with breast cancer family
history, may not consent to BRCA DNA testing
• Treatment allocation bias, if treatment assignment is based on a
subjective assessment, as in some histo-pathological endpoints
• Being asked the same question repeatedly, for example in a pain
challenge model, could induce false differences
• If a scoring method is subjective, then measures should be taken to
minimize bias, for example, with the use of scripted questions, blinding
or standard operating procedures.
• Conclusions may be subject to many caveats if they are not based upon
complete and representative data-sets
27/09/2012
PSI Biomarker SIG
8
Sources of missing data
Biomarker studies can suffer from missing data issues, especially
in exploratory situations:
•
•
•
•
•
•
Inaccessibility of tissues (e.g. lung cancer)
Low consent rates for optional samples
Lack of residual tissues in complete responders
Poor quality of fixation in archival samples
Patient drop-outs (lack of response or toxicity)
Poor data handling procedures cf. clinical data
Likely sources of missing data should be considered in advance
so as to minimise potential implications and make appropriate
assumptions
27/09/2012
PSI Biomarker SIG
9
Multiplicity when considering many markers
• Range of new high-dimensional technologies
– e.g. genetics, genomics, proteomics, metabonomics, NGS
• Allow understanding in detail at a molecular level the processes of
disease and response to treatment and identify biomarkers that can
identify or predict these changes.
• Multiplicity becomes a concern and requires new approaches to be
adopted
– e.g. False Discovery Rate, Permutation testing, Significance Analysis of
Microarrays (SAM)
• Pre-analysis filtering of variables can help
• Want to emerge with both statistical significance
scientifically relevant and meaningful effect.
&
– 2D FDR
27/09/2012
PSI Biomarker SIG
10
Composites and Multivariate Analyses
• Visualisation key
– Principal Components Analysis (PCA), Clustering
• Wide variety of supervised multivariate predictive modeling
techniques are available
–
–
–
–
Regression-based approaches e.g. Partial least Squares (PLS), Elastic Nets
Proximity-based methods e.g. Nearest Neighbours
Tree-based methods e.g. Random Forests, Gradient Boosting
Distance-based approaches e.g. Support Vector Machines (SVM)
• Not just “Black-Box”, Interpretation important
– Importance scores
• Study design is critical, as is visualization, understanding data
quality and its impact on subsequent analyses.
– MAQC-II
27/09/2012
PSI Biomarker SIG
11
Biomarkers for Personalised Healthcare
• Personalized health care offers the potential to identify patients
more likely to derive benefit from treatment and as such is of great
interest to Pharmaceutical companies, Regulatory Authorities and
Health care Providers
• Predictive markers – marker by treatment interaction
• Gives rise to new set of challenges : eg. Low power to robustly
identify biomarkers in exploratory studies
• Developing from a Biomarker to a Companion Diagnostic
• Understanding and quantifying the factors that may impact the
performance of the diagnostic
• Identifying an optimal cut-off
• Additional regulatory process
27/09/2012
PSI Biomarker SIG
12
Study design options
• All-comers design
Biomarker +
– Subjects enrolled into groups, and
retrospectively measured
• Stratification design
Control
Biomarker Randomise
– Biomarker status at screening used
as a stratification factor in
randomisation to ensure balance
Biomarker +
Experimental
Biomarker -
• Targeted design
– Only biomarker positive patients recruited into study
• Enriched design
– Hybrid, recruiting a limited number of biomarker negative patients
with a greater representation of biomarker positives
• Adaptive design
– Many options allowing testing and refinement of a biomarker
27/09/2012
PSI Biomarker SIG
13
Safety biomarkers
preclinical & translation
• Predictive safety biomarkers allow early detection of toxicity and
assessment of human risk
• Preclinical qualification including mechanistic understanding of the
relationship between the biomarker and organ damage
• Use known organ toxicants
• Determination of organ toxicity in rat by qualified toxicological
pathologist
• Understanding properties of translation to man
• Animal Model Framework (AMF) project is a collaborative effort
combining pre-clinical and clinical data to determine operating
characteristics and refine thresholds of Safety Pharmacology
models
27/09/2012
PSI Biomarker SIG
14
Safety biomarkers - clinical
• Clinical qualification less straightforward because generally
verification of organ toxicity not possible
• Utilise methods that can assess performance in absence of gold
standard including Bayesian approaches
• Currently lab parameters used to indicate kidney and liver injury,
but poor performance
– e.g. by time serum creatinine indicates drug induced kidney injury, a
high degree of kidney function loss has already occurred
• Public-private precompetitive partnerships established for
searching for, validating and qualifying safety biomarkers for
predicting drug-induced organ injury
– Predictive Safety Testing Consortium (C-Path), SAFE-T (IMI)
27/09/2012
PSI Biomarker SIG
15
Biomarker qualification
• Formal biomarker qualification process have been introduced:
– FDA (qualification process for drug development tools)
– EMA (qualification of novel methodologies and biomarkers)
– Examples on EMA and FDA websites
• These work alongside usual scientific advice routes and are a way to
seek regulatory opinion on acceptability of a marker for a given use
• This is not mandatory and so is usually not required, but may be an
advantageous step, particularly for a marker with wide applicability
(For example for collaborative groups seeking to develop a new
endpoint, particularly for toxicity)
• Diagnostic biomarkers would need to follow the in-vitro diagnostic
regulatory processes
27/09/2012
PSI Biomarker SIG
16
Surrogate endpoints
• Those outcomes for which treatment effects correlate well with
those for an accepted clinical outcome at an individual and group
level could potentially substitute for a recognized clinical endpoint
• Should lead to the same decision being made
as if the clinical outcome had been used
• Examples exist (eg in diabetes or HIV), but
a large body of evidence is required and this
is not required for many purposes
• Surrogacy is often an unrealistic target.
• Statistical methods have been developed to examine the association
between endpoints, but these can also be applied to other
situations, for example when considering assurance when endpoints
differ between studies
27/09/2012
PSI Biomarker SIG
17
Future challenges
Issues to address in the future
i
How can the traditional clinical development program be reengineered, so allow for
greater learning around a biomarker without delaying the drug program?
Ii
What study designs can be used to translate markers of drug toxicity into man given
that the pre-clinical studies used for validation are not feasible to conduct in humans?
iii
Many technologies suffer from a ‘batch effect’. What pre-processing techniques can be
developed to ensure we are measuring true biology rather than sample quality?
iv
New techniques have moved the degree of dimensionality by several orders of
magnitude. How, both practically and analytically, can we cope with this deluge of
data?
v
Effective ways, possibly Bayesian, could be developed to build existing pathway
knowledge into data analyses.
vi
When identifying biomarkers for PHC, interaction analyses typically have low power,
especially with many potential markers. How can such markers be reliably identified?
vii
As PHC becomes the processes for gaining approval of a drug, diagnostic and
biomarker qualification may become more burdensome. How can this evidence be
streamlined?
27/09/2012
PSI Biomarker SIG
18
Conclusions & Recommendations
Key considerations for statisticians in biomarker development
i
Define clear objectives shaped by the potential future use of the marker
- This will clarify what can reasonably be claimed based upon the research
- Validation work and study designs should be “fit-for-purpose” relative to these aims
ii
Prospectively plan studies even if they are exploratory
- Defining what represents “success” in advance will aid objective decision-making
iii
Consult expert colleagues to understand and shape the endpoints definitions
- Learn about distributions and pre-processing
- Influence scoring methods so as to get maximum information
iv
Gather prior knowledge on sources of variability and causes of missing data
- Plan ahead to avoid sample and data management issues rather than being reactive
- Adjust missing data assumptions accordingly
v
Learn about a new marker using methodology studies or translational science
- Estimates of variability or effect size can be valuable if reflective of situation of use
27/09/2012
PSI Biomarker SIG
19
Conclusions & Recommendations
Key considerations for statisticians in biomarker development
vi
Analyse a biomarker using the appropriate methodology for the given endpoint type
- As with other endpoints there is no universal approach
vii
Be alert when using large numbers of candidate biomarkers
- Particular attention should be paid to issues of multiplicity and model validation
viii
Maintain a sceptical and questioning point of view to manage expectations
- Consider results in relation to prior knowledge and study aims, avoiding extrapolation
ix
Consider how to add confidence to findings using other sources of information
- Integrate biological information as well as numerical results
x
Remember that there is nothing mysterious about biomarkers
- A consideration of key statistical principles can be instructive in most situations
27/09/2012
PSI Biomarker SIG
20
PSI Biomarker Special Interest Group
Please get in touch if you would like to get involved!
• Join the mailing list or linked-in group in
• Help on the SIG committee
• Review papers, training, discussion groups…
• Attend one of our free meetings /or our conference sessions
Biomarker Validation - Case Studies and Approaches
MedImmune, Cambridge, 10 October 2012
• Go to www.psiweb.org and click on “committees and sigs” in
the menu to find our webpage with more information
27/09/2012
PSI Biomarker SIG
21