Transcript slides - Non-Clinical Statistics Conference

Demonstrating the
impact of statistics
in the preclinical
area using Bayesian
analysis with
informative priors
Ros Walley, John Sherington, Joe Rastrick,
Gill Watt
Non Clinical Statistics Conference
8th-10th October 2014
2
Outline
‫ן‬
‫ן‬
‫ן‬
‫ן‬
‫ן‬
‫ן‬
‫ן‬
Strategy to demonstrate impact
• Features of Bayesian designs
• Selling points
Preclinical setting
• Contrast with Clinical
• Related issues
Case study
First steps
Types of control groups
Bayesian methodology
Summary to date
3
Strategy to demonstrate impact
Features of Bayesian designs
•
Explicit way of combining information sources
•
Forces early agreement as to relevance of all information sources
•
Reduce costs and resources (animal numbers) through informative
priors/predictive distributions
•
Reduce costs and resources through interim analysis
•
Allows more relevant statements to be made at the end of the study e.g. the
probability the response rate for drug A is more than 10% better than drug B
•
Ranking compounds
•
Comparing a combination with its components
•
Flexibility in estimation. E.g. one can analyse on the log scale and estimate
differences on the linear scale
•
Allows for a wide variety of models to be fitted and can address issues such as lack
of convergence or outlier-prone data
•
Model averaging, model selection
4
Strategy to demonstrate impact
“Selling points of Bayesian methods”
High impact
Focus on in vivo studies that are run again and again
Saving even a few animals per study results in large savings, easily demonstrated
Ground-breaking
Quick search in the literature suggests little use in vivo except some focused
applications:
•
PK & PK/PD models
•
SNPS/genes – pathway analysis. Including a Nature reviews article called “The
Bayesian revolution in genetics”
•
Lookup proteins
Complements clinical strategy
*
5
Preclinical setting
Contrast with clinical
Preclinical
Clinical
Data recording
Focus on individual study 
Hard to pool
Standardized, locatable 
Easy to pool
Software
Graphpad PRISM, EXCEL
SAS, R
Relevant data for
priors
In house, Run to same protocol;
any changes noted.
Fantastic source.
Usually from publications.
Differences between protocols &
populations. Summary level data
only.
Size of experiment
For in vivo studies, typically
N=3-10 per group
Typically, N=20-100 per group for a
PoC.
Protocol finalisation A set of experiments may be
further optimised during their
running
Reluctance to trigger a protocol
amendment
Novel stats
methods
Very traditional approaches
used. Little stats support.
Bayesian and adaptive methods
implemented and published.
Data interpretation
Common to compare
compounds across experiments
Focus on individual experimental
compound
Preclinical setting
Related issues to consider
Different
output:
internally &
externally
Getting hold of the
data; in the right
format
Bayesian
designs for
repeated
studies
Allied
designs:
modelled
approach,
QC charts
Rely entirely on
historic data for
one group
Supplement a
group with prior
info  analysis
issues
7
Case study
Background
Mouse model to study inflammatory response after a challenge.
Measures a number of cytokines at 2.5 or 3hrs.
Includes test compounds + three control treatments:
•
Negative group (no challenge).
•
‘Positive’ group (challenge, no treatment).
•
Comparator with known efficacy
Main comparisons of interest:
•
Test compound vs. +ve group (untreated) – Does test compound reduce the response?
Data on 19 studies available (reasonably consistent protocol).
Most frequent in-vivo assay in this therapeutic area.
Data typically presented an experiment at a time, in PRISM
8
First steps
Strategy to smooth the introduction of Bayesian methods
QC charts to demonstrate reproducibility over time
Promoting interval estimates rather than a focus on p-values
Before committing to reducing animals, show “what would have happened if we
had done this in the last study” by removing observations from the data set
Advertising and speaking to stakeholders
Publication strategy:
1. A statistical paper
• case studies with details removed
2. For each assay, publish the meta-analysis of the historic data and the planned
future data analysis
• preferably in the selected journal to be used for data for new compounds
3. Publish data relating to specific compounds
• having de-risked by the preceding 2 publications
9
First steps
QC charts for control groups
Well received.
Introduces the idea of expt-to-expt variation.
Intuitively, the relevance of the historic controls depends on the size of the study to
study variation.
Low expt-to-expt variation
High expt-to-expt variation
Bayesian analysis can use the historic control information, down-weighting it
according to the amount of experiment-to-experiment variation
10
Types of control groups
■ Not used for formal statistical comparisons. Example uses:
• To ensure challenge is working; to establish a “window”; to check
consistency with previous studies; to convert values to %.
• Replace group with a range from a predictive distribution
■ Used for formal comparison vs. test compounds/doses
• Used as the comparison in t-tests ..etc
• Combine down-weighted historic data with the current experiment
11
Bayesian methodology
Outline
■ Analyse historic control treatment group data, excluding the last study.
• Bayesian meta-analysis
■ Analyse the last study
• Show what would have happened if we had “bought into” the Bayesian approach; omit
animals if necessary
■ Possible options for future studies:
• Omit all/some animals from all/some control groups.
• Use historic data as prior information combined with observed data in a Bayesian
analysis.
• Use historic data to give a predictive distribution for control group. i.e. don’t include that
treatment group in current study.
■ Statistical model based on meta-analytic predictive methodology in Neuenschwander et
al
12
Bayesian meta-analytic predictive approach
Statistical model based on Neuenschwander et al (2010)
■ Historic data:, h=1…H:
Observed study data:
Yh | θh, σa2~ N(θh, σa2)
Study means:
θ1… θH ~ N(μ, τ2)
■ Predictions for next study, denoted by *:
True study mean
θ* ~ N(μ, τ2)
 prior for next study
Observed study mean
Y *| θ*, σa2~ N(θ*, σa2/n*)
 predictive dn for next study
■ Priors and hyperpriors
μ has vague normal prior
Τ has vague half normal prior (sensitivity analysis carried out)
σa2 has vague gamma prior
13
Bayesian methodology (2)
“Replacing” control groups with predictive distributions
Bayesian analysis
of historic control
data
QC-chart like limits for
control group
Suitable for control groups not
used in statistical comparisons
New study data:
8 per group (for the
other groups)
Traditional analysis
of current study
Overlay Bayesian
analysis in data
presentations
14
Case study
Predictive distribution
15
Bayesian methodology (4)
Full Bayesian analysis
Bayesian analysis
of historic control
data
New study data:
8 per group
Effectively N control
animals with mean, m
Bayesian analysis
of current study
Suitable for any control groups
but requires a Bayesian
analysis for each data set.
This is a simplification of the exact analysis
Results and
conclusions
16
Case Study
Full Bayesian analysis
The Bayesian analysis gives narrower confidence intervals. It is comparable to
using 3 extra animals (11 instead of 8)
__________
___________
__________
___________
A@3mg/kg
A@30mg/kg
B@3mg/kg
B@30mg/kg
17
Summary to date
Predictive approach developed for two models
Biologists very positive; implementation being considered
Potential saving: one control group per study
Full Bayesian approach developed for one model
Modest savings in numbers of animals
Software issue; potentially lengthening turnaround times for rapid screens
Biologists suggested starting with something slightly more low-key
QC charts provided an excellent introduction to between and within study
variation
Bayesian methods can reduce required resource (animal numbers)
To have the greatest impact, focus on studies repeated very frequently
Even for well controlled in vivo experiments study-to-study variation is not
negligible
,
18
References
•
Neuenschwander, B., Capkun-Niggli, G., Branson, M. and Spiegelhalter, DJ.
Summarizing historical information on controls in clinical trials., Clin Trials
2010 7: 5
•
Di Scala L, Kerman J, Neuenschwander B. Collection, synthesis, an
interpretation of evidence: a proof-of-concept study in COPD. Statistics in
Medicine 2013; 32: 1621–1634.
•
Evans, M, Hastings, N and Peacock, B (2000). Statistical Distributions. 3rd
Edition. New York: Wiley
•
Beaumont and Rannala, The Bayesian revolution in genetics. Nature Reviews
Genetics 5, 251-261 (April 2004)
19
Questions?
Thanks!