Transcript Vladimir Anisimov

Predictive Modelling of Operational
Characteristics at the Design and
Monitoring Clinical Trial
PSI Annual Conference
11-14 May 2014, London
Dr. Vladimir Anisimov
Sr Strategic Biostatistics Director
Predictive Analytics, Innovation
[email protected]
Copyright © 2013 Quintiles
Background
Stochastic behaviour and complexity of trial operation require
developing predictive analytic techniques accounting for
• major uncertainties in input information
• stochasticity of basic processes:
enrolment, trial start-up, various events (clinical & non-clinical)
Most of trial operational characteristics are driven by patient enrolment
which is stochastic by nature.
Inefficient enrolment forecast is one of the main reasons of trial failure:
- more than 60-70% of trials fail to recruit in time
The majority of existing tools in pharma companies still use ad-hoc
simplified or deterministic models.
2
Outline
Discussion of predictive analytic techniques for
• Patient enrolment modelling
• Predicting trial/site enrolment performance and risk-based
monitoring
• Forecasting trial operational characteristics
- follow-up patients, visits, events, costs
• Event predictive modelling in event-driven trials
3
Patient enrolment
Analytic data-driven statistical methodology is developed *
Stochastic modelling technique:
•
•
•
Patients arrive at sites according to Poisson processes
Variation in enrolment rates is modelled using a gamma distribution
Delays in site initiation also be random (uniform, beta, gamma)
In site i,
ni(t) = πλi (t – vi) χ(vi ≤t ≤ bi )
λi = γ(α,β) - rate, vi – delay in site initiation, bi - closure time.
Start-up predictions are created using expected or historical data.
Interim re-forecasting uses real data and Bayesian re-estimation.
Closed-form expressions for mean and predictive bounds are
derived (no need in Monte Carlo simulation).
Methodology is world-wide accepted, validated on tens of real trials.
Basic version is implemented in some pharma companies.
* Anisimov & Fedorov, 2005, 2007; Anisimov, 2009-2011,
presented at DIA, JSM (invited sessions), ISCB, PSI,...
4
Enrolment forecaster tools
The developed techniques and software tools (in C# and R)
allow:
• Compute mean and credibility bounds for the predictive number of
patients recruited over time and for time to complete trial at any
stage of the trial
• Realize a new paradigm: Plan with confidence e.g.
“70% chance that trial will complete enrolment in time”
• Evaluate the optimal number of sites needed to complete in time
• Advise on adaptive adjustment:
o Evaluate Probability to recruit in time;
o If study is likely to go late, calculate the number of additional sites
needed to complete in time with a given confidence
5
Enrolment adjustment
Scenario: 400 pts, 70 sites, time = 365 days. Sites initiated in 5-month period,
half of sites will be closed in two months before the end of enrolment.
Initial design: to complete with 90% confidence.
Predictive area: mean and confidence bounds.
Interim analysis after 150 days: 88 pts recruited.
Enrolment is slower than predicted.
Interim adjustment:
to complete with 90% confidence:
22 new sites to add.
Adjusted prediction is going on time.
6
Site performance and risk-based monitoring
Technique/tools* for forecasting site/trial performance:
 Triggers for detecting unusual behavior
•
•

Low and High-enrolling sites,
Late-start, inactive, high number of AE, etc.
Predictive triggers (interim time, data-driven):
•
•
Predict performance
Create dynamic forecasts in future periods
Opportunity for optimal decision-making:
choose the optimal # of sites and allocation accounting for study
costs, completion time and risks per delays
Current triggers usually use assumptions of normality and detect
unusual behaviour of sites within cohort using Mean and SD:
X > Mean(cohort) + K*SD(cohort), K=1,2,3
However, many of variables describing operation of clinical trials
are rather far from the normal distribution.
Thus, using triggers based on Mean and SD will lead to biased results.
* Predictive Analytics Team, Quintiles
7
Real data analysis
Histogram of the enrolment rate
(# of patients)/(site enrolment duration)
far from normal distribution,
Red dot – 0.004: Mean + 3*SD,
heavy tailed.
Adequate model – Poisson mixed
with gamma
Histogram - Time from Last Patient Enrolled
till current time
– red line – fitted Pareto distribution
far from normal distribution, heavy tailed.
Adequate model – Exponential mixed
with gamma
8
Case study
Classification of Low-High enrolling sites using Probability Quantile trigger.
Different dots – different sites - 332. Each site is defined by (# of patients, exposure time)
Two-dimensional site’s characterization.
Probability Quantiles: 95% - red line, 90% - magenta, 80% - brown.
Adequate model - Poisson process with gamma rate.
Thresholds based on normal approximation:
Black linear lines: Mean + SD; Mean + 2*SD; Mean + 3*SD – current trigger
9
Case study: predictive probabilities
Interim analysis,
real study,
330 active sites.
Sites:
1006 and 1017
– low enrollers
Sites:
1901 and 1906
–good enrollers
Predictive probabilities to enrol:
| zero | <= one |
<= two | <= three | at least one patient |
ML estimation and Bayesian re-estimation.
In particular, if at time t0 site i recruited zero patients during time interval vi , then
Prob (zero patients at time t) =
(α,β) – estimated by ML model parameters
10
Predictive enrolment bounds
SID
57
58
59
60
61
62
63
64
66
67
68
69
70
71
72
15 days
2
7
8
2
9
2
9
9
26
8
2
7
7
2
5
30 days
2
12
13
2
14
2
14
14
45
13
2
10
10
2
8
45 days
3
15
17
3
19
3
19
19
63
18
3
14
14
3
10
Real trial,
interim look,
544 sites active;
1790 patients screened
Predictive 90%-upper bounds for the number of recruited patients in the future.
Goal: to predict supply in advance, prepare staff in hospitals,..
Site 66 is highly enrolling; sites 57, 60, 68, 71 – low enrolling
11
Dynamics of non-enrolling sites
Blue curve:
site’s start-up
green curve:
mean # of empty
sites
red curves- 90%
predictive bounds
Artificial case study: number of patients =450;
number of clinical sites =175;
time to complete enrolment =9 months;
all centres initiated uniformly in 3-month period.
At the end:
mean 55,
low bound = 45,
upper bound = 67
Proportion of empty
sites at study
completion ~ 30%
12
Dynamics of highly performing sites
Green curve:
mean # of sites
recruited > 8 pts
red curves- 90%
predictive bounds
At the end of
enrolment:
mean = 8,
low bound = 4,
upper bound = 13
Predicted number of sites recruited more than 8 pts each.
The same case study:
number of patients =450;
number of clinical sites =175;
time to complete enrolment =9 months;
all centres initiated at start up.
13
Modelling trial operation
Next stage: Predictive modelling operational processes.
A new technique using hierarchic evolving processes is developed.
Model:
In centre i recruited at time tki patient k generates some follow-up
evolving process with random life time ξki(t- tki), tki ≤ t ≤ tki +τki, e.g.:
• follow-up visits, related events
(clinical or operational, e.g. - AE, CRF, monitoring visits,...)
• associated costs (visits, maintenance, supply)
Zi(t) = Σk ξki(t – tki) - sum of evolving processes in centre i,
Statistical technique for forecasting evolving processes is developed.
Closed-form solutions are derived for many practical scenarios.
Tools in C#, R and RExcel are created *
* Predictive Analytics Team, Quintiles
14
Operation cost modelling
Monthly cost of trial operation
Input: Costs per patient visits
Visit No
1
2
3
4
5
6
7
8
9
10
Mean and 90% credibility intervals.
Input – tables:
• Site’s initiation data
• # of sites
• Costs per sites/patient visits
Time # of visits
1
600
30
450
58
400
87
350
114
304
143
297
172
290
198
283
282
276
366
269
Cost
100
50
40
40
50
40
30
40
45
40
Input: Operational costs:
• Opening site = $10,000
• Maintenance cost per site/day = $100
• Site closing cost = $1,000
• Monitoring cost per patient/day = $20
15
Patients in trial/visits
Study design: Sites – 200, patient’s target – 800, enrolment duration – 1 year
Predictive number of follow-up patients
Predictive number of Visits No. 3
Mean, Low and Upper 90% bounds
Visits time-schedule:
Region with 100 sites, μ – drop-out rate,
4 visits in total, each after 60 days
Follow-up period L=180 days.
At some conditions – stable regime: for L < t <T,
n(t) ~ nstat – Poisson with rate λ(1 – e- μL) /μ
16
Event Forecaster tool
Event-driven trials. Patients are followed-up.
Analytic technique* is developed for predicting over time the number
of events together with ongoing enrolment and time to stop trial
Assumptions:
•
•
•
Enrolment follows a Poisson-gamma model
Can be different types of events
For multiple events Markov evolving processes are used.
Predictive process:
and
- transition probabilities of Markov chain,
- global posterior rate (using Bayesian re-estimation in sites).
Predictive characteristics are derived in a closed form
(no need in Monte Carlo simulation)
Opportunity: to combine prediction of the # of events with predictive power.
* Anisimov, Pharma Stats, 2011
17
Probability of operational success
A
Scenario.
Enrolment completion target time 21 months,
Trial completion target time 70 months
Other 150 sites (with 0.4 pts/month/site) are
added after interim look
PA = 100% (enrolment complete in time)
PB = 95%
(events complete in time)
Hazard ratio – 0.8, test on non-inferiority.
Power depends on the # of events - random
PC = 97.5% - Prob( Power >0.9)
B
Trial completion target time
C
Trial completion target time
Target number of events
Acknowledgment: Val Fedorov, Xiaoqiang Xue
18
Conclusions
The advanced statistical techniques for predictive analytic
modelling of trial operation are developed:
•
•
Predict and adaptively adjust patient enrolment
Forecast trial operational characteristics
trial/site performance, visits, follow-up patients, operational costs...
•
•
Provide risk-based monitoring of the main characteristics
Forecast of different event’s
We are not aware about similar tools used by other pharma
companies and CRO.
Supporting software tools are created and in use by
Predictive Analytics Team, Innovation, Quintiles.
Tools open wide opportunities for improving the efficiency and quality
of CT prediction, additional benefits and cost savings.
19
References
1. Anisimov V., Fedorov V., Modelling, prediction and adaptive adjustment of
recruitment in multicentre trials, Statistics in Medicine, 26, 2007, 4958–
4975.
2. Anisimov V., Predictive modelling of recruitment and drug supply in
multicentre clinical trials, Proc. of JSM, Washington, August, 2009, 12481259.
3. Anisimov V., Predictive event modelling in multicentre clinical trials with
waiting time to response, Pharmaceutical Statistics, 10, iss. 6, 2011, 517522.
4. Anisimov V., Statistical modelling of clinical trials (recruitment and
randomization), Communications in Statistics – Theory and Methods, 40:
19-20, 2011, 3684-3699.
5. Anisimov V., Predictive hierarchic modelling of operational characteristics
in clinical trials, Communications in Statistics - Simulation and
Computation (submitted)
20