Transcript PhRMA Biostatistics & Data Management Career Achievement Award

Challenges and Opportunities
for 21st Century
Pharmaceutical Statisticians
Christy Chuang-Stein, PhD
Statistical Research and Consulting Center
Pfizer Inc
1
delete these guides from slide master before printing or giving to the client
Outline
 Evolving role of statisticians in pharmaceutical
industry
 The external environment
 Responses to the changes in the environment
 Develop partnerships and collaborations
 Examples of collaborations
 A new metric for determining sample size for a
confirmatory trial
 Summary
delete these guides from slide master before printing or giving to the client
2
delete these guides from slide master before printing or giving to the client
Evolving Role of Stat in Pharma Industry
Little use of Statistics ==>
“Required” use of Clin Statistics ==>
Tactical use of Statistics ==>
Strategic use of Statistics & “Statistical
Thinking”
1955
2009 and beyond
delete these guides from slide master before printing or giving to the client
3
delete these guides from slide master before printing or giving to the client
Industry Perspective: “Then”
 Hired some statisticians to get things through the
regulatory agency (mostly in the US)
 “Number crunchers” to get analyses done
 “Blessed” clinical trial designs with minimal
intellectual participation except sample size
 Clinical and manufacturing focus
 Very little input outside of “necessary”, low
involvement in non-clinical areas
 Statisticians played a secondary role
delete these guides from slide master before printing or giving to the client
4
delete these guides from slide master before printing or giving to the client
Statistician’s Role: Now and the Future
 Full and equal partner with basic, clinical &
regulatory scientists.
 Focus on experimental design and development
strategy.
 Application of statistical thinking throughout the life
cycle of a pharmaceutical product.
 Parallel development in other disciplines such as
epidemiology, genomics, biomarker development,
and risk management expands statistician’s
contributions.
delete these guides from slide master before printing or giving to the client
5
delete these guides from slide master before printing or giving to the client
Statisticians in Pharma Industry
 Being technically smart is not enough:
 Understand the broad clinical, regulatory and publichealth context
 Communicate statistical strengths and weaknesses
 Proactive, not passive:





Design: Options available, decision analysis
Execution: Quality control and risk mitigation
Analysis: Planned and unplanned, strengths/weaknesses
Interpretation: Pre-planned or data-driven
Presentations/Publications: Keeping audience in mind.
Statistics is a Collaborative Science!!
delete these guides from slide master before printing or giving to the client
6
delete these guides from slide master before printing or giving to the client
The External Environment
 Desire for more transparency in clinical research
and clinical reporting
 A perceived need for more public “control” over the
search for valuable medicines
 Loss of public confidence in the clinical research
process and the pharmaceutical industry
 Loss of public confidence in the regulators
 An increasing demand for product safety
delete these guides from slide master before printing or giving to the client
7
delete these guides from slide master before printing or giving to the client
Our External Stakeholders
 Health care providers
 Patient care givers
 Regulators
 Journal editors
 Shareholders
 Health care payer
 And most importantly - THE PATIENTS
delete these guides from slide master before printing or giving to the client
8
delete these guides from slide master before printing or giving to the client
Data Disclosure
 Desire for more “transparency” in the clinical
research process
 Protocols
 Results
 Logistics
 Registries of trials (www.clinicaltrials.gov)
 Results from trials involving marketed products are
posted (www.clinicalstudyresults.org). Some
companies have extended this to all trials and
maintained their own trial results websites.
delete these guides from slide master before printing or giving to the client
9
delete these guides from slide master before printing or giving to the client
How Can Statisticians Help?
 Leverage the drive for
transparency.
 Develop partnerships with
academia and government.
 Within the limits allowed by
internal rules and external
regulations, share our
processes and approaches
to the outside world.
delete these guides from slide master before printing or giving to the client
10
delete these guides from slide master before printing or giving to the client
Leverage the Drive for Transparency
 Improve study designs
 All studies will be public and designs will be subject to
public scrutiny.
 Improve communication
 Provide informative displays and presentations of
results that are customized for the audience.
 Endorse the use of common standards
 Maximize the efficiency of collecting and sharing data
via the use of common standards.
delete these guides from slide master before printing or giving to the client
11
delete these guides from slide master before printing or giving to the client
Develop Partnerships
 Foster partnerships among academia, industry and
government.
 Reasons:
 Solve complex problems by sharing resources and
knowledge
 Integrate ideas and increase credibility through peer
review
 Create solutions that take into consideration differing
perspectives
 Belief:
 Working together is more effective than working in
isolation
delete these guides from slide master before printing or giving to the client
12
delete these guides from slide master before printing or giving to the client
Examples of Collaborations
 Adaptive Designs
 Multiple Co-primary Endpoints
 Active Control Study
 QT Effect of a Pharmaceutical Product
 Methods to Handle Missing Data
 Dichotomizing Continuous Endpoint
 Biomarker Quantification
 Predictive Model (Pre-clinical to Clinical)
 Safety Data Evaluation
 Multi-regional Clinical Trials
delete these guides from slide master before printing or giving to the client
13
delete these guides from slide master before printing or giving to the client
Adaptive Designs
PhRMA Definition
 uses accumulating data
to decide on how to
modify aspects of the
study
 without undermining the
validity and integrity of
the trial
ADWG: Adaptive Design
Working Group
Validity means
 providing correct statistical
inference (such as adjusted
p-values, estimates and confidence
intervals)
 minimizing operational bias
Integrity means
 preplanning, as much as possible,
based on intended adaptations
 maintaining confidentiality of data
delete these guides from slide master before printing or giving to the client
14
delete these guides from slide master before printing or giving to the client
Drug Development Process
Level of uncertainty
Exploratory phase
Population
End points
Dose range
Dosing frequency
Exposure
Drug formulation
Confirmatory phase
Sample size
Add treatment
Drop treatment
Randomization fraction
Population
Sample size
Modify delta
Randomization fraction
Drop treatment
Population
Development process
delete these guides from slide master before printing or giving to the client
15
delete these guides from slide master before printing or giving to the client
Adaptive Design Working Group
 The Group was formed in 2005 of industry and academic
members to investigate and facilitate opportunities for wider
acceptance and usage of adaptive designs and related
methodologies.
 The Group met face-to-face about twice a year, has
published numerous papers & given multiple presentations.
 It has met with regulators in US, Canada, Europe and Japan
to share experience and discuss issues of concern.
 It continues to hold monthly telecons and sponsor a monthly
Key Opinion Leader lecture series.
 Is planning a workshop with the US Food and Drug
Administration (FDA) to discuss a forthcoming FDA
guidance on adaptive designs.
delete these guides from slide master before printing or giving to the client
16
delete these guides from slide master before printing or giving to the client
10 Workstreams of ADWG
 Education Effort
 ICH harmonization
 Data monitoring issues and processes
 Good adaptive practices
 Case studies to share experience
 Software user requirements
 Material manufacturing and supply
 Communications
 Key Opinion Leader lecture series
 Documentation
delete these guides from slide master before printing or giving to the client
17
delete these guides from slide master before printing or giving to the client
Multiple Co-Primary Endpoints

There are many disorders where
regulators require a new treatment to
demonstrate statistically significant benefit
on multiple endpoints before accepting
the efficacy of the new treatment.
delete these guides from slide master before printing or giving to the client
18
delete these guides from slide master before printing or giving to the client
Definition of “Co-Primary” Endpoints

These endpoints describe the disease in the sense
that an experimental drug that does not show
superiority over placebo on all these endpoints is
not a viable treatment for the disease.

This could arise because clinicians cannot agree
on which endpoint (among several endpoints) is
the most relevant one. So, they want evidence of
effect on all of them.

In other words, they want statisticians solve the
problem for them!
delete these guides from slide master before printing or giving to the client
19
delete these guides from slide master before printing or giving to the client
How Prevalent Is the Problem?
Migraine (4)
Alzheimer’s (2)
Acute Pain (3)
Acne (3)
Fibromyalgia (3)
Male Pattern Baldness (2)
Low Back Pain (3)
Vaginal Atrophy (4)
Erectile Dysfunction (3)
Organ Transplantation (2)
Osteoarthritis (2)
BPH (2)
Menopausal Symptoms (4)
Primary Biliary Cirrhosis (4)
Fracture Healing (2)
Multiple Sclerosis (2)
Offen et al. Drug Information Journal (2007) 41(1):31-46.
delete these guides from slide master before printing or giving to the client
20
delete these guides from slide master before printing or giving to the client
Two Examples


Alzheimer’s Disease

Alzheimer’s Disease Assessment Scale - Cognitive
(ADAS-Cog)

Clinician Interview-Based Impression of Change
(CIBIC)
Vaginal Atrophy
 Self-reported frequency of the moderate or severe
symptoms that were most bothersome
 Vaginal pH
 % of vaginal parabasal cells from the maturation index
 % of vaginal superficial cells from the maturation index
delete these guides from slide master before printing or giving to the client
21
delete these guides from slide master before printing or giving to the client
The Hypotheses

Assume (D1, …, DK) represents the mean effect of a
new treatment over a placebo on K co-primary
endpoints, positive Di implying positive treatment
effect.

The null (H0) and alternative (H1) hypotheses are
H0: D1  0 or D2  0 or … or DK  0
H1: D1 > 0 and D2 > 0 … and DK > 0

Usual requirement: false positive rate  2.5% (onesided).
delete these guides from slide master before printing or giving to the client
22
delete these guides from slide master before printing or giving to the client
Current Regulatory Practice

Test H0i vs H1i (the effect on the ith endpoint), each
at the one-sided 2.5% level, i = 1, …, K.
H0i: Di  0
H1i: Di > 0

Declare efficacy only if all H0i (i = 1, …, K) are
rejected at the one-sided 2.5% level. This is the
intersection-union test (IUT).

IUT is necessary to control the false positive rate at
the one-sided 2.5% level over the entire null space.
delete these guides from slide master before printing or giving to the client
23
delete these guides from slide master before printing or giving to the client
Overall Statistical Power
Number of co-primary endpoints
Correlation
2
3
4
9
0
64%
51%
41%
13%
0.2
66%
55%
47%
24%
0.5
69%
61%
56%
40%
0.8
73%
69%
66%
57%
*Power for each endpoint is 80%, one-sided a=0.025
delete these guides from slide master before printing or giving to the client
24
delete these guides from slide master before printing or giving to the client
Correlation Coefficient (AD)

Alzheimer’s Disease

Alzheimer’s Disease Assessment Scale Cognitive (ADAS-Cog)

Clinician Interview-Based Impression of Change
(CIBIC)

Correlation coefficient between ADAS-Cog and
CIBIC is 0.22.
delete these guides from slide master before printing or giving to the client
25
delete these guides from slide master before printing or giving to the client
Correlation Coefficient (VA)
Vaginal pH
% Parabasal % Superficial
Cells
Cells
Freq of
Symptoms
0.20
0.11
-0.06
Vaginal pH
__
0.32
-0.13
% Parabasal
Cells
0.32
__
-0.26
% Superficial
Cells
-0.13
-0.26
__
delete these guides from slide master before printing or giving to the client
26
delete these guides from slide master before printing or giving to the client
% Increase on Sample Size
Number of co-primary endpoints
Correlation
2
3
4
9
0
31
49
62
96
0.2
29
46
58
91
0.5
25
39
49
74
0.8
17
27
32
48
 Assume same effect size on each endpoint, tested at one-sided
2.5% level. The objective is to have an 80% overall power.
delete these guides from slide master before printing or giving to the client
27
delete these guides from slide master before printing or giving to the client
Possible Solutions

Optimal solution is a medical one

Reduce to a single dimension
–
–

Choose a single primary endpoint
Create a composite endpoint such as ACR20 for
rheumatoid arthritis
Prioritize endpoints, requiring only a positive “trend”
in some endpoints. A positive trend could be, e.g. a
two-sided P-value < 0.15.
 Propose statistical approaches that could provide
rationales for adopting a higher significance level for
testing each endpoint.
delete these guides from slide master before printing or giving to the client
28
delete these guides from slide master before printing or giving to the client
ACR20 (-50, -70) Definition

At least 20% (50%, 70%) improvement in



# of swollen joint count, AND
# of tender joint count, AND
At least three of the following
– Patient’s global assessment of disease activity
– Physician’s global assessment of disease activity
– Patient’s assessment of pain (VAS, Likert, NRS)
– Acute-phase reactants (ESR, CRP)
– Patient’s assessment of disability (HAQ)
 Often the primary endpoint to evaluate RA treatments.
delete these guides from slide master before printing or giving to the client
29
delete these guides from slide master before printing or giving to the client
Adopting a Higher Sig Level (1-sided)
Chuang-Stein et al (2007), Stat in Med, 26:1181-1192.
Correlation
2
3
4
0
0.036
0.055
0.082
0.2
0.036
0.052
0.075
0.4
0.035
0.048
0.066
0.6
0.032
0.043
0.055
0.8
0.030
0.037
0.044
delete these guides from slide master before printing or giving to the client
30
delete these guides from slide master before printing or giving to the client
Impact on Sample Size (K = 2)
 The entry in black (red) corresponds to % increase in
sample size under the new (current) approach.
Correlation
80% Power
90% Power
0
18%
31%
12%
23%
0.2
16%
29%
11%
22%
0.4
14%
26%
10%
20%
0.6
14%
22%
11%
18%
0.8
11%
16%
8%
13%
delete these guides from slide master before printing or giving to the client
31
delete these guides from slide master before printing or giving to the client
Impact on Sample Size (K = 3)
 The entry in black (red) corresponds to % increase in
sample size under the new (current) approach.
Correlation
80% Power
90% Power
0
19%
49%
11%
36%
0.2
18%
46%
11%
34%
0.4
17%
41%
11%
31%
0.6
15%
35%
11%
27%
0.8
12%
25%
9%
21%
delete these guides from slide master before printing or giving to the client
32
delete these guides from slide master before printing or giving to the client
A More Serious Issue
 A common question – Why can’t a sponsor just
increase the sample size to get an adequate overall
power?
 Statistical power gives the probability of concluding
a treatment effect when the true treatment effect is
the amount used to size the study.
 For a trial to be successful, the new treatment
needs to deliver the above assumed effect on all
endpoints.
 A greater demand of a new treatment!
delete these guides from slide master before printing or giving to the client
33
delete these guides from slide master before printing or giving to the client
A New Sample Size Paradigm
 Relevant to trials expected to provide confirmatory
evidence for product registration.
 The primary objective of a confirmatory trial is to
confirm that the effect of the new treatment is
statistically better than a placebo. Recall that
statistical power gives the probability of concluding
a treatment effect when the true treatment effect is
equal to the effect used to size the study.
 But, we don’t know for sure if the treatment could
deliver on the assumed effect.
delete these guides from slide master before printing or giving to the client
34
delete these guides from slide master before printing or giving to the client
An Example
 A small PoC study, n = 25 per group, estimated
treatment effect over a placebo = 2.5 units, estimated
standard deviation = 7.14 units. So, estimated effect
size = 0.35 (2.5/7.14).
 The temptation is to design the next study to detect an
effect size of 0.35 based on a two-sided test at the 5%
significance level.
 Need 128 subjects per group for 80% power, or 172
per group for 90% power.
Source: Chuang-Stein (2006), Pharmaceutical Statistics,
5(4):305-309.
delete these guides from slide master before printing or giving to the client
35
delete these guides from slide master before printing or giving to the client
Question
 What is the probability that we will have a
successful trial? Define a successful trial as
obtaining a P-value less than 5%. Other
criteria can be used.
 Is it 80% if we enroll 128 subjects per group,
or 90% if we enroll 172 subjects per group?
delete these guides from slide master before printing or giving to the client
36
delete these guides from slide master before printing or giving to the client
A Frequentist-Bayesian Approach
 We know there is variability in our estimate
for the treatment effect. Assume we can
describe this variability by
D | PoC data ~ N(2.5 ; (2/25) (7.14)2)
 We could obtain “average success prob” by
Prob of Success   Pr( P  value  0.05 | D) P(D | PoC data ) dD
delete these guides from slide master before printing or giving to the client
37
delete these guides from slide master before printing or giving to the client
A Frequentist-Bayesian Approach
 The success prob is 63.3% for 128 subjects per
group (80% power); and 67.7% for 172 subjects
per group (90% power). These are much lower
than the corresponding statistical power.
 If the PoC study had 70 subjects per group
(instead of 25) with the same estimated effect and
standard deviation, then the prob of success is
69.2% for 128 subjects per group and 75.6% for
172 subjects per group.
delete these guides from slide master before printing or giving to the client
38
delete these guides from slide master before printing or giving to the client
Observations
 If the required number is much higher than the number
obtained under the traditional approach, this could be a
signal that our prior information on treatment effect is
not sufficiently strong.
 The above suggests that we need to select appropriate
metrics to make decisions.
 For proof-of-concept study, the metric might be the
precision of the treatment effect estimate.
 For a dose-response study, the metric might be the
probability for selecting the right dose.
 For a confirmatory trial, a better metric might be the
probability of a successful trial.
delete these guides from slide master before printing or giving to the client
39
delete these guides from slide master before printing or giving to the client
Summary
 Pharmaceutical industry, as a whole, is facing
unprecedented challenges. These challenges also
create a lot of opportunities.
 A successful statistician needs to be responsive to
these challenges. Statisticians need to be able to
anticipate these challenges and offer solutions!
 It is not the strongest of the species that
survive, nor the most intelligent, but the one
most responsive to change.
- Charles Darwin
delete these guides from slide master before printing or giving to the client
40