Transcript The place of meta-analyses in the regulatory evaluation process

Comparative Drug Effects
and
Network Meta-Analyses
Maylis COSTE, Chrissie FLETCHER
EFSPI Statistical Leaders Meeting, June 8th 2011
1
Introduction
 Network Meta-Analysis: context and example
 Overview of documents and initiatives on
Network Meta-Analysis
Discussion on NMA




Context of CER
Involvement of pharmaceutical statisticians
A multidisciplinary collaboration
Need for reference documents and Guidance
EFSPI Statistical Leaders Meeting, June 8th 2011
2
Context of Network Meta-Analysis
Aim: To evaluate relative efficacy of several drugs or interventions for
decision-making
Method:
• To establish a network of evidence
• To perform an integrated and unified analysis
• To estimate pairwise treatment comparisons and/or classify
competitors, incorporating direct and indirect evaluations
• To investigate validity and assumptions (heterogeneity,
consistency,…)
EFSPI Statistical Leaders Meeting, June 8th 2011
3
Meta-Analyses and types of comparisons
Treatment X versus
• Placebo
• An other active treatment
• A therapeutic class
• Some active treatments
(considered sequentially)
• Several (all) active
treatments
(considered simultaneously)
• Direct comparisons
 Meta-analysis
• No direct comparison but a
common comparator
 Indirect comparison
• Direct comparison(s) and indirect
comparison(s)
 Mixed comparison
(including Meta-analysis)
• Direct comparison(s) and indirect
comparison(s) with a (several)
common comparator(s)
 Network Meta-Analysis
EFSPI Statistical Leaders Meeting, June 8th 2011
4
An example of Network Meta-Analysis
Comparative efficacy and acceptability of 12 new-generation
antidepressants: a multiple-treatments meta-analysis
A. Cipriani and al., Lancet 2009
EFSPI Statistical Leaders Meeting, June 8th 2011
5
An example of Network Meta-Analysis (cont.)
Figure 3: Efficacy (response rate) and acceptability (drop-out rate) of the
12 antidepressants
Odds ratios – 95% Credibility Intervals
Random effect model within a Bayesian framework
EFSPI Statistical Leaders Meeting, June 8th 2011
6
An example of Network Meta-Analysis (cont.)
Figure 4:
Ranking for efficacy (solid red line) and acceptability (dotted blue line)
EFSPI Statistical Leaders Meeting, June 8th 2011
7
An example of Network Meta-Analysis (cont.)
Investigation of inconsistency (Dias, 2010)
Distributions of 2 selected treatment difference
Posterior densities of the mean log-odds ratio (LOR) calculated using the full MTC model
(black), and direct and indirect evidence only (red and blue respectively)
MTC : LOR = -0.181 [-0.359 ; -0.001]
Direct : LOR = -0.394 [-0.640 ; -0.144]
Indirect : LOR = 0.037 [-0.209 ; 0.277]
Inconsistency estimate = -0.429 [-0.775 ; -0.085]
Test for Inconsistency:
P = 0.008 < 5%
EFSPI Statistical Leaders Meeting, June 8th 2011
8
Perspectives on Comparative Drug Effect
Relative efficacy of drugs: an emerging issue between
regulatory agencies and third-party payers
HG Eichler & coll
Nature Reviews - Drug Discovery Vol 9, Apr. 2010, pp 277-291
“All stakeholders, including academic groups, will need to agree
on and apply common statistical research standards for non
interventional approaches to Relative Efficacy Assessment: that is,
Indirect Comparisons, Network Meta-Analyses and Observational
Studies. This may encompass some form of preregistration of study
protocols before funding and start of the study.”
EFSPI Statistical Leaders Meeting, June 8th 2011
9
ISPOR initiative
Internal Society of Pharmacoeconomics and Outcome
Research (ISPOR)
Part 1: Interpreting Indirect Treatment Comparisons & Network MetaAnalysis for Health Care Decision-making
Report of the ISPOR Task Force on Good Research Practices
JP Jarsen (MAPI Values) and US&EU collaborators
•
•
•
•
Multiple treatment comparison & evidence networks
Synthesis of evidence
Analysis
Critically reviewing and interpreting an indirect treatment
comparison or network meta-analysis
• Decision making in the absence of direct / indirect treatment
comparisons of RCTs
EFSPI Statistical Leaders Meeting, June 8th 2011
10
ISPOR initiative (cont.)
Part 2: Conducting Indirect Treatment Comparisons & Network MetaAnalysis Studies
Report of the ISPOR Task Force on Indirect treatment Comparisons
D. Hoaglin & US and EU collaborators
Good Research Practices Guidance on more-technical aspects of
conducting NMA
• Models (fixed and random effects)
• Frequentist versus Bayesian framework
• Model validation
• Example Bayesian Network
 Two research practices papers to be published in Value in Health
EFSPI Statistical Leaders Meeting, June 8th 2011
11
EUnetHTA Initiative
Work Package 5 – Relative Effectiveness Assessment
(REA) of Pharmaceuticals
EUnetHTA : European network for Health Technology Assessment
(network of government appointed organisations, a large number of regional agencies
and non-for-profit organisations producers or contributors to HTA in 29 European
countries)
 Lead Partners: the Netherlands, France
 Project: to scientifically summarize the available methodology
on REA and come to common methodology that will closely relate
to what is already happening in daily practice
 Two assessments of RE:
• Rapid assessment for a new technology at the time of
introduction to the market and in comparison to standard care
• Full assessment of (all) available technology(ies)
 Presentation of Rapid and Full model (12/2012)
EFSPI Statistical Leaders Meeting, June 8th 2011
12
DIA initiative
DIA Comparative Effectiveness Research Scientific Working
Group (CER-SWG)
MD Rotelli, FDA & EU members(C. Fletcher, J. Roehmel, M.A. Paget)
 Non competitive collaboration among staff from regulatory agencies,
pharmaceutical and biotech companies and academia to share ideas and
advance on the science of CER
• Scientific forum for the discussion and development of innovative
methods and software for the design, analysis, and interpretation
of CER
• Education and promotion of the dissemination of methods and
best practices in CER, setting expectations for high quality
standards, and sharing experiences on case studies
• Engagement in dialogue and advocacy efforts with industry
leaders, the scientific community and regulators to develop a
world-wide consensus position on when and how to consider the
use of CER
 Scientific White Paper, DIA public conferences, submissions to DIJ,…
EFSPI Statistical Leaders Meeting, June 8th 2011
13
EFSPI/PSI HTA SIG Initiative
Network Meta-Analysis for Health Technology
Assessment
Seminar at PSI Conference (May 2011)
B. Jones and collaborators (C. Fletcher)
 Concepts used in NMA
 Description of work of Evidence Synthesis sub-team
 Forthcoming paper: methods, example and statistical programs
Publication to be submitted in Pharmaceutical Statistics Journal
“Network Meta-analysis: what every pharmaceutical statistician
should know?”
EFSPI Statistical Leaders Meeting, June 8th 2011
14
Basler Biometric Section Initiative
BBS Seminar (May 2011)
Comparative Quantitative Assessments Benefit-Risk &
Effectiveness
Industry Perspective on Comparative Effectiveness Research (CER)
and the impact of Health Technology Assessment (HTA) in Europe
(C. Fletcher)
EFSPI Statistical Leaders Meeting, June 8th 2011
15
Discussions on NMA
1. Relevance for CER
2. Involvement of pharmaceutical statisticians
3. A multidisciplinary collaboration
4. Need for reference documents or guidance from Working
groups or Regulatory bodies
 Position of Statistical Leaders
EFSPI Statistical Leaders Meeting, June 8th 2011
16
Discussion (1)
Relevance of Network Meta-Analysis for CER
 Do we consider that statistical methods on NMA are sufficiently
developed and published to generate relevant and reliable
information from a CER perspective?
 Are there technical aspects on NMA that need further research
(investigation of assumptions, inconsistency, meta-regression,...)?
 Should other considerations on NMA application be considered
(structure and properties of network, multiple endpoints,
heterogeneity with small numbers of studies by treatments, placebo
arm…)?
 Could Individual Data Bases improve notably the conclusions drawn
from NMA? Feasibility?
How could we support this statistical (academic) research on NMA?
EFSPI Statistical Leaders Meeting, June 8th 2011
17
Discussion (2)
Involvement of pharmaceutical statisticians in Network
Meta-Analyses
 Are pharmaceutical statisticians sufficiently aware and
experienced on NMA?
 How to develop the methodology and scientific skills among the
non-academic community of statisticians to be qualified to
conduct or interpret NMA?
 What should be the topics a qualification on NMA should cover:
protocol, analysis (statistical methods, validation and software)
and interpretation?
 How could we support informing and training the statistical (non
academic) community on NMA?
EFSPI Statistical Leaders Meeting, June 8th 2011
18
Discussion (3)
Network Meta-Analysis : a cross-functional collaboration
 Who is the initiator of a NMA (HTA, “Big Pharma”, Cochrane
Collaboration,…)?
 For HTA initiative, who would participate in protocol, analysis and
communication of results (pharmaceutical companie(s), regulatory
bodies, specialized CROs, academics…)?
 What would be its place in drug development (initial submission,
B/R reevaluation,…)?
 For pharmaceutical initiative, which departments should be involved
: statistics, clinics, regulatory affairs, scientific documentation…?
 Should a NMA be periodically updated to take into account new
evidence?
How could we support the involvement of pharmaceutical
statisticians in NMA?
EFSPI Statistical Leaders Meeting, June 8th 2011
19
Back-up
Specifications of Network Meta-Analysis
 Criteria for trials selection
• pathology
• patients (with or without restriction to indication,…)
• trial design (RCT, cross-over, placebo-controlled,…)
• competitors (registered or in development, whatever the
dosage, individually or according to therapeutic class,
placebo arm, with or without background therapy,…)
• evaluation criteria (“common” criteria on efficacy, safety,
drop-out, other…)
 Data sources
• publications
• EPAR, SBA, EudraCT,
• Data Bases
EFSPI Statistical Leaders Meeting, June 8th 2011
20
Discussion (4)
Need for reference documents or guidelines from
Working groups or Regulatory bodies on Network
Meta-Analyses
 Are there other groups involved in this area?
 Who would be the best partner?
 Do we need (further) guidance on NMA? from HTA? Pharmacoeconomics? CHMP?
 What are the need in terms of good practice of NMA of non
academic statisticians (PSI initiative)? of other contributors?
 How could we participate in Working Groups to disseminate
information and participate in Guidelines on NMA?
EFSPI Statistical Leaders Meeting, June 8th 2011
21