Transcript Deriving minimally important difference for PRO data

Deriving the minimally
important difference for PRO
data
EFSPI HTA Scientific Meeting
Berlin, 2014-09-25
Dr. Christoph Gerlinger
Acknowledgements
This talk is based on joint work with my colleagues
 Florian Hiemeyer
 Dr. Thomas Schmelter
<2> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Outline
Problem statement
Statistical methods to derive a minimally important difference (MCID) for the
patient
Worked example
Points to consider in deriving an MCID for the patient
Open issue: MCID between treatments
<3> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Problem statement
For every measure we need an interpretation what the number does mean
 50 kg body weight
 4.9 mmol/l of cholesterol
 14 mm difference in pain of on a 100 mm visual analogue scale
 Baseline
 End of study
<4> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Problem statement – 2
For body weight or cholesterol this is a clinical judgment
 Statistics can’t help
For pain we can ask the patient what the measure does mean to her/him
 And use some statistics to derive the minimal important difference for the
patient
 Use the minimal important difference for the patient to derive a responder
definition
Open problem: How to derive the minimal important difference between different
treatments
 Suggestions very welcome: [email protected]
<5> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Methods to derive MCID for PROs
Anchor based with patient satisfaction as anchor
 Discrimant analysis
 ROC analysis
 Mean of patients who changed (Juniper et al. 1994)
Distribution based
 Half standard deviation rule (Norman et al. 2003)
 Standard error of measurement (Wyrwich et al. 1999)
<6> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
FDA Guidance (Dec 2009)
www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf (2014-09-03)
<7> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Worked example - menopause
Menopause: the permanent cessation of the primary functions of the human
ovaries
Typically occurs in women in midlife, during their late 40s or early 50s, and
signals the end of the fertile phase of a woman's life.
The menopause is a natural and irreversible process, it is not a disease.
The decline in estrogen results in a range of symptoms, e.g. vasomotor
symptoms (hot flushes, night sweats and palpitations), insomnia, mood changes,
loss of libido, etc.
These symptoms can have a significant effect on a woman’s quality of life.
<8> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Data from a clinical trial
735 patients randomized (1:1:1:1) to 3 active doses and placebo for 12-weeks
minimum of 50 moderate to severe hot flushes per week
 Moderate hot flush: heat sensation with sweating
 Severe hot flush: sweating so intense cause interruption of current activity
primary efficacy variables were mean changes from baseline to weeks 4 and 12
in the weekly frequency and weekly mean daily severity of moderate to
severe hot flushes (as per FDA guidance)
hot flushes recorded daily on diary cards
Archer DF et al. Menopause. 2014 Mar;21(3):227-35.
MCID analysis on clean database but without treatment information
Results of blinded analyses sent to FDA
FDA Type A meeting prior to unblinding to agree on methods and results
<9> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Patients‘ view
Original1 CGI
scale
Percentage
Very much improved
42.5%
Much improved
24.7%
Minimally improved
18.0%
No change
11.9%
Minimally worse
aggregated CGI
scale
Percentage
Very much / much improved
67.1%
Minimally improved
18.0%
No change / Worse
14.9%
1.7%
Much worse
0.9%
Very much worse
0.3%
Note: low proportion of no change/worse due to several active arms and pronounced placebo effect
1 Guy
W (Ed). ECDEU Assessment Manual for Psychopharmacology. 1976. Rockville,
MD, US Department of Health, Education, and Welfare.
<10> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Change in hot flushes by aggregated
CGI
<11> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Density estimates for hot flushes by
aggregated CGI
<12> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
MCID: Anchor based approach
According to non-parametric discriminant analysis
 the cut-off between “no change/worse” and “minimally improved” was -19.1
moderate to severe hot flushes per week
 this cut-off value can be interpreted as the minimal clinically important
difference (for the patient)
 The cut-off between “minimally improved” and “much/very much improved”
was -40.3 moderate to severe hot flushes per week
 this cut-off value can be interpreted as a clinically important difference (for
the patient)
<13> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
ROC
0.6
0.4
19.1 used as discrimination point between
no change or worse
and minimally improved
0.0
0.2
sensitivity
0.8
1.0
(no change or worse vs. minimally improved)
0.0
0.2
0.4
0.6
1-specificity
<14> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
0.8
1.0
MCID: Distribution based approach
According to Norman et al. (2003), a MCID for a patient reported quality of life
outcome can be calculated by using half the standard deviation of the population
 The resulting MCID is 19.2 hot flushes per week.
Compares well to the MCID of 19.1 for the anchor based approach
<15> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Summary of worked example
MCID of the patients could be derived empirically
We translated the MCID of the patients into a responder definition: a subject is a
responder if and only if she has
 a reduction of at least 19.1 moderate to severe HF per day at week 4 AND
 a reduction of at least 40.3 moderate to severe HF per day at week 12
Confidence interval for MCID can be derived by bootstrapping
Methods and results were discussed and agreed upon with regulator before
unblinding the trial
Details in Gerlinger et al. 2012
<16> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Points to consider – anchor question
Choice of anchor question
 Many anchor questions available
 Makes little difference (work in progress)
 Satisfaction with treatment influenced by efficacy and safety
<17> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Points to consider – anchor method
Choice of statistical method
 Discriminant analysis
 ROC curve
 Mean of improved
<18> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Points to consider – distribution based
<19> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Summary - points to consider
Several methods to derive the patients’ MCID
No clearly best method
 One bad method (mean of improved patients), in my view
Results of different methods are often close
Patients’ MCID can be transformed into responder definition
The empirically derived MCID and responder definition were acceptable to
stakeholders in several indications
 Hot flushes, pain, acne
<20> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
MCID between treatment arms
So far, we derived the MCID of a patient
Question from a stakeholder: MCID between treatment arms?
Quite obvious to me
MCIDpatient  MCIDbetween treatments
But not so obvious whether there is a strict inequality
If 15 mm in pain reduction make a difference to patients
 Are two treatments with a 30 mm and a 40 mm reduction equivalent or
different ?
 Judgment ?
<21> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
One suggestion
define the MCID between treatments in terms of the difference in response rates
You can then “reverse engineer” the MCID on the original scale
Yes, but
A 5%-point difference in the rate of patients satisfied with their acne treatment is
not the same as a 5%-point difference in mortality, in my view
Again, this is a judgment
<22> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
Conclusion
The patients’ MCID can be derived for a PRO using statistics
 This is better than arbitrarily choosing a nice cut-off like 3.1415 or 2.7182 or
log2(1.4142)
The MCID can be used to derive a responder definition
No method to derive MCID between treatment arms, yet
Your ideas, please
<23> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25
References (recommended reading)
Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of
half a standard deviation. Med Care 2003;41:593–596.
Gerlinger C, Schmelter T. Determining the Non-Inferiority Margin for Patient Reported Outcomes. Pharmaceutical Statistics,
2011 Sep;10(5):410-3.
FDA PRO Guidance.
www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf (2014-09-03)
Juniper EF, Guyatt GH, Willan A, Griffith LE. Determining a minimal important change in a disease-specific Quality of Life
Questionnaire. J Clin Epidemiol. 1994 Jan;47(1):81-7.
Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful
intra-individual changes in health-related quality of life. J Clin Epidemiol. 1999 Sep;52(9):861-73.
Gerlinger C, Städtler G, Götzelmann R, Graupe K, Endrikat J.: A non-inferiority margin for acne lesion counts. Drug
Information Journal. 2008 Nov;42(6):607-15.
Gerlinger C, Schumacher U, Faustmann T, Colligs A, Schmitz H, Seitz C. Defining a minimal clinically important difference
for endometriosis-associated pelvic pain measured on a visual analog scale: analyses of two placebo-controlled,
randomized trials. Health Qual Life Outcomes. 2010 Nov 24;8(1):138
Gerlinger C, Gude K, Hiemeyer F, Schmelter T, Schäfers M. An empirically validated responder definition for the reduction
of moderate to severe hot flushes in postmenopausal women. Menopause. 2012 Jul;19(7):799-803.
Archer DF, Schmelter T, Schaefers M, Gerlinger C, Gude K. A randomized, double-blind, placebo-controlled study of the
lowest effective dose of drospirenone with 17β-estradiol for moderate to severe vasomotor symptoms in
postmenopausal women. Menopause. 2014 Mar;21(3):227-35.
<24> • Christoph Gerlinger • EFSPI HTA Meeting • Berlin, 2014-09-25