Transcript Assessing the Total Effect of Time

Exploratory Analyses Aimed at
Generating Proposals for Individualizing
and Adapting Treatment
S.A. Murphy
BPRU, Hopkins
September 22, 2009
Outline
• Why Adaptive Treatment Strategies?
– “new” treatment design
• Constructing Strategies
• Why SMART experimental designs?
– “new” clinical trial design
– Q-Learning & Voting
• Example using CATIE
2
Adaptive Treatment Strategies operationalize multistage decision making.
These are individually tailored sequences of
interventions, with intervention type and dosage
adapted to the individual.
•Generalization from a one-time decision to a
sequence of decisions concerning interventions
•Operationalize clinical practice.
Each decision corresponds to a stage of intervention
3
Why use an Adaptive Treatment
Strategy?
– High heterogeneity in response to any one
intervention
• What works for one person may not work for
another
• What works now for a person may not work later
– Improvement often marred by relapse
• Remitted or few current symptoms is not the same
as cured.
– Co-occurring disorders/adherence problems are
common
4
Example of an Adaptive Treatment Strategy
Drug Court Program for drug abusing offenders. Goal
is to minimize recidivism and drug use.
High risk offenders are provided biweekly court
hearings; low risk offenders are provided “as-needed
court hearings.” In either case the offender is provided
standard drug counseling. If the offender becomes nonresponsive then intensive case management along with
assessment and referral for adjunctive services is
provided. If the offender becomes noncompliant during
the program, the offender is subject to a court
determined disposition.
The Big Questions
•What is the best sequencing of treatments?
•What is the best timings of alterations in treatments?
•What information do we use to make these decisions?
(how do we individualize the sequence of
treatments?)
Outline
• Why Adaptive Treatment Strategies?
– “new” treatment design
• Constructing Strategies
• Why SMART experimental designs?
– “new” clinical trial design
– Q-Learning & Voting
• Example using CATIE
7
Short Term Decision Making
• In short term decision making, decision makers use
strategies that seek to maximize immediate rewards at each
stage of treatment.
Problems:
– Ignore longer term consequences of present actions.
– Ignore the range of feasible future actions/interventions
– Ignore the fact that immediate responses to present actions
may yield information that pinpoints best future actions
8
Basic Idea for Constructing an
Adaptive Treatment Strategy:
Move Backwards Through Stages.
Action
Observations
Action
Observations
Stage 1
Stage 1
Reward
Stage 2
Stage 2
(Pretend you are “All-Knowing”)
9
Why SMART Trials?
What is a sequential multiple assignment randomized
trial (SMART)?
These are multi-stage trials; each stage corresponds to a
critical decision and a randomization takes place at each
critical decision.
Goal is to inform the construction of adaptive
treatment strategies.
Sequential Multiple Assignment Randomization
Initial T xt
Intermediate Outcome
Secondary T xt
Relapse
Early
Responder
R
Prevention
Low-level
Monitoring
Switch to
Tx C
Tx A
Nonresponder
R
Augment with
Tx D
R
Early
Responder
Relapse
R
Prevention
Low-level
Monitoring
Tx B
Switch to
Tx C
Nonresponder
R
Augment with
Tx D
Sequential Multiple Assignment Randomization
Initial T xt
Intermediate Outcome
Secondary T xt
Relapse
Early
Responder
R
Prevention
Low-level
Monitoring
Switch to
Tx C
Tx A
Nonresponder
R
Augment with
Tx D
R
Early
Responder
Relapse
R
Prevention
Low-level
Monitoring
Tx B
Switch to
Tx C
Nonresponder
R
Augment with
Tx D
Alternate Approach
• Why not use data from multiple trials to construct the
adaptive treatment strategy?
• Choose the best initial treatment on the basis of a
randomized trial of initial treatments and choose the
best secondary treatment on the basis of a
randomized trial of secondary treatments.
Delayed Therapeutic Effects
Why not use data from multiple trials to construct the
adaptive treatment strategy?
Positive synergies: Treatment A may not appear best
initially but may have enhanced long term
effectiveness when followed by a particular
maintenance treatment. Treatment A may lay the
foundation for an enhanced effect of particular
subsequent treatments.
Delayed Therapeutic Effects
Why not use data from multiple trials to construct the
adaptive treatment strategy?
Negative synergies: Treatment A may produce a
higher proportion of responders but also result in side
effects that reduce the variety of subsequent
treatments for those that do not respond. Or the
burden imposed by treatment A may be sufficiently
high so that nonresponders are less likely to adhere to
subsequent treatments.
Diagnostic Effects
Why not use data from multiple trials to construct the
adaptive treatment strategy?
Treatment A may not produce as high a proportion of
responders as treatment B but treatment A may elicit
symptoms that allow you to better match the
subsequent treatment to the patient and thus achieve
improved response to the sequence of treatments as
compared to initial treatment B.
Cohort Effects
Why not use data from multiple trials to construct the
adaptive treatment strategy?
Subjects who will enroll in, who remain in or who
are adherent in the trial of the initial treatments may
be quite different from the subjects in SMART.
Sequential Multiple Assignment Randomization
Initial T xt
Intermediate Outcome
Secondary T xt
Relapse
Early
Responder
R
Prevention
Low-level
Monitoring
Switch to
Tx C
Tx A
Nonresponder
R
Augment with
Tx D
R
Early
Responder
Relapse
R
Prevention
Low-level
Monitoring
Tx B
Switch to
Tx C
Nonresponder
R
Augment with
Tx D
Examples of “SMART” designs:
•CATIE (2001) Treatment of Psychosis in Alzheimer’s
Patients
•CATIE (2001) Treatment of Psychosis in
Schizophrenia
•STAR*D (2003) Treatment of Depression
•Pelham (on-going) Treatment of ADHD
•Oslin (2009) Treatment of Alcohol Dependence
Constructing proposals for more deeply
tailored adaptive treatment strategies:
Q-Learning
Q stands for “Quality of Treatment”
Q-Learning is a generalization of regression to
multistage treatment
20
Sequential Multiple Assignment Randomization
Initial T xt
Intermediate Outcome
Secondary T xt
Relapse
Early
Responder
R
Prevention
Low-level
Monitoring
Switch to
Tx C
Tx A
Nonresponder
R
Augment with
Tx D
R
Early
Responder
Relapse
R
Prevention
Low-level
Monitoring
Tx B
Switch to
Tx C
Nonresponder
R
Augment with
Tx D
In Q-Learning we run a regression at
each stage, moving backwards
through the stages.
Action
Observations
Action
Observations
Stage 1
Stage 1
Reward
Stage 2
Stage 2
22
Clinical Antipsychotic Trials of
Intervention Effectiveness
(Schizophrenia)
• Multi-stage trial of 18 months duration
• Relaxed entry criteria
• A large number of sites representing a broad
array of clinical settings (state mental health,
academic, Veterans’ Affairs, HMOs, managed
care)
• Approximately 1500 patients
23
CATIE Randomizations (simplified)
Stage 1
Randomized Treatments
OLAN QUET RISP ZIPR PERP
Stage 2
Treatment preference
Efficacy
Randomized Treatments CLOZ OLAN QUET RISP
Tolerability
OLAN QUET RISP ZIPR
Stage 3
Treatments selected
by preference
many options
24
Constructing Dynamic Treatment
Regimes using CATIE
• Reward: Time to Treatment Dropout
• Stage 1 regression analysis:
– Controls: TD, recent exacerbation, site
– Tailoring variable: pretreatment PANSS
• Stage 2 regression analysis:
– Controls: TD, recent exacerbation, site
– Tailoring variables: “treatment preference,” stage 1
treatment, end of stage 1 PANSS
25
Voting
(exploratory analysis)
•Our goal is to estimate the probability that a treatment
would look best if we repeat the CATIE study. We want
to estimate this chance for each treatment at each phase.
• We “simulate” the action of repeating the study using
bootstrap samples. Each bootstrap sample “votes” for
the treatment it finds best at stages 1 and 2. The fraction
of votes for a treatment is the estimate of the
probability that this treatment will be found best.
26
27
28
Challenges
• We have since improved the voting and can
now add confidence intervals.
• Clinical Decision Support Systems
– We need to be able construct adaptive treatment
strategies that recommend a group of treatments
when there is no evidence that a particular
treatment is best.
29
Acknowledgements: This presentation is based on
work with many individuals including Eric Laber,
Dan Lizotte, John Rush, Scott Stroup, Joelle
Pineau and Susan Shortreed.
Email address: [email protected]
Slides with notes at:
http://www.stat.lsa.umich.edu/~samurphy/
Click on seminars > health science seminars
30
Voting
(exploratory analysis)
•Use bootstrap samples to estimate percentage of the
time that treatment A1=1 is favored:
•Natural approach will not work, e.g.
is not necessarily consistent.
• We use an adaptive bootstrap
31
Voting
(exploratory analysis)
•Use an “adaptive” bootstrap method to estimate
percentage of the time that treatment A1=1 is favored:
•Adaptive bootstrap estimator:
32
Treatment of Schizophrenia
•
Myopic action: Offer patients a treatment that reduces
schizophrenia symptoms for as many people as possible.
•
The result: Some patients are not helped and/or experience
abnormal movements of the voluntary muscles (TDs). The
class of subsequent medications is greatly reduced.
•
The mistake: We should have taken into account the variety
of treatments available to those for whom the first treatment is
ineffective.
•
The message: Use an initial medication that may not have as
large a success rate but that will be less likely to cause TDs.
33
Treatment of Opioid Dependence
•
Myopic action: Choose an intensive multi-component
treatment (methadone + counseling + behavioral
contingencies) that immediately reduces opioid use for as
many people as possible.
•
The result: Behavioral contingencies are
burdensome/expensive to implement and many people may
not need the contingencies to improve.
•
The mistake: We should allow the patient to exhibit poor
adherence prior to implementing the behavioral
contingencies.
•
The message: Use an initial treatment that may not have as
large an immediate success rate but will allow us to ascertain
whether behavioral contingencies are required.
34
Example of an Adaptive Treatment Strategy
Treatment of depression. Goal is to achieve and
maintain remission.
Provide Citalopram for up to 12 weeks gradually increasing dose
as required.
If, there is no remission yet either the maximum dose has been
provided for two weeks, or 12 weeks have occurred, then
if there has been a 50% improvement in symptoms,
augment with Mirtazapine.
else switch treatment to Bupropion.
Else (remission is achieved) maintain on Citalopram and provide
web-based disease management.
35