Transcript Carroll presentation

Kathleen Carroll & Brian Kiluk
Division of Substance Abuse
Yale University School of Medicine
Supported by NIDA Supplement to R01 DA15969
and P50 DA09241, U10 DA015831,
R01 DA019078, & R01 DA 10679
Why do we need a sound and
valid indicator?






Facilitation of comparisons
across
projects, meta-analyses
Set and monitor performance
standards
Benchmarking
Clearly convey magnitude of treatment effects to
stakeholders
Facilitate comparisons across common standard
Lack of incentive to improve performance and
outcome (retention not appropriate standard)
Overview
Desirable
characteristics of
indicators
 Strengths and
weaknesses of
common approaches
 Overview of our
project

“Traditional” indicators of clinical
significance almost always
translate to complete abstinence
Return to normative levels
 Reliable change indices
 Return to healthy functioning? (e.g.,equivalent
of ‘no heavy drinking days’ for stimulant users)

What are we looking for in an indicator?
Easy to calculate, interpret
 Psychometrically sound, reliable, replicable
 Low susceptibility to missing data
 Verifiable (biologic indicator, other)
 Independence from baseline measures
 Sensitive to treatment effects
 Low(er) cost
 Predicts long-term cocaine outcomes
 Related to indicators of good long term functioning
 Acceptable to field
 Easily interpreted by clinicians, policy makers, payors

What is ‘success’ in treating
stimulant users?
Durable periods of abstinence
 Employment, productivity
 Lack of criminal activity
 Reduced use of expensive, avoidable
health care resources

11% at end of treatment, 21% at end of 1 year
follow up
Why not complete abstinence?
Insensitive to change
 Difficult standard for most individuals
(14% of our sample of 434)
 Chronically relapsing disorder
 Change is dynamic
 Starting and remaining abstinent may imply
questionable need for treatment
 Our data: Weak relationship with cocaine
use and functioning outcomes at one year

Retention
Pros
Cons
 May be more meaningful in
 Easy to calculate
some contexts than others
 Available for all
 Participants leave treatment
participants
for different reasons
 Indicator of treatment
 Is retention with continued use
acceptability
meaningful?
 Indicator of
 Is compliance with ineffective
differential
treatment meaningful?
attrition/data
 Not related to long-term
availability across
outcome in our sample
conditions
Percent negative urines
Pros





Widely used and accepted
Less susceptible to
demand characteristics,
misrepresentation
Quantifiable, ability to
detect new episodes
Very accurate, if
appropriate schedule of
collection and minimal
missing data
Timing is critical
(overlap, missing data)
Cons







Recent use only (3-5)
High cost for frequent or
quantitative
Sensitive to missing data,
esp. with differential
attrition
Depends on assumptions
(missing, denominator)
Stimulants or all drugs?
Can’t back-fill
Problems with assuming
missing=positive*
Calculating percent urine
samples
Example: 1 negative urine, 2 sessions, then
dropout of 12 week trial.
Based on submitted:
Based on possible:
Based on expected/ 1x
Based on expected/ 3x
Percent cocaine positive
100%
50%
8%
3%
0%
Longest consecutive x-free urine specimens
Pros
Cons
Strong evidence of
meaningful abstinence
 Less susceptible to
demand characteristics,
misrepresentation
 Quantifiable, ability to
detect new episodes
 Very accurate, if
appropriate schedule of
collection and minimal
missing data
 Timing is critical (overlap,
missing data)






High cost for frequent or
quantitative
Very sensitive to missing
data, esp. with differential
attrition
Depends on assumptions
(missing, denominator)
Stimulants or all drugs?
Can’t back-fill
Percent days abstinent, self
report
Pros





Widely used
Potentially available for all
participants and all days if
TLFB used with high data
completion; highly flexible
True intention to treat possible
Can be reliable if methods to
enhance reliability used (at a
cost)
Our discrepancy rate=8-12%
Cons


With high/differential
dropout, what’s the
denominator? Days in
treatment versus days
expected?
Not easy to correct with
urine data if
discrepancies high
Maximum days of abstinence,
overall or in final x weeks
Pros
 Linked linked to
longer-term cocaine
use
 Potentially verifiable
if urines collected at
appropriate intervals
 Provides ‘grace
period’
 Easily dichotomized
(eg 3 plus weeks)
Cons
 High complexity with
missing data, especially
dropouts
 High complexity if
discrepant urine data
 Participants last 2 weeks
or last 2 weeks of trial?
 End of treatment or
sometime within
treatment?
Reduction in use:
Frequency and or quantity
Pros
Alternative to
abstinence; more
achievable target?
 Highly compatible with
random regression
models
 Sensitive to treatments
that may take time for
effects to emerge
 Provides ‘grace period’
 Easily dichotomized

Cons
 Complexity obtaining
accurate estimates of
frequency/quantity of use
prior to baseline
 When is reduction
measured (last weeks?
Entire course?
 Costs of repeated
quantitative urines,
sensitivity to missing
data
Issues in defining ‘reduction’
Patterns vary widely (binge versus low
use)
 Reliable estimation of quantity complex
(illicit, no standard units, adulterants
common, potency varies, hard to
standardize ‘hits’ ‘joints’ ‘dime bags’)
 Difficulty of estimating dollar value
(commerce, shared use, sex for drugs)

Which indicator of treatment
response?
Loss of power with dichotomous, but also
easily interpretable, calculable for all,
relevant to clinical significance
 Candidates
 *Complete abstinence
 *3 or more weeks of abstinence
 *End of treatment abstinence
 *Reduction of x percent
 “Good functioning”
Indicator
Days retained in
1 treatment protocol
Percentage of urine
specimens testing
2 positive
Maximum consecutive
days abstinent
Ease of
computation
Verifiability
Vulnerability to missing
data
Relative cost
Operationalization
for these analyses
Easy
Yes-
Low
Low
Easy for complete data
Yes, by definition
Assumes independence of
urine specimens
(denominator), assumes
numerator is unbiased by
collection schedule or missing
data.
High
Days from
randomization to endpt
Number of cocainenegative urine
specimens collected /
all specimens collected
Yes, provided
appropriate schedule of
data/urine collection
Likely to result in casewise
missingness or reduced
sample size
Moderate, due to
biological
verification and
derivation from
TLFB
Depends on treatment
Yes, provided
duration, level of
appropriate schedule of
missing data, and
data/urine collection
intermittent
missingness
Likely to result in casewise
missingness or reduced
sample size
Moderate, due to Number of self-reported
biological
days of abstinence from
verification and
cocaine / days in
derivation from
treatment (retention)
TLFB
Complex for intermittent
Yes, provided
and monotone,
appropriate schedule of
dropouts
data/urine collection
Low
Moderate, due to
biological
verification and
derivation from
TLFB
For those retained 14+
days, longest cluster of
abstinence in final 2
weeks; otherwise 0
For those retained 14+
days, 0 days of use in
last 14 days, otherwise
0
C
C
C Easy for complete data
3
Percent days of
abstinence from
4 cocaine
Maximum days of
continuous abstinence
during last two weeks
5 of treatment
Completely abstinent
last two weeks of
6 treatment
C
C
Easy
Yes, provided
appropriate schedule of
data/urine collection
Low
Moderate, due to
biological
verification and
derivation from
TLFB
Easy
Yes, provided
appropriate schedule of
data/urine collection
Low
Moderate, due to
“Yes” if participant
biological
retained 21+ days, max
verification and
days abstinent > 20.
derivation from
Otherwise No
TLFB
Easy
Yes, provided
appropriate schedule of
data/urine collection
Low
Moderate, due to
“Yes” if participant
biological
retained 14+ days, max
verification and
days abstinent > 13.
derivation from
Otherwise No
TLFB
D
3 or more weeks of
7 continuous abstinence
Longest continuous
cluster of self-reported
abstinence within
treatment
D
2 or more weeks of
D
8 continuous abstinence
Note. C=continuous, D=Dichotomous, TLFB=Timeline Followback method
Ease of computation
Verifiability
Vulnerability to
missing data
Easy
Yes, provided
appropriate schedule of
data/urine collection
Low
Moderate, due to
“Yes” if participant retained
biological verification
7+ days, max days
and derivation from abstinent > 6. Otherwise No
TLFB
Easy
Same
Low
Moderate, due to 0 days of use and 0 positive
biological verification urines
and derivation from
TLFB
Easy
Yes
Low
Low
Completion of treatment, 0
days of use in final week
Complex, baseline
definition can be
arbitrary
No, relies on accurate
baseline/pretreatment
assessment
Moderate
Low
Percent days of use in final
28 days of treatment/
percent days of use in 28
days prior to baseline
Complex, baseline
definition can be
arbitrary
Relies on access to
accurate
baseline/pretreatment
level of use
Moderate
Low
% reduction is 50% or
higher
Complex, baseline
definition can be
arbitrary
Same
Moderate
Low
% reduction is 75% or
higher
Easy
Partial
Low
Low
Completes treatment, 0
days of problems in drug,
legal, employment and
psych ASI in past 28 days
Indicator
Relative cost
Operationalization for
these analyses
1 or more weeks of
9 continuous abstinence
Completely abstinent
from cocaine during
treatment
D
D
10
Completed treatment
and abstinent in last
11 week
Percent reduction in
frequency of use (28
days prior/days last 4
12 weeks)
D
C
50% reduction in
13 frequency of use
D
75% reduction in
14 frequency of use
D
Report no drug use,
legal, employment, or
psychological problems
last 28 days of
D
15 treatment
Note. C=continuous, D=Dichotomous, TLFB=Timeline Followback method
Vulnerability to Sensitivity to Relationship
treatment with post tx
missing data
Ease of
computation
Biological
verification
21-30 days of
abstinence
X
Relies on
appropriate
schedule
Low
Completed treatment
and abstinent in last 2
weeks
X
Same
Low
Indicator
effects
Complex, baseline Relies on having
definition can be
accurate
arbitrary
baseline/pretreat
ment assessment
of use
Moderate
% days
abstinent
Depends on treatment
duration, complex for
dropouts, and
intermittent
missingness
X, provided
appropriate
schedule of
data/urine
collection
Moderate
Max days
consecutive
abstinence
Complex for
intermittent and
monotone
missingness, dropouts
X, provided
appropriate
schedule of
data/urine
collection
Likely to result in
casewise
missingness or
reduced sample
size
Percent neg ative
urine specimens
Easy except
when missing
data
yes
50 % reduction
outcomes
Independent
from baseline
indicators
Relationship to
measures of
general function
So far…





Carroll, K.M., Kiluk, B.D., et al. (2014). Towards empirical identification of a
reliable and clinically meaningful indicator of treatment outcome for illicit drug
use. Drug and Alcohol Dependence, 137, 3-19.
Kiluk, B.D., et al. (2014). What happens in treatment doesn’t stay in
treatment: Cocaine abstinence during treatment is associated with fewer
problems at follow-up. J Consulting and Clinical Psychology, 82:619-27.
DeVito, E.E., et al. (2014). Gender differences in clinical outcomes for
cocaine dependence: Randomized clinical trials of behavioral therapy and
disulfiram. Drug and Alcohol Dependence, 145: 156-167.
Decker, S.E., et al. (2014). Assessment concordance and predictive validity
of self-report and biological assay of cocaine use in treatment trials. The
American Journal on the Addictions, 23, 466-74.
Kiluk, B.D., et al. (in press). Prompted to treatment by the criminal justice
system: Relationships with treatment retention and outcome among cocaine
users.