Transcript Developing a Clinician Resource for Evidence-Based

Toolkits
Developing a Clinician Resource for
Evidence-Based Treatment Delivery
Deni Carise, Ph.D.
Tom McLellan, Ph.D.
Adam Brooks, Ph.D.
Robert Forman, Ph.D
Supported by NIDA Grant (R21DA-015977)
The Research-Practice Gap
• 25 years of heavy research investment in:
–Medication Development
–Behavioral Therapies
• Numerous treatments have:
–Demonstrated efficacy in clinical trials
–Demonstrated effectiveness in
dissemination studies
The Research-Practice Gap
• Research shows very low rates of adoption of
these evidence-based practices
• Possible factors leading to non-adoption:
– Financial
– Complexity of interventions
– Provider organizational and policy concerns
The Research-Practice Gap
• We interviewed counselors and program
directors in the Delaware Valley Research
Practice Collaborative (PA, DE, NJ)
• Initial Focus Groups revealed two barriers as
greatest contributor to the “gap”:
– Finance
– Training/Supervision
The Research-Practice Gap
New therapies won’t be adopted if:
– the treatments cannot be
supported by managers and funding
agencies
– the training and supervision
burdens of the treatments are
overwhelming
The Financial Barrier
• It is a challenge to implement evidencebased treatment practices within the
severely challenged infrastructure*
•Addiction treatment system has
experienced a 20-year period of declining
funds:**
*McLellan, Carise and Kleber, 2003.
**Mark, Levit, et. al. 2007.
The Financial Barrier
• General healthcare funding
declined 12% between 1988 and
1998*
• During the same period,
addiction treatment funding
declined 75%*
*Galanter, Keller et. al. 2000.
Financial Barrier Consequences
Program closures or re-organizations
• Increased counselor and management
turn-over*
• In a survey of 450 treatment programs:
• 32% faced the threat of closure
• 5% actually closed during the year of
the survey**
*Gallon, Gabriel et. a. 2003; McLellan, Carise et al. 2003. Knudsen, Ducharme et. al.
2004. **Roman, Blum 1997.
Financial Barrier Outcomes
Increased Reliance on Group Therapy
• Reimbursement for individual
therapy is under-funded or not
funded
• Offered as primary treatment
modality
Group Psychotherapy
• Can be as effective as individual
therapy*
• Are a top priority for bridging the gap
• Evidence-based approaches have often
not been adapted for group treatment
*Weiss, Jaffee et al. 2004.
Supervision Barrier
•Research shows that clinical
supervisors:
• spend less time mentoring and
training
• spend more time addressing human
resource or regulatory compliance
issues
• often have ongoing case loads of their
own
Training Barrier
• Of 400 national substance abuse treatment
centers, 20% had no staff training budget*
Decreased funding
+
Increased licensing,
accrediting, and
funding compliance
requirements

Decreased clinical
training
and
Increased attention to
administrative issues
*Johnson 2000.
Training Demands of EBPs
• Research on Training EBPs shows:
– Manuals (if read) are useful for learning
interventions, but not enough to change
counselor practice*
– To be proficient in conducting new
interventions, training needs to be ongoing:
•Requires more time than a standard 2-3
day workshop
•Requires ongoing, expert supervision
*McCarty, Fuller et al. 2007
The Big Problem
•We have complex treatments
– Content heavy
– Require intensive training
– Proven efficacy when delivered correctly
•We have a resource starved environment
•We have heavy turn-over in the field
Looking to Other Fields
• The field of education has experienced
similar problems
– Minimal resources
– Stressed workforce
– Complicated interventions (lessons) with
little time to prepare
Possible Solution
Education field Toolkits An evidence-based approach to
addressing the challenge of upgrading
skills in a resource restricted
environment
Possible Solution
•Teachers use packaged lesson plans to
assist teachers in conveying complex
concepts
•They can be taught effectively, require
little supervision and have been
constructed, tested and refined through a
scientific process
Possible Solution: Examples
Possible Solution: Examples
Possible Solution: Examples
Existing Treatment Curricula
• Currently, there are some existing
prepackaged curricula for substance
treatment
•Vary to the degree that they are informed
by evidence based content
• Few, if any, have been rigorously tested to
assess their impact on treatment or
outcomes
Testing a Curriculum Sample
• We conducted significant background work
to determine provider interest in a
curriculum-based approach
• We developed and pilot-tested a singlesession curriculum “Toolkit”
Surveying Interest
•21-item survey designed for the Clinical
Trials Network projects called: “What Do
You Need?”
•Administered in 18 community-based
treatment programs in the Delaware Valley
Practice Research Collaborative (PA, NJ, DE)
Surveying Interest
•269 treatment program staff completed the
survey
•195 employees identified themselves as
“treatment providers”
–Other options: researcher, managed care
or faith-based organizations, support staff,
or consumers were not included in the
analyses (n=74)
Surveying Interest
Self-report survey included:
•Demographic questions: Age, gender,
ethnicity, education, certification, years in
the field
•Forced choice of preferred Training Topics
(n=26)
Sample Demographics
Overall demographics:
•Age (Mean=42; SD=11)
•Years of Experience (Mean=9; SD=7)
•Gender: Female = 57%
•Ethnicity: Caucasian = 71%, African
American = 22%, Hispanic/Latino = 3%,
Other = 4%
Analyses 1
•Which Topics were most frequently endorsed?
•Was endorsement related to counselor background?
–Certification was defined as having any of the
following certifications:
•Certified Drug/Alcohol Counselor (PCACB, PCB, etc)
•ASAM Certification or APA Qualified in Addiction
Medicine/Psychiatry
•Licensed Health Professional
•Ordained Clergy
–Experience was defined as having been in the field
for at least 6 years (Range: 0-35 yrs)
Analyses 2
Years of Experience and Certification were examined,
creating 4 groups:
Uncertified
Certified
Inexperienced
(n=34, 17%)
(n=51, 26%)
Experienced
(n=61, 31%)
(n=24, 12%)
Note: Participants without both Experience and Certification data were
omitted from analysis (n=25; 14%)
Results 1
Percent Endorsed: Training Topic
62
49
47
47
45
27
25
25
21
19
18
18
-Increasing Client Motivation
-Relapse Prevention Techniques
-Co-Occurring Disorders
-Group Therapy Techniques
-Spirituality & Recovery
-12-Step Oriented Approaches
-Ethnicity & Diversity
-Biological Basis of Addiction
-Addictions Medications
-Sexual Addiction
-Gender Specific Treatment
-Alternative Therapies
Most frequently endorsed
Research-related
Topics analyzed
17
17
15
15
14
14
12
11
11
11
9
5
4
-Obtaining Project Funding
-Finding Treatment Information
-Harm Reduction Strategies
-Applying Research in Practice
-Age Specific Treatment
-Sexuality Related Treatment
-Documentation Software
-Tobacco Addiction
-Use of Incentives & Rewards
-Internet Addiction Resources
-Research & Evaluation Methods
-Using Research Databases
-Statistics & Research Findings
Results 2
Training Topic Preferences by Experience and
Certification:
% Inexperienced
& Uncertified
(n=34)
% Inexperienced
& Certified
(n=51)
% Experienced
& Uncertified
(n=61)
% Experienced
& Certified
(n=24)
Increasing Client
Motivation
68
65
57
58
Relapse
Prevention
56
59
36
50
Co-Occurring
Disorders 1
44
31
56
58
Group Therapy
Techniques 3
59
49
33
58
Note: 1 = Main Effect (ME) for Experience, 2 = ME for Certification, 3 = Interaction, at p < .05
Note: Participants can choose more than one category
Binary Logistic Regression
Results 3
Training Topic Preferences by Experience and
Certification:
% Inexperienced
& Uncertified
(n=34)
% Inexperienced
& Certified
(n=51)
% Experienced
& Uncertified
(n=61)
% Experienced
& Certified
(n=24)
Spirituality &
Recovery
32
49
48
50
12-Step Oriented
Approaches 1
27
39
20
17
Addiction
Medications
35
18
23
4
21
10
13
25
1,2
Applying
Research in
Clinical Practice
Note: 1 = Main Effect (ME) for Experience, 2 = ME for Certification, 3 = Interaction, at p < .05
Note: Participants can choose more than one category
Binary Logistic Regression
Results 4
•Out of the 26 possible Training Topics, the
majority of counselors chose the same 5,
making up 45% of the responses
•These “top 5” topics were most related to
the treatment counselors were already
providing
Survey: “Staff Beliefs about Addiction
Treatment and Clinical Trials”
NIDA Clinical Trials Network (CTN) – DE
Valley
22-items assessing 317 counselors’:
• Beliefs about addiction treatment
• Willingness to try new treatment approaches
– Endorsed by 80% of surveyed counselors
– Willingness to try research-based
innovations
– Endorsed by 82% of surveyed counselors
Survey: Videotape Use in Addiction
Treatment Settings
129 counselors were asked about their
clinic’s videotape usage for group sessions:
• 100% used videotapes with patients
• 97% agreed videotapes were “a useful aid
in educating patients about addiction and
recovery”
Survey: Videotape Use in Addiction
Treatment Settings
• 96% wanted “a library of up-to-date,
scientifically accurate videotapes...”
• 63% agreed only showing “brief sections
of videotapes with a discussion following
them” was best
• 58% were dissatisfied with their clinic’s
current education tools
Toolkit Development
• We incorporated provider feedback into
the development of a single “Toolkit”
• Goal was to:
– assess provider satisfaction
– assess client satisfaction
– assess continued provider use of the Toolkit
Developing the Toolkit
Step 1 – Choosing Content
Examples:
Relapse prevention
HIV risk reduction
Biology of Addiction
Medications
Motivational Interviewing
Toolkit Prototype – MI Component Decisional Balance
Developing the Toolkit
Step 2: Designing the Toolkit
• Multiple formats (Digital & Hand-Outs)
• Created Toolkit prototype
Components:
“Toolbox”
Clinician Guide
DVD
Worksheet
Wallet Cards
Toolkit Components
Worksheet
D
V
D
Wallet
Card
Clinician’s Guide
Toolkit Video
Good Things and
Not So Good Things about Recovery
Methods - Counselors
Counselors
- 26 Counselors from 6 clinics
- Experienced Counselors (> 10 years)
Procedures
- Counselors consented & oriented
- Use the Toolkit in Group & Complete Clinician survey
- One month later - Asked to use 2nd time
- Next 3 months: Record use of Toolkit components
Methods - Patients
Group Members/Patients
N=230
Procedures
- Consented prior to group
- Attended Toolkit group
- Completed Patient
Satisfaction Survey
Research Questions
• Will clinicians be able to use the
Toolkit after only a brief orientation?
• Will clinicians continue to use it, even
when not required?
• How will patients evaluate the toolkit
group?
Will Clinicians Be Able To Use It After
Only a Brief Orientation?
Mean
(SD)
Was the Toolkit orientation adequate?
4.5
0.8
Were you satisfied with the Toolkit group?
4.2
0.7
Was the Toolkit "User-Friendly?"
Do you think other Clinicians would like to use
the Toolkit?
4.4
0.7
4.4
0.7
Scale:
1=Not at all, 2=Somewhat, 3=Moderately, 4=Quite a Bit,
5=Very Much
N=26 (counselors)
Counselor Ratings
14
12
10
8
6
Counselor
Satisfaction
It's user friendly
Recommend it to
other clinicians
4
2
0
Not at all
Somewhat
Moderately
Quite a bit
Very Much
Patient Satisfaction
100
80
60
Was this session helpful?
Did you benefit from
attending the group?
40
20
0
Not At All
A Little
Moderately
Quite a Bit
Very Much
Will Clinicians Continue To Use It After
the Study Ends?
• 96% of counselors have reused at least one
component of the Toolkit
• On at least one occasion, 63% of clinicians
used all 5 components of the Toolkit as
directed
• The most frequently and broadly used were
the core components:
• Counselor Guide & Patient Worksheet
Will Clinicians Continue To Use It After
the Study Ends?
Over a 3-month tracking period:
• Counselor Guide: 83% used it again
(M = 2.5 times)
• Patient Worksheet: 96% used it again
(M = 3.2 times)
Times Used Toolkit Components
Toolkit
component
ClipWorkWallet
DVD
Guide
Cards
board
sheet
Mean (sd)
5 (9) 1 (1)
3 (3)
3 (3)
2 (2)
Range
1 - 42 1 - 3 1 - 12 1 - 12
1-9
Conclusions
Preliminary results in
community-based settings
support the Toolkit’s:
• feasibility
• acceptability
• sustainability
Future Directions
• Will curriculum impact
clinician behavior in treatment?
• Will curriculum beneficially
impact client outcomes?
Translating a Complete Treatment
• We will translate a full treatment
intervention into a Toolkit format
• Starting with 6 Toolkits on Relapse
Prevention Techniques
– Content and didactic heavy
– Popularly selected by providers
– Evidence Based
– Useful in group treatment
Testing a Complete Curriculum
In a larger sample, we will randomly
assign sites to receive:
6 training manuals
OR
6 training manuals +
6 Toolkits Curriculum Supplements
Future Directions, Cont’d
• Toolkits for other evidence-based treatments
(12-step, etc)
• Toolkits to address comorbidity, HIV
prevention, etc
• Arming clinicians with enough evidencebased content to fill 50-75% of group
sessions
Future Directions
Your
Thoughts?