Transcript Therapists effects - A Collaborative Outcomes Resource Network

Implications of Therapist Effects
for Employers and Health Plans
American Psychological Association Convention
San Francisco, August, 2007
.
G.S. (Jeb) Brown, Ph.D.
Center for Clinical Informatics
Therapists effects
• Wampold and others argue that researchers have
ignored the individual therapist as a source of
variance.11, 16-24
• The person of the therapist is necessary to delivery
the treatment, and personal characteristics of the
therapist modify the effect of the treatment.
• Factors contributing to therapists effects may
include elements clinical skill and knowledge as
well as personality traits.
Variance due to the clinician
• Published research making use of HLM points to
the conclusion that the clinician accounts for much
more of the variance in psychotherapy outcomes
that treatment method per se. 11, 17-21
• Analyses of PacifiCare Behavioral Health’s massive
database database on patient outcomes confirms
significant variance in psychotherapy outcomes at
the clinician level. 24,25
PBH research collaboration
• PBH actively sought the involvement of leading
psychotherapy outcomes researchers from leading
academic institutions.
• External researchers actively involved in design of
the measurement system and ongoing analysis of
the data.
• PBH encouraged publication of findings in
academic journals.
The (almost) Bell Curve
PBH data
Solo clinicians with sample sizes => 20
% of clinicians
25%
20%
15%
10%
5%
0%
-0.2 -0.1 0
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Effect Size
Therapists and meds
Outcomes (residualized scores) of 15 therapists for
patients with concurrent medication or no medication 25
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
meds
nomeds
Cross validation analysis
• Psychotherapists in PBH network ranked based on
all cases from 1999-2002 if sample size =>30;
N=116.
• If a therapist’s mean residualized final score < 0
then clinician rated “Highly effective”; else
clinician rated “Less effective”.
• Outcomes evaluated in the 2003-2004 cross
validation period for a new sample of cases.
Cross validation results
Clinician cross validation results - 2003 to 2004
Therapists assessed on at least 30 cases between 1999-2002.
Highly effective clinicians had mean Change Index Score > 0
Effect size
1
Highly effective clinicianspsychotherapy only
0.9
0.8
0.7
0.6
Highly effective clinicianspsychotherapy and medication
0.5
0.4
Less effective clinicianspsychotherapy only
0.3
0.2
0.1
0
Less effective clinicianspsychotherapy and medication
0-50
51-120
Intake scores (mean split)
Psychiatrist effects
• Wampold and colleagues also used HLM to
reanalyze the results antidepressant and placebo
legs of the TDCRP study. 28
• Included the 9 individual psychiatrists as a variable.
• Outcome measured by change on patient self report
measure (Beck Depression Inventory).
• 9.1% of the variance due to the psychiatrist; only
3.4% due to the medication.
• Top third of psychiatrists achieved a better outcome
with placebo than bottom third achieved with the
antidepressant.
Honor for Outcomes
• Honors for Outcomes Selection Criteria:
– Minimum of 10 cases with two Y/LSQ data points in past
3 years
– Average patient change must be reliably above average:
65% confidence that the provider’s Change Index >0
– Change Index is a case-mix adjusted measure, compares
outcomes to PBH’s large normative database
• Honors for Outcomes is updated quarterly
Website
Outcomes and cost
$600
$500
Average cost per episode
$400
$300
$200
$100
$0
No outcome
data for
provider
Honors:
Groups
Honors:Solo
clinicians
Non-Honors:
Groups
Non-Honors:
Solo
Value Index
•Value Index = Average effect size per $1000
expenditure (Effect Size/Cost of Care) x $1000
2
Honors: Groups
Value Index
1.5
Honors:Solo clinicians
Non-Honors: Groups
Non-Honors: Solo
1
0.5
0
Honors: Groups
Honors:Solo
clinicians
Non-Honors:
Groups
Non-Honors:
Solo
Implications for clinicians
• Good news: The clinician matters!!!!!!
• All treatments (including medications!?) are only as
effective as the clinicians delivering the treatment.
• Clinicians have an ethical responsibility to assess
and improve their personal effectiveness as
clinicians… they cannot rely on the treatments alone
to be curative.
• Effective clinicians deliver high value services and
are worth more money!!!
Implications for employers
and health plans
• Strong business case can be made for identifying
“high value outcomes informed clinicians”
• Steering referrals to highly effective clinicians results
in better treatment outcomes without increasing cost.
• Use of highly effective clinicians leads to greater
“return on investment” for spending on behavioral
healthcare services

Workplace productivity gains

Reduce medical costs
Barriers to identifying
effective clinicians
• Clinician resistance to use of outcome questionnaires

“The questionnaires don’t give me any information I don’t already know.”

“I don’t believe the questionnaires provides a valid measure of outcome.”
• Over reliance on “evidence based treatments” and
medications
“I know I get good outcomes because I am an expert in how to provide
__________ treatment.”

• Active resistance from professional guild organizations
• Inability of a health plan to collect data on more than a
small percentage of a clinician’s case load
Winners and losers?
• Four major categories of “stake holders”

Consumers/Patients

Highly effective clinicians

Ineffective clinicians

Payers: Employers and health plans
• Who benefits if outcomes informed care become
widespread?
• Who benefits if professional guilds and practitioners
continue to resist outcomes informed care?
Something different….
• Major employer in the Pacific Northwest has partnered
with Regence to implement an outcomes informed care
pilot project
• Regence project seeks to identify and promote
“outcomes informed clinicians”
• Clinicians are encouraged to use questionnaires with all
patients and are given access to all of their outcome data
in near real time.
• Clinician participation is entirely voluntary and at no
cost to the clinician.
A Collaborative Outcomes Resource Network
• ACORN Organization Inc. is a non-profit organization
devoted to furthering the science and practice of
outcomes informed care.
• ACORN maintains a TWiki which which provides
information on freely available questionnaires and
methods for benchmarking outcomes.
• ACORN site utilized by a number of organizations to
support outcomes informed care initiatives.
• www.psychoutcomes.org
References & suggested readings
1.
2.
3.
4.
5.
Kirsch, I & Sapirstein, G. 1998. Listening to Prozac but hearing
placebo: A meta analysis of antidepressant medication. Prevention &
Treatment. 1, Article 0002a, No Pagination Specified
Kirsch, I. 2000. Are drug and placebo effects in depression additive?
Biological Psychiatry 47, 733-73.
Kirsch, I, Moore, TJ, Scoboria, A, Nicholls, SS. 2002. The emperor's
new drugs: An analysis of antidepressant medication data submitted
to the U.S. Food and Drug Administration. Prevention & Treatment.
5(1), No Pagination Specified
Rosenzweig S. 1936. Some implicit common factors in diverse
methods of psychotherapy: “At last the Dodo said, ‘Everybody has
won and all must have prizes.’” Am J Orthopsychiatry 6:412-5.
Shapiro DA & Shapiro D. 1982. Meta-analysis of comparative
therapy outcome studies: A replication and refinement. Psychol Bull
92:581-604.
References & suggested readings
6.
Robinson LA, Berman JS, Neimeyer RA. 1990. Psychotherapy for
treatment of depression: A comprehensive review of controlled
outcome research. Psychol Bull 108:30-49.
7. Wampold BE, Mondin GW, Moody M, et al. 1997. A meta-analysis
of outcome studies comparing bona fide psychotherapies:
Empirically, “All must have prizes.” Psychol Bull 122:203-15.
8. Ahn H, Wampold BE. 2001. Where oh where are the specific
ingredients? A meta-analysis of component studies in counseling and
psychotherapy. J Counsel Psychol 48:251-7.
9. Chambless DL, Ollendick TH. 2001. Empirically supported
psychological interventions: Controversies and evidence. Annual Rev
Psychol 52:685-716.
10. Luborsky, L., Rosenthal, R., Diguer, L., et al. 2002. The dodo bird
verdict is alive and well--mostly. J. Psychotherapy Integration Vol
12(1) 32-57
References & suggested readings
11. Wampold BE. 2001. The great psychotherapy debate: Models,
Methods, and Findings. Mahwah NJ: Lawrence Erlbaum Associates.
Wampold BE, Mondin GW, Moody M, et al. 1997. A meta-analysis
of outcome studies comparing bona fide psychotherapies:
Empirically, “All must have prizes.” Psychol Bull 122:203-15.
12. McCall, WA. 1923 How to experiment in education. New York:
McmIllan.
13. Fisher, RA. 1935 The design of experiments. Edinburgh: Oliver and
Boyd.
14. Gehan, E. & Lemark, NA. 1994. Statistics in medical research:
Developments in clinical trials. New York: Plenum Press.
15. Martindale C. 1978. The therapist-as-fixed-effect fallacy in
psychotherapy research. J Consult Clin Psychol 46:1526-30.
References (continued)
16. Luborsky L, Crits-Christoph P, McLellan T, et al. 1986. Do therapists
vary much in their success? Findings from four outcome studies. Am
J Orthopsychiatry 56:501-12.
17. Crits-Christoph P, Baranackie K, Kurcias JS, et al. 1991. Metaanalysis of therapist effects in psychotherapy outcome studies.
Psychother Res 1:81-91.
18. Crits-Christoph P, Mintz J. 1991. Implications of therapist effects for
the design and analysis of comparative studies of psychotherapies. J
Consul Clin Psychol 59:20-6.
19. Wampold BE. 1997. Methodological problems in identifying
efficacious psychotherapies. Psychother Res 7:21-43,
20. Elkin I. 1999. A major dilemma in psychotherapy outcome research:
Disentangling therapists from therapies. Clin Psychol Sci Prac 6:1032.
References (continued)
21. Wampold BE, Serlin RC. 2000. The consequences of ignoring a
nested factor on measures of effect size in analysis of variance
designs. Psychol Methods 4:425-33.
22. Huppert JD, Bufka LF, Barlow DH, et al. 2001. Therapists, therapist
variables, and cognitive-behavioral therapy outcomes in a
multicenter trial for panic disorder. J Consul Clin Psychol 69:747-55.
23. Okiishi J, Lambert MJ, Nielsen SL, et al. 2003. Waiting for
supershrink: An empirical analysis of therapist effects. Clin Psychol
Psychother 10:361-73.
24. Brown GS, Jones ER, Lambert MJ, et al. 2005. Identifying highly
effective psychotherapists in a managed care environment. Am J
Managed Care 11(8):513-20.
25. Wampold BE, Brown GS. 2005. Estimating variability in outcomes
due to the therapist: A naturalistic study of outcomes in managed
care. J Consul Clin Psychol. 73(5): 914-923.
References (continued)
26. Elkin, I, Shae, T, Watkins, JT., et al. 1989. National Institute of Mental
Health Treatment of Depression Collaborative Research Program:
General effectiveness of treatments. Archive of General Psychiatry. 46:
971-982.
27. Kim DM, Wampold BE, Bolt DM. 2006. Therapist effects and
treatment effects in psychotherapy: Analysis of the National Institute of
Mental Health Treatment of Depression Collaborative Research
Program. Psychother Res. 16(2): 161-172.
28. McKay, KM, Imel, ZE & Wampold, BE. 2006. Psychiatrist effects in
the pharmacological treatment of depression. J. Affective Disorders.
92(2-3): 287-290.
29. Hannan C, Lambert MJ, Harmon C et al. 2005. A lab test and
algorithms for identifying clients at risk for treatment failure. J Clin
Psychol 61(2):155-63.
30. Lambert MJ, Harmon C, Slade K et al. 2005. Providing feedback to
psychotherapists on their patients progress: Clinical results and practice
suggestions J Clin Psychol 61(2):165-74.
References (continued)
31. Harmon C, Hawkins, Lambert MJ et al. 2005. Improving outcomes for
poorly responding clients: The use of clinical support tools and
feedback to clients. J Clin Psychol 61(2):175-85.
32. Brown GS, Jones DR. 2005. Implementation of a feedback system in a
managed care environment: What are patients teaching us? J Clin
Psychol 61(2):187-98.
33. Claiborn CD, Goodyear EK. 2005. Feedback in psychotherapy. J Clin
Psychol 61(2):209-21.
34. Lueger RJ. 1998. Using feedback on patient progress to predict the
outcome of psychotherapy. J Clin Psychol 54:383-93.
35. Lambert MJ, Whipple JL, Smart DW, et al. 2001. The effects of
providing therapists with feedback on patient progress during
psychotherapy: Are outcomes enhanced? Psychother Res 11(1):49-68.
References (continued)
36. Lambert MJ, Whipple JL, Vermeersch DA, et al. 2002. Enhancing
psychotherapy outcomes via providing feedback on client progress: A
replication. Clin Psychol Psychother 9:91-103.
37. Whipple JL, Lambert MJ, Vermeersch DA, et al. 2003. Improving the
effects of psychotherapy: The use of early identification of treatment
failure and problem-solving strategies in routine practice. J Counsel
Psychol 50(1):59-68.
38. Lambert MJ, Whipple JL, Hawkins EJ, et al. 2003. Is it time for
clinicians to routinely track patient outcome? A meta-analysis. Clin
Psychol Sci Prac 10:288-301.
39. Bachelor, A., & Horvath, A. (1999). The therapeutic relationship. In
M.A. Hubble, B.L. Duncan, and S.D. Miller (eds.). The Heart and Soul
of Change: What Works in Therapy. Washington, D.C.: APA Press,
133-178.
40. Blatt, S. J., Zuroff, D.C., Quinlan, D.M., & Pilkonis, P. (1996).
Interpersonal factors in brief treatment of depression: Further analyses
of the NIMH Treatment of Depression Collaborative Research Program.
J Consul Clin Psychol. 64, 162-171.
References (continued)
41. Bordin, E. S. (1979). The generalizability of the psychoanalytic
concept of the working alliance. Psychotherapy: Theory, Research, and
Practice, 16, 252-260.
42. Burns, D., & Nolen-Hoeksema, S. (1992). Therapeutic empathy and
recovery from depression in cognitive-behavioral therapy: A structural
equation model. J Consul Clin Psychol. 60, 441-449.
43. Connors, GJ, DiClemente, CC., Carroll, KM, et al. 1997 The
therapeutic alliance and its relationship to alcoholism treatment
participation and outcome. J Consul Clin Psychol, 65(4), 588-598.
44. Horvath, A. O., & Symonds, B. D. (1991). Relation between working
alliance and outcome in psychotherapy: A meta-analysis. J Consul
Clin Psychol. 38, 139-149.
45. Krupnick, J., Sotsky, SM, Simmens, S et al. 1996. The role of the
therapeutic alliance in psychotherapy and pharmacotherapy outcome:
Findings in the National Institute of Mental Health Treatment of
Depression Collaborative Research Project. J Consul Clin Psychol. , 64,
532-539.
About the presenter
G.S. (Jeb) Brown is a licensed psychologist with a Ph.D. from Duke
University. He served as the Executive Director of the Center for Family
Development from 1982 to 19987. He then joined United Behavioral
Systems (an United Health Care subsidiary) as the Executive Director for
of Utah, a position he held for almost six years. In 1993 he accepted a
position as the Corporate Clinical Director for Human Affairs
International (HAI), at that time one of the largest managed behavioral
healthcare companies in the country.
In 1998 he left HAI to found the Center for Clinical Informatics, a
consulting firm specializing in helping large organizations implement
outcomes management systems. Current client organizations include
Accountable Behavioral Health Care Alliance, Kaiser Permanente,
Regence, Resources for Living, and United Behavioral Health.
Dr. Brown also works part time as a psychotherapist and he does
measure his outcomes.
http://www.clinical-informatics.com
[email protected]
1821 Meadowmoor Rd.
Salt Lake City, UT 84117
Voice 801-541-9720