Transcript The Mind & Body Center An Overview Todd Solomon BS

Patient Feedback:
A Quality Improvement Study In Outpatient Settings
2
Forman
1
Rotrosen
2
Crits-Christoph
Agatha Kulaga M.S.W., Robert
Ph.D., John
M.D., Paul
Ph.D., Matt
1NYU School of Medicine and VA NYHHS, 2University of Pennsylvania
1
Clinical Trials Network
Overview of PF Survey (cont’d)
The Clinical Trials Network (CTN) is a national partnership between researchers and community treatment
providers. The mission is to conduct effectiveness research in order to determine if interventions
developed and studied in research clinics work when tested in real world settings. Research results are
disseminated to providers and their patients within the community with the goal of implementing such new
treatments as part of usual care. Current and past CTN protocols address substance use disorders,
infections and addiction, smoking cessation, issues related to women and trauma, adolescent substance
abuse, and HIV/AIDS.
Item Scale – The first 7 PF Survey items use a five point Likert-like scale:
The New York Node
The New York Node, a consortium of academic and clinical investigators and providers, has contributed to
the CTN mission by collaborating in research, training, clinical service delivery, policy making, and funding
for addictive disorders. Participating Community Based Treatment Programs (CTPs) include a wide range
of programs and treatment models, and provide services for, and access to, a diverse clinical population.
Performance Improvement
Patient Feedback (PF) is a web based quality improvement (QI) system designed to monitor patient
ratings of therapeutic alliance and other quality indicators in order to empower clinical staff. Improvement
in therapeutic alliance has been shown to be associated with improvement in a variety of clinically
important outcomes including attendance, retention, and abstinence. This feasibility study was
designed and implemented at 6 CTN clinics beginning in April 2004:
New York:
Lower East Side Service Center &
Addiction Research Treatment Corporation
Chelsea Arbor Addiction Treatment Center
Adcare Outpatient Treatment Center
SouthLight, Inc.
Dorchester Alcohol and Drug Commission
Great Lakes:
New England:
North Carolina:
South Carolina:
B.A.
Therapeutic Alliance and Length of Stay
Not at all - A little Bit – Moderately - Quite a bit - Very much so
Patient Demographic Items (Survey Items 8-10)
Item 8. Do you consider yourself (please select only one): White; African American/Black; American
Indian or Alaska Native; Hispanic or Latino; Asian; Native Hawaiian or Pacific Islander
Item 9. Are You: Male; Female
Item 10. Concerning this admission, about how long have you been in treatment? Less than 1 week;
1 – 4 weeks; 1 - 3 months; More than 3 months
Self-reported Substance Use (Survey Items 11-12)
Item 11. How many days in the PAST WEEK did you drink any alcohol (beer, wine, liquor)?
Item 12. How many days in the PAST WEEK did you use any drugs (marijuana, cocaine, heroin, other)?
Unlike items 1-9, responses to items 10-12 are not be fed back to supervisors or clinicians; instead they
are used in the outcomes evaluation.
Implementation of Patient Feedback
Attendance Data
In addition to the data obtained from the PF Surveys, the second data source for PF is attendance data
extracted from the clinic’s administrative record. Every other week, the administrative assistant extracts
attendance data from the clinic’s records and enters those data onto the Attendance Form. The PF data
system converts data from the Attendance Form into attendance rates using the following calculation:
Attendance=
Total # of pt. sessions attended by the clinician caseload
Total # of sessions scheduled
A total of 41 clinicians and 9 supervisors participated in the Patient Feedback study.
Mean age of clinicians was n=47.
46% of clinicians were female.
71% of clinicians were Caucasian
Level of education: 7 Bachelor
31 Master
2 Doctorate
1 MD/DO
Implementation Results
The Patient Feedback System
1. Every other week, at the close of group, clients are invited to complete a 12-item self-administered
confidential PF Survey in which they rate therapeutic alliance and their satisfaction with the group.
2. The surveys are collected and faxed by an administrative assistant to the University of Pennsylvania,
where they are converted into feedback reports.
3. Confidential Clinician and Clinic PF Reports are posted to a password protected website within about
an hour of being faxed.
4. Individual clinicians access Feedback Reports for their own caseloads, as well as the aggregated data
from the clinic, from the PF website. Clinic supervisors access Feedback Reports summarizing data from
the combined clinic caseload.
5. On a monthly basis, the clinic supervisors and clinicians meet as a team to discuss the Feedback
Reports, identify quality indicators they would like to improve, and discuss the improvement strategies
they plan to implement.
6. A monthly newsletter publicizes clinic successes.
7. The PF cycle is repeated every other week.
2
Worley
Feedback Reports
100% of supervisors conducted monthly Team Meetings, and faxed the Team Meeting form. 86% of
the eligible clinicians participated in the monthly team meetings.
A total of 2,814 PF Surveys were collected and faxed by the six participating clinics, for an overall
collection rate of 75.5% of the clinic census. 564 Feedback Reports were posted to the PF website.
100% PF Reports were posted within one-week of faxing, and 83% were posted within 48 hours.
Therapeutic
Alliance
Ethnicity
All Current
Data
Treatment
Satisfaction
100% of the clinic supervisors downloaded their Clinic Feedback Report at least monthly.
100% of clinicians downloaded at least one Caseload Report, and a total of 112 Caseload Reports, or
2.7 reports per clinician, were downloaded during the feasibility trial.
100% of the Attendance Forms were faxed on schedule by the participating clinics.
Gender
The PF Website operated continuously throughout the feasibility study, with a total of nearly 5,000 web
views by the 50 participating staff members.
The PF News was published on a monthly basis with a circulation of nearly 300 recipients.
Length of Stay
Abstinence
Item Ratings
Attendance
Patient Feedback
System
www.patient-feedback.org
Clinic and Caseload Feedback Reports
The Feedback Reports are provided to study participants (clinicians) as seven graphs and two data tables.
Individual clinicians can access both the Clinic and Caseload Reports; supervisors can only access the
Clinic Report. The seven time-series graphs reflect:
a) Therapeutic Alliance and Length of Stay
b) Therapeutic Alliance and Ethnicity
c) Therapeutic Alliance and Gender
d) Treatment Satisfaction and Length of Stay
e) Treatment Satisfaction and Ethnicity
f) Treatment Satisfaction and Gender
g) Attendance.
During the one year sustainability phase (post-intervention), in which clinics were allowed but not
obligated to use the Patient Feedback system as often as they wished, three clinics continued to use
PF at the same rate (every other week) as during the intervention phase; two clinics faxed PF surveys
on about a monthly basis; one clinic chose to use PF on a semi-annual basis.
Patient Feedback Results and Study Limitations
Overall, patients reported clinicians were performing very well, with relatively low average levels of
patient drug and alcohol use. Similarly, alliance and treatment satisfaction ratings were high across all
assessments. Only one clinic showed relatively higher levels of drug use at the start of the intervention
phase, although there is evidence of some improvements in drug use outcomes over time. There was
also evidence of improvements in average patient therapeutic alliance scores over time during the
intervention phase for those clinicians who had initially poor treatment satisfaction ratings.
This study demonstrated that the implementation of a semi-automatic quality improvement system for
clinicians in addiction treatment facilities was feasible from both a research and clinical perspective.
The limitations of this study include a) there was no control condition; b) the study sample was small;
and c) the PF intervention was employed for only 3 months.
The Future of Quality Improvement
Overview of PF Survey
Therapeutic Alliance Items (Survey Items 1-4)
Item 1. Did you feel accepted and respected by your clinician?
Item 2. Did you feel that you and your clinician were working together to overcome your problems?
Item 3. Did you feel that your clinician understood what you hoped to get out of your treatment?
Item 4. Did you feel confident that through your own efforts and those of your clinician you will gain relief
from your problems?
Treatment Satisfaction Items (Survey Items 5-7)
Item 5. Did you feel comfortable raising issues or concerns?
Item 6. Were things explained to you in a way you could understand?
Item 7. Was the session helpful?
Beneath each graph, a data table for each graph is appended. Finally, one data table presents the
combined data for all of the graphs in the current Report, and a second table presents “Frequency Data”
for each of the first seven items for each Survey distribution.
Sample PF Feedback Report
The following is an example of one of the seven graphs included in each PF Report. The graph shows
feedback from a clinic where the percent of clients giving the highest rating on therapeutic alliance (“very
much so”) increased from 67% in May 1, 2004 to 84% in July 10, 2004. This graph presents ratings of
therapeutic alliance based on how long clients have been in treatment. Within this specific clinic there
were few differences between the ratings from new patients (<1 week) compared to others who had been
in treatment for longer periods (>3 months). Other graphs present ratings of therapeutic alliance or
treatment satisfaction based on gender and ethnicity. Training was provided on how to quickly “read” all
Patient Feedback Reports.
Quality Improvement (QI) is a core component of healthcare delivery systems throughout the US, yet
we are unaware of any published trials testing the effectiveness of QI in addiction treatment settings.
QI is viewed as one potential bridge for introducing evidence-based practices into community-based
treatment programs. In the original Patient Feedback feasibility study presented, the feasibility and
acceptability of PF was established. A larger scale Patient Feedback effectiveness trial has been
funded by the National Institute on Drug Abuse and will begin October 2006. This trial is being
conducted by the University of Pennsylvania and NYU School of Medicine, with up to 32 clinics
participating. It is expected that this clinical trial will provide valuable information about the PF
intervention, as well as QI interventions, in substance use treatment settings. Additional knowledge will
be gained about the relationship between patient, clinic, clinician, and supervisor characteristics and
improvements in patient attendance and abstinence in outpatient settings.
Acknowledgements
Lead Investigator: Robert Forman, Ph.D.
National Lead Node: University of Pennsylvania, Treatment Research Institute
Supported and developed by the National Institute on Drug Abuse Clinical Trials Network