Transcript Programme evaluation of an adolescent psycho

Programme evaluation of an
adolescent psycho-educational
group for eating disorders entitled
“New Directions”
Authors: Mary Kaye Lucier MSW, RSW, Executive
Director, Bulimia Anorexia Nervosa Association
(BANA), Rosanne Menna PhD, C Psych. , University of
Windsor; Department of Psychology
Date: May 18, 2007
Executive Summary
•
•
•
•
•
•
•
•
New Directions is a ten week-1.5 hour per session, psycho-educational group
invention programme for adolescents aged 12 – 18 years, who have been diagnosed
with an eating disorder.
The goal of the New Directions programme is to increase insight and awareness of
four related factors contributing to eating disorders: (i) eating disorder symptoms; (ii)
factors contributing to the development of eating disorders; (iii) the physical and
mental effects of eating disorders; and (iv) developing a plan for recovery.
The programme was designed to interrupt symptoms related to eating disorders such
as extreme dieting, body dissatisfaction and unhealthy weight control methods.
Referral into New Directions is based on the results of a specialized structured eating
disorder diagnostic assessment.
Following the specialized assessment, each referral is presented at the weekly
rounds, for a diagnosis.
The diagnosis is based on the DSM IV criteria for eating disorders. Each person is
provided with a treatment plan.
Those with the diagnosis of an eating disorder are invited to attend the New
Directions group.
Following the completion of New Directions, the participants are reassessed and
presented for a re-diagnosis and re-evaluation of the treatment plan.
Evaluation questions and
purpose for the evaluation:
•
•
•
•
•
•
•
•
•
The evaluation questions were:
(1) What are the characteristics of the adolescent clients, aged 12 – 18
years, who present for assessment at a community based treatment centre
for eating disorders?
(2) What are the changes in eating disorder symptoms from the pretreatment assessment to the post treatment assessment at the end of the
New Directions programme?
(3) What is the average length of stay in the programme?
The purpose for the evaluation was to:
To create a snapshot of the participants in the New Directions programme
by analysing archival data collected on participants of the New Directions
programme from September 1, 03 - March 31, 07;
To enhance the agency’s capacity for more complex research on client
outcomes;
To disseminate project findings;
To create an evidence-based programme for adolescents with an eating
disorder that is accessible through a training manual suitable for delivery in
other agencies.
Description of the Methods Used
•
•
•
•
•
•
•
•
•
•
•
The specialized eating disorder assessment consists of a diagnostic interview, followed by the
administration of psychometric tests that measure eating disorder psychopathology (i.e.
symptoms, body dissatisfaction, and size acceptance), related psychological functioning (i.e., selfesteem, depression) and demographic information.
The diagnosis is based on the DSM IV criteria for eating disorders, and is under the supervision of
either a clinical psychologist or physician.
Two assessments were done, each comprising of the above measures. Assessments were done
before the groups began (at initial intake; pre-test) and following completion of the New Directions
programme (post-test). Ninety-eight participants presented for treatment at the agency.
Of those 74 were assessed for the New Directions programme for the period 2003 - 2007.
Twenty-four adolescents completed the programme.
The diagnostic interview and questionnaires were administered on all agency clientele and were
subject to agency policy and procedures.
The data collected for programme evaluation was explained at the time of the initial visit and was
presented on the agency "Consent to Treatment and Research Form".
Subjects had the option to withdraw their consent at any time.
All of the diagnostic assessments were conducted by trained clinicians in the field of eating
disorders.
Policies and procedures exist within the agency that reflect the most current and up to date
legislation on the collection, retrieval and storage of personal and health information.
The data entry personnel were required to sign confidentiality statements, prior to data entry.
Steps Involved in the Research
Project
• 1. Archival data was entered into agency database.
• 2. Data was collected from a specialized diagnostic
assessment interview of eating disorder behaviours, and
self report questionnaire that measure attitudes,
feelings, thoughts and behaviours related to the
symptomatology of eating disorders, and related
psychopathology.
• 3. Participants’ participation in agency research
component for the purposes of data collection and data
analysis, is voluntary.
• 4. Participants were reminded that participation in no
way affects their ability to participate in the programmes
offered at the agency.
Summary of Main Findings
Demographic Description
• The majority of the participants in the New
Directions programme were female, from
divorced or single parent families.
• The average age for the adolescents in the
programme was 15. 3 years (SD= 1.8 years)
years.
• The majority were Caucasian, and all spoke
English.
• As a group, the sample tended to be middle
class.
Changes in Psychological
Functioning
•
•
•
•
•
•
•
The majority of the participants of New Directions reported suicide attempts,
suicide thoughts and plans for suicide at intake.
Although most of the participants of New Directions reported no form of
drug or alcohol use, one third reported using alcohol or cannabis socially, or
referred to themselves as an alcoholic and/or a binge drinker at intake.
Self-harming behaviours, sexual and emotional abuse were indicated by
nearly one-half of the participants at intake as well.
Most participants reported that the New Directions group was the first
attempt at any form of psychiatric treatment.
With the exception of cutting which showed a slight increase in the
behaviour for those who completed the group, there was a general
indication of some change in self-harm behaviours reported.
Self-esteem scores indicated a small increase from pre to post New
Directions.
Depression scores showed little change from pre to post New Directions.
Changes in Eating Disorder
Symptoms
• Overall, the participants who completed New
Directions reported a general reduction in eating
disorder behaviours.
• Feelings and reactions to possible weight gain
did not change from pre to post New Directions.
• At post assessment, those who completed five
or more sessions of the program, reported fewer
binge episodes, vomit days, laxative days, and
diet pill days than those who completed less
than five sessions of New Directions.
• Participants reported less dieting and fewer
attempts to lose weight after the programme
Changes in Diagnosis:
• For the purposes of this study, there were three possible
diagnostic categories:
– Anorexia Nervosa (AN), those with a Body Mass Index (BMI) of
less than 85% of BMI 20, severe dieting behaviours and body
image disturbance;
– Bulimia Nervosa (BN), those who exhibited binge and/or purge
behaviours and body image disturbance;
– and those with not otherwise specified eating disorders
(EDNOS).
• Few were AN (7 percent), one third were BN, one third
were EDNOS, and nearly one quarter could not be
diagnosed due to lack of information or did not have an
eating disorder.
• At post assessment approximately 70 percent of the
categories of diagnoses did not change.
Average length of stay in New
Directions
• Seventy-four adolescents were referred to the New
Directions programme.
• Twenty-eight percent did not begin the programme.
These individuals either lost contact with the agency
(e.g., moved away) or chose not to begin the
programme.
• The completion of the New Directions programme was
determined by two factors:
– those who completed less than five of the New Directions group
– and those who completed five or more of New Directions group.
• Nearly one-half of the adolescents referred, completed
the programme.
Summary of Implications of
Findings
• The drop out rate from eating disorder treatment
programmes is generally 50% for AN, and 30% for BN
(Vandereyeken, & Pierloot, 1983; Mahon, 2000).
• This study indicated an overall drop out rate from New
Directions of 28%.
• Those who completed more than five New Directions,
reported less abuse, alcohol/drug use, and suicidal
thoughts, plans and attempts at pre-treatment
assessment.
• This finding implies that a history of alcohol/drug use,
abuse, and suicidality may be a factor affecting the
completion of New Directions programme.
• The high incidence of reported self-harm,
abuse and suicidality implies that
posttraumatic stress disorder and other
psychiatric conditions may be present and
affect the completion of New Directions
programme.
• The high incidence of reported self harm,
abuse and suicidality, also implies that the
expected increase in self-esteem, lowered
depression scores and reduction in eating
disorder symptoms, were less than
expected.
Summary of Recommendations
• Five recommendations resulted from the project:
• The New Directions programme does show promise.
• The evaluation should include additional screening tools to assist
with a better treatment fit, and a 6-month follow-up.
• The Beck Depression Inventory-II should be included to assess
severity of depression
• The Pre-treatment assessment should include a readiness to
change measurement
• The assessment should screen for posttraumatic stress disorder.
• Develop a pre-contemplation group for those who are not ready to
change and/or are experiencing symptoms of posttraumatic stress.
• The pre-contemplation group would focus on symptom management
and harm reduction.
Next Steps
• Presentations and workshops about the programme and
research findings will be developed for presentation at
agencies across Ontario and Canada.
• An executive summary of recommendations to the
Ontario Network of Eating Disorder Service Providers
regarding treatment outcomes and programme
evaluation will be prepared.
• Study findings will also be available on the BANA
website, University of Windsor REB website, and
Provincial Centre of Excellence for Children and Youth
Mental Health at CHEO website.
• BANA will also deliver presentations at professional
meetings and conventions, and publish findings in
newsletters, and professional journals.
Methodology
Design
• 98 adolescents presented for treatment at the agency.
• Of those 74 were offered the New Directions
programme.
• Twelve percent of the 74 adolescents were referred to
another treatment.
• Twenty-eight percent did not begin the programme.
• These individuals either lost contact with the agency
(e.g., moved away) or chose not to begin the programme
at the initial assessment (pre-test).
• Thirty-eight adolescents began the programme.
• Fourteen participants attended five or less sessions.
• In total 24 adolescents completed the programme.
• Referral into New Directions was based on the results of a
specialized structured eating disorder diagnostic assessment.
• The specialized eating disorder diagnostic assessment consists of a
diagnostic interview, followed by the administration of psychometric
tests that measure both eating disorder psychopathology, and
related psychological functioning.
• Following the specialized structured eating disorder diagnostic
assessment, each referral was presented at the weekly rounds, for a
diagnosis.
• The diagnosis is based on the DSM IV criteria for eating disorders,
and is under the supervision of either the clinical psychologist for
persons aged 16 and older, or the physician for persons under the
age of 16 years.
• Following the diagnosis, each person was provided a treatment
plan. Those with the diagnosis of an eating disorder were invited to
attend the New Directions group.
• Following the completion of New Directions, the participants were
reassessed and presented for a re-diagnosis and re-evaluation of
the treatment plan.
Data Collection
Demographic Questionnaire.
• Adolescents were asked to provide their age, height,
weight, language spoken, ethnicity, and parent’s
occupation.
• Weight and height were used to determine body mass
index (BMI; calculated using the following formula:
weight (kg)/height (m)2).
Diagnostic Interview.
• Adolescents were interviewed about their typical day of
eating, binge, vomiting, fasting episodes, laxative,
diuretics, and diet pill use, exercise, body image, body
satisfaction, abuse history, alcohol and drug use, suicidal
thoughts and behaviour and self-harm behaviour.
Eating Disorders Inventory-2 (EDI:
Garner, 1991).
• The EDI-2 is a 91 item self-report measure of eating
disorder symptoms.
• In completing the measure participants answer a number
of questions about their shape, weight, and eating on a
6-point scale ranging from never to always.
• The EDI Drive for Thinness, Body Dissatisfaction, and
Bulimia subscales were used in this research to describe
the severity of participants’ eating disorder pathology.
• Extensive psychometric support for this instrument and
norms for adolescents are available in the treatment
manual (Garner, 1991).
Child Depression Inventory (CDI;
Kovacs, 1983).
• The CDI is a 27 item self-report measure of
depression.
• Each item consists of three statements graded in
order of increasing severity from 0 to 2.
• For each item participants respond by marking
the sentence that best describes them during the
past 2 weeks.
• The measure has high levels of internal
consistency, re-test reliability and convergent
validity.
Rosenberg Self-Esteem
Questionnaire (Rosenberg; 1979).
• This measure consists of 10-items addressing global
positive or negative attitudes toward the self.
• Items are of the following type; I feel that I have a
number of good qualities.
• All in all, I am inclined to feel that I am a failure.
• Participants rated on a five-point scale (1=not at all
accurate, 5=completely accurate) the extent to which
each statement is felt or experienced.
• A total is computed by summing the ratings on all items.
• The possible scores on self-esteem range from 1 (low
self-esteem) to 50 (high self-esteem).
• The instrument has been used widely and has good
reliability and validity.
Psychometric Testing
•
•
•
•
•
•
•
The forms and questionnaires were administered on all agency clientele
and were subject to agency policy and procedures.
The data collected for programme evaluation was explained at the time of
the initial visit and was presented on the agency "Consent to Treatment and
Research Form".
Opportunities for debriefing occurred at the time the consent was given.
At the time consent was given, participants were provided with copies of the
consent form, and opportunity to ask questions about the research
component of the agency programme.
At the beginning of the specialized diagnostic assessment interview, each
participant was informed that the data collected for their file would be used
for the purposes of programme evaluation.
Each participant was also informed that they have a right to withdraw from
the programme evaluation project at any time.
Each participant was informed that the data collected for the purposes of
programme evaluation would be kept confidential, and would not be used
for other purposes without their prior consent.
• Participants’ participation in the agency research
component for the purposes of data collection
and data analysis, is voluntary.
• Participants were reminded that participation in
no way affects their ability to participate in the
programming offered at the agency.
• Each participant who agreed to participate in the
programme evaluation, was provided with a
research data entry number, and thereby, the
identifying information in the database is
unknown to the researcher.
Sources of information and data
report
•
•
•
•
•
•
•
•
•
•
Adolescents completed self-report questionnaires and structured interviews.
The questionnaires and diagnostic interviews were administered by trained qualified clinicians in
the field of eating disorders.
Each therapist has access to the hard copy file of the adolescents’ interviewed.
Each clinician is required to sign a confidentiality agreement, as a condition of employment.
Agency policies and procedures are in place, which reflect the Personal Health Information
Privacy and Protection Act (2005) to safeguard the collection, storage, and disposal of the data
that is being collected.
Undergraduate and graduate students in social work and psychology programmes entered data.
The data entry personnel were required to sign a confidentiality agreement and an offer to hire
stating that they will abide by agency policy and procedures relating to data collection and
storage, as a condition of employment and prior to entering data.
All hard copy data is stored in a locked file cabinet. All data is stored on the agency data base,
protected by password.
The principal investigator has access to the hard copy and the research database for the
purposes of checking the data for accuracy.
The co-investigator also has access to the hard copy of the data collected and the research
database for the purposes of checking accuracy.
Evaluation limitations
•
•
•
•
•
•
The sample size was small, making the findings and implications for the
drop and completion rates from New Directions difficult to interpret.
Generally the drop out rate for eating disorder treatment programmes is
50% for AN, and 30% for BN. Twenty-eight percent of the adolescents
participated in the pre-assessment but did not attend programme.
Forty-one percent did not complete the program (attended five or less
sessions).
The research on treatment outcomes for eating disorders indicates that
substance abuse, sexual abuse, psychiatric co-morbidities and suicidality
negatively affect outcome.
This evaluation project revealed that those who completed more than five
sessions of New Directions, reported less abuse, alcohol/drug use, and
suicidal thoughts, plans and attempts at pre- treatment assessment.
The evaluation did not include a follow-up assessment component (e.g. 6month post-testing), therefore any positive or negative changes over time
are unknown.
Results
• Research question #1: What are the characteristics of the
adolescent clients, aged 12 – 18 years, who present for
assessment at a community based treatment centre for eating
disorders?
• The adolescent clients presenting for assessment at BANA reported
typical patterns of eating and compensatory behaviours to persons
with eating disorders.
• At intake, the participants mean height and weight was 5’3”
(SD=.39), and 132 lb. (SD=52.33). The mean body mass index
(BMI) was 24.1 (SD=7.76).
• There were four main classifications of diagnoses: Anorexia
Nervosa (AN), Bulimia Nervosa (BN), Binge Eating Disorder (BED),
and Eating Disorder Not otherwise Specified (EDNOS).
• Approximately 7% were diagnosed with AN; 30% with BN; 30% with
EDNOS, and 9% diagnosed with BED.
• The majority of the clients were female (94
percent).
• Fifty-seven percent were from divorced or single
parent families.
• The average age for the adolescents was 15. 3
years (SD= 1.8 years).
• The majority of the adolescents were Caucasian
(97%) and the remaining adolescents were
aboriginal.
• All participants spoke English.
• As a group, the sample tended to be middle
class.
Table 1: Height, weight, and Body
Mass Index of population
N
Mean
Std. Dev.
Range
Height at
Intake (feet
and inches)
92
5.3
.39
4.05-5.98
Weight at
Intake
(pounds)
93
132
52.33
45-368
Body Mass
Index
86
24.14
7.76
14.6-50
Suicide
• History of suicide attempts, current and
past thoughts or plans for suicide were
reported for many (58%) of the
participants.
Table 2 illustrates a description of the suicidality for
participants of New Directions at intake.
No Suicide
History
Current
Suicidal
Thoughts
Current
Past Suicide
Suicidal Plan Thoughts
Past Suicide
Attempt
42%
17%
3%
11%
27%
Alcohol/Drug Use
• The majority of the participants indicated no form of drug
use, (80%), or alcohol use (73%).
• Of those who reported using drugs, 15% indicated
cannabis.
• Of those who reported using alcohol, 18% indicated
being a social drinker, and 6% referred to themselves as
an alcoholic and/or binge drinker.
• Although the majority of the participants of New
Directions reported no form of drug or alcohol use, one
third reported using alcohol or cannabis socially, or
referred to themselves as an alcoholic and/or a binge
drinker at intake.
Self-Harm
• Approximately sixty-four percent of the
participants reported no self-harm behaviours,
and the remaining one third of the participants
(36%) reported some form of self-harming
behaviours.
• Most participants reported that the New
Directions group was the first attempt at any
form of psychiatric treatment.
• Most participants reported no previous
psychiatric treatment. Less than 1% had
previous inpatient psychiatric treatment and 2%
reported outpatient treatment.
Abuse
• The majority reported no form of abuse
(60%).
• Abuse was indicated by nearly forty
percent of the participants at intake.
• Of those reporting abuse the following
table indicates the type of abuse reported.
Table 3: Reported abuse at
intake
Physical
Sexual
Emotional
Physical & Physical & Emotional
Emotional Sexual
& Sexual
11%
26%
43%
14%
3%
3%
Research question #2:
• Research question #2: What are the
significant changes in eating disorder
symptoms from the pre-treatment
assessment to the post treatment
assessment at the end of psychoeducation?
Changes in Eating Disorder
Symptoms:
• Paired-sample t tests were conducted to
evaluate the effects of the group on eating
disorder symptoms.
• Overall, the findings were not significant.
• The pre and post mean scores indicate a
general reduction in eating disorder
behaviours.
Table 4 shows the means
scores pre and post the New
Directions programme.
• Table 4: Change in eating disorder
symptoms: pre - post New Directions
Variable/Behaviour
Pre
Mean
Post
SD
n
Mean
t-test
SD
n
Eating Disorder Symptoms
Dieting days in past 28 days
21.28
12.20
25
16.80
14.00
25
t(24) = 1.69, p = 0.10
Dieting days in past 3 months
67.25
37.92
24
57.50
42.35
24
t(23) = 1.35, p = 0.19
Binge episodes in past 28 days
6.54
11.10
24
4.58
6.68
24
t(23) = 1.19, p = 0.25
Binge episodes in past 3 months
25.21
42.73
24
15.83
21.01
24
t(23) = 1.12, p = 0.27
Binge days in past 28 days
7.33
10.57
21
4.10
5.91
21
t(20) = 1.80, p = 0.09
Binge days in past 3 months
22.19
28.29
21
14.48
17.90
21
t(20) = 1.38, p = 0.18
Vomit days in past 28 days
9.12
10.87
25
7.84
9.78
25
t(24) = 0.64, p = 0.53
Vomit days in past 3 months
30.64
40.29
25
32.08
43.11
25
t(24) = -0.20, p = 0.84
Fasting days in past 28 days
8.48
10.42
25
7.52
9.39
25
t(24) = 0.52, p = 0.61
Fasting days in past 3 months
26.52
32.46
25
27.482
28.45
25
t(24) = -0.20, p = 0.84
Laxative days in past 28 days
0.04
0.20
25
0.00
0.00
25
t(24) = 1.00, p = 0.33
Laxative days in past 3 months
0.24
0.83
25
0.12
0.44
25
t(24) = 0.62, p = 0.54
Exercise days in past 28 days
10.08
10.98
25
9.08
10.27
25
t(24) = 0.41, p = 0.69
Exercise days in past 3 months
28.64
32.50
25
23.24
29.75
25
t(24) = 0.91, p = 0.37
Changes in Diagnosis
• Table 5 reports the frequencies and
percentage for categories of diagnosis pre
and post programme.
• From pre to post New Directions, there
were no changes in seventy percent of the
clients’ categories of diagnoses.
Table 5: Change in client diagnoses
from pre to post New Directions
Presenting Problem
N
% of Total
No change
14
70.00%
BED to EDNOS BN-P
1
5.00%
From an ED diagnosis to no diagnosis
1
5.00%
BN-P to EDNOS BN-P
1
5.00%
BED to BN-NP
1
5.00%
BN-P to pending diagnosis
1
5.00%
EDNOS AN-R to BN-P
1
5.00%
Table 6 shows the mean pre- and postassessment scores on eating disorders
symptoms, self-esteem and depression
inventories.
• Significant differences between the scores were not
found, however the sample size limits data analyses and
interpretation.
• Self-esteem mean scores indicated a small increase
from pre to post New Directions programme.
• Eating disorder symptoms and depression scores
showed little change from pre to post intervention.
Variable/Beha
viour
Pre
Post
t-test I
EDI
Mean
SD
n
Mean
SD
n
EDI Drive for
Thinness
12.46
5.90
24
12.08
6.53
24
t(23) = 0.28,p =.78
EDI Bulimia
4.67
6.13
24
4.88
5.41
24
t(23) = -0.38,p= 71
EDI Body
Dissatisfaction
17.96
7.98
24
19.13
8.64
24
t(23) = -0.72, =.48
6.79
15
t(14) = -0.13,p=.90
Rosenberg
Rosenberg
Self-Esteem
22.40
5.68
15
23.13
CDI
CDI Total
17.67
10.99
6
18.17
11.27
6 t(5) = -0.28, p= .79
Research question # 3: What is the
average length of stay in the
programme?
• The average number of sessions attended for the entire sample was
5.60 (SD =2.2; range 2-10 sessions).
• Table 7 shows the percentages for sample drop out and completion
rates.
• The drop out rate was created by tabulating those who did not
attend any New Directions sessions.
• There were two types of dropouts: those who chose to drop after the
pre-treatment assessment and those who lost contact with the
agency after the pre-treatment assessment.
• There were two types of treatment outcomes:
– (1) those that completed five or fewer sessions of the New Directions
group and
– (2) those that completed five or more of the treatment sessions.
Table 7: Sample drop out and
completion rates
Drop out rate (N=74)
17% after pre- 11% Lost
treatment
contact with
assessment
agency
Completion rate (N=34)
41%
Completed <
5 sessions
59%
Completed >
5 sessions
Table 8: Characteristics of clients who
completed group and clients who attended five
or less sessions
Variable/Behaviour
Pre
Mean
Post
SD
n
Mean
t-test
SD
n
Eating Disorder Symptoms
BMI
26.36
10.78
14
23.05
6.31
20
t(32) = 1.13, p = 0.27
Dieting days in past 28 days
20.53
12.82
14
23.33
10.74
18
t(31) = -0.68, p = 0.50
Dieting days in past 3 months
64.29
42.19
14
78.11
29.52
18
t(22.31) = -1.04, p = 0.28
Dieting days in past 6 months
128.57
84.38
14
147.22
59.59
18
t(22.46) = -0.70, p = 0.49
Binge episodes in past 28 days
14.36
16.89
14
12.74
21.15
19
t(31) = 0.22, p = 0.89
Binge episodes in past 3 months
40.00
49.27
14
45.21
70.98
19
t(31) = -0.24, p = 0.82
Binge episodes in past 6 months
80.14
95.90
14
82.68
129.7
19
t(31) = -0.06, p = 0.95 9.3
Binge days in past 28 days
9.33
11.00
12
10.31
11.77
13
t(23) = -0.21, p = 0.83
Binge days in past 3 months
31.50
33.04
12
30.77
30.94
13
t(23) = 0.06, p = 0.96
Binge days in past 6 months
66.33
69.58
12
62.46
59.88
13
t(23) = 0.15, p = 0.88
Vomit days in past 28 days
16.57
26.25
14
12.00
15.57
20
t(32) = 0.64, p = 0.53
Vomit days in past 3 months
45.71
63.13
14
36.90
42.60
20
t(21.14) = 0.45, p = 0.63
Vomit days in past 6 months
91.86
137.94
14
58.21
76.21
19
t(18.82) = 0.82, p = 0.42
Laxative days in past 28 days
2.00
5.44
14
0.40
1.79
20
t(14.99) = 1.06, p = 0.30
Laxative days in past 3 months
3.64
9.94
14
1.20
4.51
20
t(32) = 0.97, p = 0.34
• No significant differences were found between
the non-completers and completers of the
programme on eating disorder symptoms.
• Although not significant those who completed
five or more sessions of New Directions,
reported fewer binge episodes, vomit days,
laxative days, and diet pill days than those who
completed less than five sessions of New
Directions.
Conclusion &
Recommendations/Next Steps
•
•
•
•
•
•
•
a)
Discussion and interpretation of findings
The expected outcomes of the programme were increased self-esteem, reduction in
eating disorder symptoms, depression, and normalized eating behaviour.
Interpretation of the findings is limited by a small sample size and lack of follow-up
data.
Overall, the participants who completed New Directions reported a general reduction
in eating disorder symptoms.
The high incidence of reported self-harm, abuse and suicidality implies that
posttraumatic stress disorder and other psychiatric conditions are present and
possibly having an affect on the completion of the New Directions programme.
The seemingly high incidence of reported self-harm, abuse and suicidality, points to
the need for assessment of posttraumatic stress disorder and other psychiatric
conditions.
This may explain why approximately 41 percent (N=14/34) of the participants
completed less than half of New Directions programme.
Overall, these characteristics may also explain why the increase in self-esteem,
lowered depression scores and reduction in eating disorder symptoms, were less
than expected (Berkman, Lohr, & Bulik, 2007).
• b) Conclusion reflects the findings
• BANA collects a considerable amount of information on
each client.
• Each variable was collected at pre and post treatment,
utilizing a standardized structured interview and
administration of psychometric tests.
• The total number of adolescent client files for the
calendar years from 2002 – 2006 was 98.
• Thirty-eight clients attended the New Directions
programme.
• The programme evaluation indicated that psychoeducation slightly reduced eating disorder symptoms, but
did not lead to a state of recovery from the eating
disorder for most of the participants who completed New
Directions.
• A limitation of this conclusion is the sample size and lack
of follow-up data.
• A system for consistent data entry and analysis
by trained and skilled data entry personnel was
developed and put into place.
• A tracking system for client data over the course
of the New Directions programme through prepost and follow-up assessments was created
and put into practice.
• BANA’s capacity to conduct outcome research,
and collaborate with other outpatient clinics is
also greatly enhanced.
• The agency’s capacity to successfully apply for
research grants is also increased.
• Agency Benchmarks were identified around drop out
rate, the need for assessing readiness to change,
posttraumatic stress, and the need for following up with
persons who drop out before and after the assessment.
• Participants need more information regarding the
management of self-harm behaviours during the course
of the treatment.
• The literature on the drop out rate for eating disorder
treatment specifies 50% for persons with Bulimia and
35% for persons with Anorexia Nervosa.
• This project pointed out that the drop out rate (before the
programme began) for the New Directions programme is
28%, less than what is expected for either Anorexia
Nervosa or Bulimia Nervosa (Bulik, Berkman, Brownley,
Sedway, & Lohr, (2007). Approximately 41 percent did
not complete the programme (attended five or less
sessions).
Recommendations
• There were a number of adolescents with eating
disorders who presented for treatment who could have
benefited from a more intensive and specific psychiatric
assessment.
• An additional referral for a more comprehensive
assessment could be incorporated into the treatment
plan.
• It is also suggested from these results that completion
rate, was made more difficult due to the increased level
of psychological distress experienced by some of the
participants as evidenced by the self harm and
suicidalality self reports (Shapiro, Berkman, Brownley,
Sedway, J, Lohr, K, Bulik, 2007).
• Readiness to change and posttraumatic stress
could be a factor in the outcome analysis.
• The next steps in the multi stage programme
evaluation project would be to review the pre
and post assessment protocol, include a followup component to the assessment and measure
outcomes of the New Directions Group by
creating a wait list control group and compare
outcomes between the two groups.
• The agency and the University of Windsor,
Psychology Department have agreed to partner
again to continue to pursue funding from CHEO
towards that end.
Lessons learned from evaluation
activities
• The agency was well equipped to collect, enter and analyze the
data.
• Not enough time was allotted to analyze the date, and write the
report.
e) Impact of having done this evaluation on clients served, staff,
and the organization as a whole
• Because the project was analyzing archival data, the impact on the
clients was negligible.
• The impact on staff was important because the project allowed the
opportunity to develop and appreciate the unique set of skills
suitable for conducting research.
• The organization has increased its capacity for conducting research.
Next steps
• This proposal is the first step in a multi-step approach to
programme evaluation.
• Step 1 provided a snapshot of the New Directions
programme.
• It identified demographic information, examined clients’
level of functioning pre and post treatment group and
generated an understanding of the drop out rate for New
Directions.
• The outcomes of the programme i.e. increased selfesteem, reduction in eating disorder symptoms,
depression, body dissatisfaction, and normalized eating
behaviour were less than expected.
• The second step will be to expand on the methodology by including
a waitlist control group, which will be compared to the treatment
group pre and post evaluation.
• The third step in the programme evaluation approach will be to
include follow-up assessment approximately three months after the
intervention is complete in order to examine long-term effectiveness
(e.g., identify rates of recovery, relapse, and remission).
• Presentations and a possible publication about the programme and
research findings will be developed. Study findings will also be
available on the BANA website, University of Windsor REB website,
and CHEO website.
• We will also deliver presentations at professional meetings and
conventions, and publish findings in newsletters, and professional
journals.
Knowledge Exchange Plan
•
•
•
•
•
•
•
•
•
•
•
Overview of knowledge exchange activities relating to this programme/project
To post the results of the project on BANA website by June 30, 2007
To publish the results of the project in the BANA newsletter by June 30, 2007
To submit an article suitable for publication in a professional journal by September 30,
2007
Post the results of the project on the CHEO website by September 30/07;
Post the results on the University of Windsor Research Ethics Board website
Submit applications for poster presentations to Ontario Association of Social Workers
Annual Conference in 2008; Association of Psychologists Annual Conference in 2008;
Academy of Eating Disorders Annual Conference in 2008.
Knowledge exchange activities accomplished to date
Report to University of Windsor Research Ethics Board, April 6, 2007.
Report to BANA board of directors, April 30, 2007;
Application to Donner Foundation for Excellence Award in Research, May 15, 2007;
•
•
•
•
•
Further plans regarding knowledge exchange
activities
To generate an article and a report suitable for
publication in a professional journal.
To identify the areas for more research on
programme evaluation.
To make recommendations to the Ontario
Network of Eating Disorder Service Providers
regarding treatment outcomes and programme
evaluation.
To publish the results on BANA website,
University of Windsor REB website, and
Centre of Excellence for Child and Youth
Mental Health Services of CHEO website.
Acknowledgments
• Provincial Centre of Excellence for Child
and Youth Mental Health Service of
CHEO;
• University of Windsor Department of
Psychology;
• Bulimia Anorexia Nervosa Association
End
Thank you!