Transcript Document

F
O
C
U
S
Focus on the
Outcomes of
Communication
Under
Six
Adjusting our FOCUS!
Measuring Meaningful
Clinical Outcomes
Paediatric Symposium '11 Building a Brighter Future
1
Team FOCUS
•
Prof. Nancy Thomas-Stonell, PI
•
Dr. Bruce Oddson, Co-PI
•
Dr. Peter Rosenbaum, Co-PI
•
Dr. Karla Washington, Post-Doctoral Fellow
•
Ms. Bernadette Robertson, Research Coordinator
•
Ms. Joan Walker, Research Assistant
Funding
SickKids Foundation
Canadian Institutes of Health Research (CIHR)
Bloorview Childrens Hospital Foundation
Paediatric Symposium '11 Building a Brighter Future
2
Eleven Research Partners in Five
Provinces Across Canada
•
•
•
•
•
•
•
•
•
•
•
Eastern Healthcare, St. Johns, NL
Nova Scotia Hearing and Speech Centres, NS
Beyond Words, Preschool Speech and Language Program, York Region, ON
Waterloo Preschool Speech and Language Program, ON
Holland Bloorview Kids Rehabilitation Hospital, ON
Hamilton Preschool Speech and Language Service, ON
Technology Access Clinic, ON
Wellington-Dufferin Guelph Region Preschool Sp & Lang Service, ON
ErinoakKids Centre for Treatment and Development & Halton-Peel
Preschool Speech and Language Program, ON
Calgary Health Region, AB
BC Centre for Ability, BC
Paediatric Symposium '11 Building a Brighter Future
3
A Typical Day in the Life of a SLP!
Example prepared with thanks to Laurie Graham
Paediatric Symposium '11 Building a Brighter Future
4
Preschool Speech-Language Pathologist
•
4 year old boy, A.B., presents for service
•
Formal assessment results (currently
available standardized tests) indicate a
moderate speech and language disorder
Paediatric Symposium '11 Building a Brighter Future
5
Assessment reveals…
• Has
a lot of trouble with sounds
– i.e. ‘ish’ instead of ‘fish’; ‘
– og’ instead of ‘frog’
• Has
trouble with pronouns
– i.e. often says ‘he’ instead of ‘she’
Paediatric Symposium '11 Building a Brighter Future
6
Parents reports the boy is:
•
extremely frustrated, has tantrums
•
teased at school
•
kids and teacher have trouble
understanding him
•
shy, evidence of low self-esteem
Paediatric Symposium '11 Building a Brighter Future
7
Clinical Goals for 8 week treatment block
•
A.B. will produce /f/ in word initial position
in structured settings 80% of the time.
•
A.B. will produce ‘she’ appropriately in
phrases in structured settings 80% of the
time.
•
Speech-Language Pathologist documents
parent comments in client file.
Paediatric Symposium '11 Building a Brighter Future
8
Eight weeks later…
•
During the last session, the SpeechLanguage Pathologist decides her goals
have not been met as the child is only
performing tasks at a 50% success rate.
•
The parents state that tantrums have
decreased, A.B. is no longer being teased
at school, and seems more confident in
communicating.
Paediatric Symposium '11 Building a Brighter Future
9
The importance of parent comments
•
The Speech-Language Pathologist has
chosen to include many parent comments in
her client notes because they give her an
indication of the child’s interactions with
others.
Paediatric Symposium '11 Building a Brighter Future
10
The importance of parent comments
– Interaction is fundamental to the
development of communication - the more
you interact, the more you practice
communication skills.
– The ability to communicate with peers and
teachers is fundamental to academic and
social success (i.e. group work).
Paediatric Symposium '11 Building a Brighter Future
11
Parent comments continued..
– Including parent comments in client
notes, although recommended by
regulatory bodies of the profession,
is not required.
– There are no valid and reliable
measures (i.e. tests/questionnaires)
that capture the real-world changes
observed by parents and clinicians!
Paediatric Symposium '11 Building a Brighter Future
12
Outcome measurement for A.B.
•
The Speech-Language Pathologist wishes
she could more thoroughly document the
behavioural, interactive and social changes
seen by A.B.’s parent.
•
She suspects that A.B.’s gains are
associated with therapy (not just normal
development) but there is no way to prove
her hypothesis.
Paediatric Symposium '11 Building a Brighter Future
13
Decision time:
•
There are other children on the
waitlist.
•
Given the limited funding available,
A.B. is not offered a second block of
therapy.
Paediatric Symposium '11 Building a Brighter Future
14
Increasing concern
•
Clinicians, researchers and disability
advocates are concerned that changes
which may be important results of therapy
are overlooked.
Paediatric Symposium '11 Building a Brighter Future
15
What are clinically meaningful
outcomes?
Paediatric Symposium '11 Building a Brighter Future
16
World Health Organization (WHO)
Health Frameworks
•
In 1980 WHO (1980) came out with their first
health framework the International Classification of
Impairments, Disabilities and Handicaps (ICIDH)
•
Impairment
(what’s broken)
•
Disability
(what can’t you do)
•
Handicap
(limitations in the real world)
Impairment
Disability
Paediatric Symposium '11 Building a Brighter Future
Handicap
17
International Classification of Functioning,
Disability and Health (ICF-2003)
Children & Youth Version (ICF-CY - 2007)
Health Condition
Body Functions &
Structures
Environmental
Factors
Activities
Participation
Personal
Factors
Paediatric Symposium '11 Building a Brighter Future
18
ICF & ICF-CY Domains
•
Body Functions: Physiological
– (e.g., voice, oral motor, speech
production)
•
Body Structures: anatomical
– (e.g., hearing loss; CL/P)
Paediatric Symposium '11 Building a Brighter Future
19
ICF & ICF-CY Domains
• Activities:
– Tasks and actions by an individual.
– ‘Capacity’ - performance of a task in a
standard environment.
• Participation:
– Involvement in a life situation.
– ‘Performance’ performance of tasks in a in
the current environment.
Paediatric Symposium '11 Building a Brighter Future
20
Contextual Factors
• Environmental
Factors:
• external
influences on functioning and disability
related to physical, social and attitudinal world.
(stairs, culture, support system)
• Personal
Factors:
• internal
influences on functioning and disability
(personality influences on coping style)
Paediatric Symposium '11 Building a Brighter Future
21
ICF Health Framework
• Has
positive and negative components.
• Uses
a bi-directional model.
• Doesn’t
take developmental stages into
account.
– e.g. temper tantrums/frustrations for
2 year olds
Paediatric Symposium '11 Building a Brighter Future
22
ICF-CY Health Framework
• New
codes to capture the functional
characteristics of a developing child.
Expanded codes include:
– Learning new skills
– Play
– Adaptability
– Persistence
– Exploration
Paediatric Symposium '11 Building a Brighter Future
23
Why use ICF-CY model?
• Outcomes
domains.
need to be evaluated across ICF-CY
• Several
studies (Sarno, 1969, Aten, 1986) have
noted poor correlation between body
structure/function outcomes and activity and
participation outcomes
– depends not only on skill levels, but also
personality, coping skills, social support
systems...
Paediatric Symposium '11 Building a Brighter Future
24
How do we measure these
outcomes?
Paediatric Symposium '11 Building a Brighter Future
25
We need treatment outcome measures!
We need outcomes measures to evaluate the impact of
treatment on children’s lives.
Paediatric Symposium '11 Building a Brighter Future
26
Outcomes vs. Outcome Measures
• Any
consequence of healthcare is an outcome!
Outcome = environment + treatment + client + severity
Paediatric Symposium '11 Building a Brighter Future
27
There are many types of outcomes.
• Avoiding
adverse affects of care (nobody dies)
• Improved
physiologic status ( phonation time)
• Reduction
in symptoms ( dysfluencies)
• Improved
functional status (use telephone)
• Minimizing
costs
• Minimizing
length of care
Paediatric Symposium '11 Building a Brighter Future
28
Outcome Measure
•A
treatment outcome measure is a validated
test designed to measure change in function.
• It
measures, in quantitative terms, the
impact of routinely delivered care on client’s
lives.
Paediatric Symposium '11 Building a Brighter Future
29
Treatment Outcome Measures need to
be proven to work!
• Garbage
in – garbage out!
• Information
generated by outcome studies is
only useful if the measure is clinically useful
and scientifically sound (van der Putten et al.,
1999).
Paediatric Symposium '11 Building a Brighter Future
30
Why can’t we use our standardized
tests?
Paediatric Symposium '11 Building a Brighter Future
31
Standardized tests...
•
Determine the presence or absence of a
communication disorder. They do not
change.
•
They provide too little information
(insufficient number and variety of items)
to monitor progress.
Huang, Hopkins & Nippold (1997). Satisfaction with Standardized Language
Testing: A survey of Speech-Language Pathologists. Language Speech &
Hearing Services in Schools 28, 12-29.
Paediatric Symposium '11 Building a Brighter Future
32
Treatment Outcome measures
• Outcomes
measures at a minimum need to be
proven to reliably distinguish between
children who improve from therapy and those
who do not improve.
Paediatric Symposium '11 Building a Brighter Future
33
Why use treatment outcome measures?
•
To improve treatment services
in an evidence-based manner.
•
To measure clinically
important change.
•
To determine optimal length
for treatment.
•
To select the best treatment
approach for each child.
Paediatric Symposium '11 Building a Brighter Future
34
CASLPA Position Statement on Outcome
Measures
• CASLPA
encourages and supports the use and
development of outcome measures by speechlanguage pathologists and audiologists
• Outcome
measures should be used to improve
practice in an evidence-based manner in the
best interests of clients.
•
CASLPA Position Statement on Outcome Measures – May, 2010
(www.caslpa.ca)
Paediatric Symposium '11 Building ASHA
a Brighter
Convention
Future
November 18-20 2010
35
The FOCUS journey began in 1998
• Holland
Bloorview Kids
Rehabilitation Hospital
wanted an outcome
measure for speechlanguage therapy that
could be used across
programs.
• Diverse
population
– CP/CLP/ABI
Paediatric Symposium '11 Building a Brighter Future
36
Began our search for a treatment
outcome measure.
• TOMS
and AusTOMS are very broad
measures of change. Scale has many
descriptors. Hard to know what changed.
• GAS
(individualized and time consuming).
• ASHA
NOMS had no proven reliability or
validity. We completed a two-year study to
evaluate the NOMS. Results indicated poor
sensitivity to change.
Paediatric Symposium '11 Building a Brighter Future
37
What do we do now?
• …with
no existing
valid, reliable and
responsive
communication
outcome measure for
preschool children
available?
Paediatric Symposium '11 Building a Brighter Future
38
Development of the ‘FOCUS’
F ocus on the
• O utcomes of
• C ommunication
• U nder
• S ix
•
•
Thomas-Stonell, N., Oddson, B.,
Robertson, B. & Rosenbaum, P.
Development of the FOCUS (Focus on
the Outcomes of Communication Under
Six), a communication outcome measure
for preschool children. Developmental
Medicine and Child Neurology: 2010,
52:47-53. ]
Paediatric Symposium '11 Building a Brighter Future
39
Our Goal
To develop a valid,
reliable, responsive
treatment outcome
measure that captures
‘real world’ changes
following speech and
language treatment.
Paediatric Symposium '11 Building a Brighter Future
40
Developing the FOCUS
• In
the previous outcome study, we
collected data from parents of 210
preschool children receiving speechlanguage treatment and their clinicians.
(Thomas-Stonell, Oddson, Robertson & Rosenbaum, Predicted and Observed
Outcomes in Preschool Children Following Speech and Language Treatment:
Parent and Clinician Perspectives. JCD 42 (2009) 29-42.)
Paediatric Symposium '11 Building a Brighter Future
41
Developing the FOCUS
• They
were asked to describe the changes
they observed in their child during/following
therapy.
– My child is now able to…
– What other changes did you see?
– Why is that important?
Paediatric Symposium '11 Building a Brighter Future
42
Method: 6 Linked-Steps
1. Content analysis of our descriptive data to create
FOCUS items.
2. Test the measure with clinicians and families.
3. Revise the measure using the parent and clinician
feedback.
4. Test the revised measure again.
5. Revise measure a second time.
6. Test measure a third time to obtain preliminary
reliability and validity data.
Paediatric Symposium '11 Building a Brighter Future
43
Content Analysis
• Content
analysis is the “systematic, objective
analysis of message characteristics” to make
valid inferences from text. (Neuendorf,
2002)
Paediatric Symposium '11 Building a Brighter Future
44
Content Analysis
• Identify
recurring categories of change and
calculate percentages of occurrence for each
category.
• The
recurring categories reflected the ICFCY framework.
Paediatric Symposium '11 Building a Brighter Future
45
Coding Comments
Working Slides
Paediatric Symposium '11 Building a Brighter Future
46
Developing the FOCUS
• FOCUS is driven by DATA,
• no preconceived ideas
• FOCUS items were developed from categories
cited by >10% of parents & clinicians.
• Resulted in 200 items, reduced to 103.
• Items used parents’ own wording.
Paediatric Symposium '11 Building a Brighter Future
47
Sample Body Functions
Item Development
Paediatric Symposium '11 Building a Brighter Future
48
Item Development: Body Functions
Parent Comment
“Pronounces words much more clearly
(specifically F sounds, L sounds when prompted
- he still has work to do with L's)”
Category/ICF-CY coding: Body Functions
– Articulation Functions; b320
FOCUS Item
“My child’s speech is clear.”
Paediatric Symposium '11 Building a Brighter Future
49
Sample Activities/Capacity
Item Development
Paediatric Symposium '11 Building a Brighter Future
50
Sample Item Development:
Activities
Parent Comment
“Says more words. Put more words together.”
Category/ICF-CY coding: Activities/Capacity
Communicating – producing
– Speaking; d330
FOCUS Item
My child can string words together.
Paediatric Symposium '11 Building a Brighter Future
51
Sample Participation
Item Development
Paediatric Symposium '11 Building a Brighter Future
52
Sample Item Development: Participation
Parent Comment
“His play with peers has improved in terms of
sharing, turn-taking, following conversations, acting
less aggressively.”
Category/ICF-CY coding: Activities & Participation
– Complex Interpersonal Interactions d720
FOCUS Item
“My child plays well with other children.”
Paediatric Symposium '11 Building a Brighter Future
53
Sample Personal Factors
Item Development
Paediatric Symposium '11 Building a Brighter Future
54
Sample Item Development:
Personal Factors
Parent Comment
“More confident in playing with peers or entering a
new group.”
Category/ICF-CY coding: Personal Factors
– Coping Style/Behavior Pattern
FOCUS Item
My child makes friends easily.
Paediatric Symposium '11 Building a Brighter Future
55
Initial FOCUS – 103 items
• Body
Functions
9%
• Activity/Capacity
28 %
• Participation/Performance
54 %
• Personal
20 %
Factors
• Environmental
Factors
3%
**Percentages exceed 100 as some items had 2 codes.
Paediatric Symposium '11 Building a Brighter Future
56
Criterion-referenced
• Performance
criterion.
is judged according to pre-stated
• Take
a verbal ‘snapshot’ of child’s skills at
Time 1 and Time 2 and use the changes in the
scores to measure change.
• Developed
a parent and a clinician version.
Paediatric Symposium '11 Building a Brighter Future
57
Response Categories #1
My child talks a lot.
Not at
all like
my
child
A little
like my
child
Somew
hat like
my
child
A fair
bit like
my
child
Quite a
bit like
my
child
Very
much
like my
child
Paediatric Symposium '11 Building a Brighter Future
Exactly like
my child
58
Response Categories #2
My child plays well with other children.
Can not Can do
do at
with a
all
great
deal of
help
Can do
with a
lot of
help
Can do
with a
bit of
help
Sometimes
does
without
help
Often
does
without
help
Paediatric Symposium '11 Building a Brighter Future
Can
always do
without
help
59
Sample FOCUS Form
FOCUS: Focus on the Outcomes of Communication Under Six
 Start Time: __________
1
My client’s speech is clear.
2
My client speaks slowly when not understood.
3
My client can string words together.
4
My client speaks in complete sentences.
5
My client uses correct grammar when speaking.
6
My client talks a lot.
7
My client is confident communicating with adults who
know my client well.
My client uses language to communicate new ideas.
8
Not
at all
like
my
client
A
little
bit
like
my
client
Somewhat
like my
client
A fair
bit like
my
client
Quite a
bit like
my
client
Very
much like
my client
Paediatric Symposium '11 Building a Brighter Future
Exactly
like my
client
How do I
know?
Parent
report
Part I
Observed
#
ID:_________
60
Sample FOCUS Items
1. My client’s speech is clear.
2. My client speaks slowly when not
understood.
3. My client can string words together.
4. My client speaks in complete sentences.
5. My client uses correct grammar when
speaking.
Paediatric Symposium '11 Building a Brighter Future
61
Sample FOCUS Items
6. My client talks a lot.
7. My client is confident communicating with
adults who know my client well.
8. My client uses language to communicate new
ideas.
Paediatric Symposium '11 Building a Brighter Future
62
FOCUS Instructions
• FOCUS
years.
designed for children from birth to 6
• If
children are too young to complete some
of the items, parents and clinicians need to
score the items as
“Not at all like my child”.
• This
allows these emerging skills to be
measured.
Paediatric Symposium '11 Building a Brighter Future
63
Instruction Example
•A
typical child of 15 months is probably only
speaking in one-word phrases, so the
response to the item:
• “My
child uses correct grammar when
speaking”
• would
be “Not at all like my child”.
Paediatric Symposium '11 Building a Brighter Future
64
FOCUS Definitions
• “Talking”,
“tell”, “speaks”, “speech”
and “words” refers to verbal speech.
• For
example,
• “My
child talks a lot.” refers to verbal
communication.
Paediatric Symposium '11 Building a Brighter Future
65
FOCUS Definitions
• “Communicating”,
“conversations”,
“participates” and “asking” can be any form
of communication.
– (e.g. pecs, AAC, sign)
• For
example
• “My
child will ask for help.”
Paediatric Symposium '11 Building a Brighter Future
66
FOCUS Definitions
• Some
children using AAC began to verbalize
during their speech therapy.
• This
is a very important functional change.
• We
needed to ensure that the FOCUS could
capture this change.
Paediatric Symposium '11 Building a Brighter Future
67
FOCUS Phase 1 Testing (N = 74)
FOCUS revised using measurement science.
• Data driven!
• Items were deleted if:
•
– Poor distribution of scores
– Poor completion rate
– Not sensitive to change
– Redundant
– Items not clear
Paediatric Symposium '11 Building a Brighter Future
68
Clinician and Parent Feedback
• Difficulty
completing the ‘school’ items in
Nova Scotia, (more rural setting).
• Both
parents and clinicians requested more
questions for younger children.
• They
suggested items such as babbling,
imitation…
Paediatric Symposium '11 Building a Brighter Future
69
Revisions
• Broadened
• Added
definition of ‘school’.
5 new items for younger children.
– My child is reluctant to talk.
– My child takes turns.
– My child does not interact with others.
– My child is independent.
– My child uses immature language.
– My child uses words to request items.
Paediatric Symposium '11 Building a Brighter Future
70
Second FOCUS Testing (N = 65)
• FOCUS
reduced to 77 items, including 5 new
items for young children.
• Tested
again with different parents.
• High
internal consistency indicated that the
FOCUS items had some redundancy.
– Parents:
– Clinicians:
• FOCUS
 = .98
 = .83
revised and reduced to 50 items.
[Thomas-Stonell et al., 2010]
Paediatric Symposium '11 Building a Brighter Future
71
Results: Item Distribution
Initial FOCUS
Final FOCUS
•
Body Functions
9%
•
Body Functions
•
Activities/Capacity
28 %
•
Activities/Capacity
34 %
•
Participation/Perf.
54 %
•
Participation/Perf.
56 %
•
Personal Factors
20 %
•
Personal Factors
10 %
•
Environ. Factors
3%
•
Environ. Factors
0 %
• Increased
• One
2 %
Activities and Participation items.
Body Function item remains (Speech Rate).
Paediatric Symposium '11 Building a Brighter Future
72
Results: Item Distribution
Initial FOCUS
•
Body Functions
•
Final FOCUS
9%
•
Body Functions
2 %
Activities/Capacity
28 %
•
Activities/Capacity
34 %
•
Participation/Perf.
54 %
•
Participation/Perf.
56 %
•
Personal Factors
20 %
•
Personal Factors
10 %
•
Environ. Factors
3%
•
Environ. Factors
0 %
• Remaining
FOCUS items demonstrated the most
sensitivity to change.
Paediatric Symposium '11 Building a Brighter Future
73
Phase 3 Testing
•Factor
analysis indicates
one construct!
•FOCUS
has 50 items.
•FOCUS
takes 10 minutes to
complete.
Paediatric Symposium '11 Building a Brighter Future
74
Communicative Participation
• “Communication
in life situations where
knowledge, information, ideas or feelings are
exchanged.” (Eadie et al, 2006)
• Life
situation = communication within a social
context.
• Exchange
= reciprocal nature of
communication.
Paediatric Symposium '11 Building a Brighter Future
75
Communicative Participation
• The
fundamental feature of ‘communicative
participation’ is the complex interaction
between the speaker and the social context.
Paediatric Symposium '11 Building a Brighter Future
76
Where are we now?
Reliability and
Validity Study
CIHR 2009 - 2011
Paediatric Symposium '11 Building a Brighter Future
77
FOCUS Journey
• Evaluating
• Coding
• Seek
other outcomes measures
collected comments
development funds
1998
2002
2003
• FOCUS
development study
2005
• FOCUS
validation study
2009
Paediatric Symposium '11 Building a Brighter Future
78
Reliability
• Parents
apart.
• Parent
completed the FOCUS twice, 7 days
test-retest reliability was high!
• The
same clinician scored the FOCUS twice during
a 30 day no treatment interval (N = 19).
• Clinician
test-retest reliability was high.
Paediatric Symposium '11 Building a Brighter Future
79
Clinician Inter-Rater Reliability
• Two
different clinicians administered the
FOCUS on the same child twice within a 30
day no-treatment interval.
• Clinicians’
high.
inter-rater reliability was also very
Paediatric Symposium '11 Building a Brighter Future
80
Validity Testing
Paediatric Symposium '11 Building a Brighter Future
81
Construct Validity
• Construct
validity is the extent to which a
measure correlates with the construct is was
designed to measure. (Streiner & Norman, 1995)
• Generally,
a number of independent studies are
required to establish the credibility of a measure.
Paediatric Symposium '11 Building a Brighter Future
82
Preliminary Validity Testing – PEDS-QL
• Parents of 22 children completed the FOCUS
and the Pediatric Quality of Life Inventory
(PedsQL) at the start and end of a treatment
block.
• Higher FOCUS scores at the end of
treatment correlated with higher PedsQL
total scores (r = .466, p = .029).
Paediatric Symposium '11 Building a Brighter Future
83
Preliminary Validity Testing – PEDS-QL
• Higher FOCUS scores were specifically
correlated with higher scores in the
psychosocial domain - emotional, social and
school functioning (r = .518, p = .013).
• Positive correlations between FOCUS scores
and the PedsQL indicate that the FOCUS has
construct validity.
Paediatric Symposium '11 Building a Brighter Future
84
Preliminary Validity Testing
Construct Hypothesis
• The
FOCUS will measure more change during
a Treatment Interval than during the Wait
List Interval.
(assuming treatment works!)
Paediatric Symposium '11 Building a Brighter Future
85
Demographics
• 43
preschool children with communication
impairments from:
–
Holland Bloorview Kids Rehabilitation Hospital,
– KidsAbility
– Alberta Health Services.
• Mean
• 63%
age = 2.7 yrs. (age range = 1.25 – 4.8 yrs)
of participants were boys.
Paediatric Symposium '11 Building a Brighter Future
86
Communication Function Classification
System (CFCS)
(Hidecker, 2008)
•
Level I: Effective Sender and Receiver with unfamiliar
and familiar partners
•
Level II: Effective but slower paced Sender and/or
Receiver with unfamiliar and familiar partners
Level III: Effective Sender and Receiver with familiar
partners
•
Level IV: Sometimes Effective Sender and/or Receiver
with familiar partners
•
Level V: Seldom Effective Sender and Receiver even with
familiar partners
Paediatric Symposium '11 Building a Brighter Future
87
Severity
• Children
(severe).
ranged in CFCS from 1 (mild) to 5
• The
majority of the children (70%) were
classified in Level IV – Level V.
• 51%
of the children also had a diagnosis of
developmental delay.
Paediatric Symposium '11 Building a Brighter Future
88
Methods
• Parents
and clinicians completed the FOCUS at
assessment, start and completion of a treatment
block.
• On
average, there were 36 days between
assessment and start of treatment.
• On
average, there were 107 days between the
start and end of treatment.
• Ave
amount of treatment provided was 9.7
hours.
Paediatric Symposium '11 Building a Brighter Future
89
Preliminary FOCUS Results
• Significant
change was noted by both parents and
clinicians after treatment. No change was noted
during the waiting list period.
• Parents
and clinicians score identical amounts of
change from T2 – T3.
• FOCUS
demonstrates both stability and the ability
to measure change.
Paediatric Symposium '11 Building a Brighter Future
90
Validity Testing – VABS II
(Washington, 2011)
• Progress
measured by the FOCUS was
compared to progress measured by the
Vineland Adaptive Behavior Scales (VABSII)
• VABS
II selected as it assesses
communication skills as well as broader
participation (i.e., Socialization) skills.
Paediatric Symposium '11 Building a Brighter Future
91
Method
• Sixty-seven
parents of preschool children
ages 3 to 6 years old with communication
disorders participated.
• Parents
recruited from one of three
agencies:
1. Holland
Bloorview Kids Rehabilitation Hospital (Integrated
Education and Therapy Program)
2. Toronto
Preschool Speech and Language Services – West
Quadrant
3. University
of Western Ontario, tykeTALK
Paediatric Symposium '11 Building a Brighter Future
92
Preschoolers’ Group Description
• Group
1 – Communication Disorder only
and receiving intervention
• Group
2 – Communication Disorder and a
developmental mobility impairment and
receiving intervention
• Group
3 - Control participants, on
waitlist for intervention
Paediatric Symposium '11 Building a Brighter Future
93
Method
• Fifty-two
children received
direct group or individual
intervention with an SLP
• Fifteen
children acted as
waiting list controls.
•A
different SLP completed
VABS-II and FOCUS by
telephone with the parent
following treatment.
Paediatric Symposium '11 Building a Brighter Future
94
VABS-II
Measure
Purpose
1. Vineland Adaptive Behavior Scales –
II (VABS-II; Sparrow, Cicchetti, & Balla,
2005)
Assessment of everyday adaptations for
four major domains, including
socialization. Raw scores used to
establish participation skills
•Interpersonal Relationships
•Play & Leisure Time
•Coping Skills
Paediatric Symposium '11 Building a Brighter Future
95
VABS-II
•
Interpersonal Relationships
– Demonstrates friendship seeking behaviors with
others the same age (e.g., “Do you want to play?”)
•
Play & Leisure Time
– Plays simple make-believe activities with others (e.g.,
plays dress-up, pretends to be superheroes)
•
Coping Skills
– Ends conversation appropriately (e.g., says “Goodbye”).
Paediatric Symposium '11 Building a Brighter Future
96
VABS-II Response Options
Response Option
Description
2
Usually
1
Sometimes or partially
0
Never
DK
Don’t know
Paediatric Symposium '11 Building a Brighter Future
97
Results
• Changes
on the FOCUS and VABS-II
Socialization domain are significantly
correlated.
• Participants
receiving intervention
experienced significantly greater gains
compared to controls
Paediatric Symposium '11 Building a Brighter Future
98
Parent Comments - Intervention
Assessment
Re-check 1
Will play at cousin's
house, can be
sociable and “hands
Parentand
Comments
on”, initiates
makes requests
-
Sometimes will wait his
turn, will share with
others and can follow
Intervention
nonverbals
Re-check 2
More confidence, more likely to
initiate, sometimes asks to play
with others
Paediatric Symposium '11 Building a Brighter Future
99
Discussion
• Correlations between the FOCUS and the
VABS-II Socialization domain
demonstrates construct validity
• The FOCUS is another measure of
Participation, although somewhat different
from the VABS-II
• SLP administered/supervised
• Shorter administration time
• Sensitive to changes in communication-level participation
Paediatric Symposium '11 Building a Brighter Future
100
Participation Predictors
• Wanted
to know which factors predicted the
Participation changes measured by the
FOCUS
• Multiple
regression analyses were preformed
on the results of the 52 children who
received therapy.
Paediatric Symposium '11 Building a Brighter Future
101
Predictors
Measure/Procedure
Predictor Variable
1. Demographic Information
Age, sex
2. The matrices component of the
Kaufman Brief Intelligence Test –II
(K-BIT2; Kaufman & Kaufman, 2004)
Nonverbal IQ
3. Communication Domain of the
Vineland Adaptive Behavior Scales
(VABS-II; Sparrow et al., 2005)
Pre-Tx Communication (parent)
4. Communication Function
Classification System (CFCS;
Hidecker et al., 2008)
Pre-Tx Communication (SLP)
5. Socialization Domain of the VABS-II
Pre-Tx Participation Skills
6. Physician/SLP Report
Presence of a Physical Disability
7. SLP/Parent Report
English as a Second Language
8. SLP Report
Amount of Direct Intervention
Paediatric Symposium '11 Building a Brighter Future
102
General Results
• SLP
treatment has a positive effect on
children’s ability to participate in their
world!
• Specific
factors unique to children
predicted improved Participation skills
• Factors
unique to children’s
environment may be predictive as well…
Paediatric Symposium '11 Building a Brighter Future
103
Case Study #1
Pretreatment Participation Skills
Parent Description of Participation Skills
“Sometimes goes to parties on weekends. Will go
to grandmother's house. Does play well with
other kids. Can take turns.”
Paediatric Symposium '11 Building a Brighter Future
104
Case Studies
Three Preschool Children Attending Speech-Language
Therapy
(Washington, 2010)
Paediatric Symposium '11 Building a Brighter Future
105
Case Study # 1:
Child with Communication Disorder and
Mobility Impairment
•5 yrs 3 month old boy with Pierre Robin
Syndrome.
•Mild physical impairment (GMFCS Level 1)
due to club foot.
•Some fine motor difficulties (OT)
•Communication disorder secondary to cleft
lip and palate.
Paediatric Symposium '11 Building a Brighter Future
106
Case Study # 1
Pretreatment Communication Skills
• Describe
your child’s communication abilities.
(e.g., listening and talking skills)
SLP
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Paediatric Symposium '11 Building a Brighter Future
107
Case Study # 1
Pretreatment Communication Skills
•
CFCS level = 3
(Hidecker, 2008)
‘Effective sender and receiver with familiar partners’
•
Difficulties with speech sounds and resonance. Mild
expressive language difficulties.
Parent Description of Communication Skills
“Okay communication. Pronunciation is hard for
strangers to understand. Better with repetition.
Makes it hard for others to understand him, but
great personality.”
Paediatric Symposium '11 Building a Brighter Future
108
Case Study # 1
Pretreatment Participation Skills
• Describe
your child’s participation skills.
(e.g., Does your child play at other children’s homes or
go to birthday parties or other social events?)
SLP &
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Paediatric Symposium '11 Building a Brighter Future
109
Case Study # 1
Pretreatment Participation Skills
Parent Description of Participation Skills
“Sometimes goes to parties on weekends. Will go
to grandmother's house. Does play well with
other kids. Can take turns.”
Paediatric Symposium '11 Building a Brighter Future
110
Case Study # 1
Treatment
• 15.5
hours of group treatment
• Total
duration = 29 weeks.
Treatment Goals
– Mark final consonants in words with hard contact,
– /t,d,f,s/-all word positions,
– Improve consonant blends,
– Reduce nasal turbulence on fricatives.
Paediatric Symposium '11 Building a Brighter Future
111
Case Study # 1
Pre - Post Treatment Scores
Parent FOCUS
Change
– Pre
280
– Post
336
+ 56 points
VABS Communication
– Pre
120
– Post
148
+ 28 points
VABS Socialization
– Pre
98
– Post
146
+ 48 points
Paediatric Symposium '11 Building a Brighter Future
112
Case Study # 1
Post Treatment Communication Skills
• Describe
your child’s communication abilities
since the last interview.
SLP
Parent
SLP
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Orange = pretreatment
Green = Post Treatment
Paediatric Symposium '11 Building a Brighter Future
113
Case Study # 1
Post Treatment Communication Skills
Parent Description of Communication Skills
• “/l
/ & /s/ have improved. He is better. Clearer
to others, especially non-family members. Now
he is using more and longer sentences.”
• “This
is very important because other people
can understand him better now.”
Paediatric Symposium '11 Building a Brighter Future
114
Case Study # 1
Post Treatment Participation Skills
• Describe
your child’s participation abilities
since the last interview.
SLP &
Parent
SLP &
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Orange = pretreatment
Green = Post Treatment
Paediatric Symposium '11 Building a Brighter Future
115
Case Study # 1
Post treatment Participation Skills
Parent Description of Participation Skills
• “He
takes turns and listens better. He responds
to questions better.”
• “This
is important because he can be with other
people better and not be sad.”
Other Observations
• “He
has become better overall. He is talking and
playing more.”
Paediatric Symposium '11 Building a Brighter Future
116
Case Study # 1
High Change FOCUS Items [> 3]
• My
child’s speech is clearer.
• My
child can string words together.
• My
child speaks in complete sentences.
• My
child uses correct grammar when speaking.
+5
• My
child can communicate independently with
adults who do not know my child well.
Paediatric Symposium '11 Building a Brighter Future
117
Case Study # 1
High Change FOCUS Items
• My
child’s communication skills get in the way
of learning.
• My
child will try to carry on a conversation
with adults who do not know my child well.
• My
child can communicate effectively with
adults who do not know my child well.
Paediatric Symposium '11 Building a Brighter Future
118
Case Study # 1
High Change FOCUS Items
• Many
of the play and peer items were scored
at level 6 (Often does without help) at the
start of treatment.
• Most
of these items also improved, but they
could only improve by 1 point.
• Therefore
they were not included in the ‘high
change’ items described above.
Paediatric Symposium '11 Building a Brighter Future
119
Case Study # 2
Paediatric Symposium '11 Building a Brighter Future
120
Case Study # 2:
Child with Communication Disorder and
Mobility Impairment
•4
yrs 1 month old boy with Cerebral Palsy
(Spastic Quad).
• GMFM
=4
• Uses
a wheelchair most of the time; Also has a
walker.
• CFCS
• “An
=1
effective communicator in most situations”
Paediatric Symposium '11 Building a Brighter Future
121
Case Study # 2
Pretreatment Communication Skills
• Describe
your child’s communication abilities.
(e.g., listening and talking skills)
SLP &
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Paediatric Symposium '11 Building a Brighter Future
122
Case Study # 2
Pretreatment Communication Skills
Parent Description of Communication Skills
“Still developing vocabulary. Using 4-5 word
sentences. Learning new words and word
approximations, but I don’t always know
what he wants which leads to frustration on
his part.”
Paediatric Symposium '11 Building a Brighter Future
123
Case Study # 2
Pretreatment Participation Skills
• Describe
your child’s participation skills.
(e.g., Does your child play at other children’s homes or
go to birthday parties or other social events?)
Parent
SLP
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Paediatric Symposium '11 Building a Brighter Future
124
Case Study # 2
Pretreatment Participation Skills
Parent Description of Participation Skills
“Not always sociable. No mobility issues affect
this. He does not imitate and changing activities
is difficult.”
Paediatric Symposium '11 Building a Brighter Future
125
Case Study # 2
Treatment
• 41
hours of group treatment
• Total
duration = 29 weeks.
Treatment Goals
– Increase vocabulary.
– Improve understanding and use of concepts.
– Improve understanding and use of action words.
– Appropriate responses to questions.
– Expand sentence length.
Paediatric Symposium '11 Building a Brighter Future
126
Case Study # 2
Pre - Post Treatment Scores
Parent FOCUS
Change
– Pre
270
– Post
246
- 24 points
– Follow-UP
309
+ 63 points
(+ 39 points)
VABS Total Score
– Pre
89
– Post
135
+ 46 points
– Follow-UP
136
+ 1 point
Paediatric Symposium '11 Building a Brighter Future
(+ 47 points)
127
Case Study # 2
Post Treatment Communication Skills
• Describe
your child’s communication abilities
since the last interview.
SLP &
Parent
SLP &
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Orange = pretreatment
Green = Post Treatment
Paediatric Symposium '11 Building a Brighter Future
128
Case Study # 2
Post Treatment Communication Skills
Parent Description of Communication Skills
• “He
is talking a lot more now and answers
questions appropriately.”
• “This
is very important to us because now we
are sure about what he wants/needs. We feel
better about addressing his needs. We feel like
better parents.”
Paediatric Symposium '11 Building a Brighter Future
129
Case Study # 2
Post Treatment Participation Skills
• Describe
your child’s participation skills.
(e.g., Does your child play at other children’s homes or
go to birthday parties or other social events?)
Parent
SLP
SLP
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Orange = pretreatment
Green = Post Treatment
Paediatric Symposium '11 Building a Brighter Future
130
Case Study # 2
Post treatment Participation Skills
Parent Description of Participation Skills
•
“Increased initiation noted. Increased attention during
circle time.”
•
“This is important because he can interact with others.”
Other Observations
•
“He is more aware that his actions lead to results. Have
an impact on others and his environment.”
Paediatric Symposium '11 Building a Brighter Future
131
Case Study # 2
Follow-Up Communication Skills
• Describe
your child’s communication abilities
since the last interview.
1
2
SLP
SLP &
Parent
3
Limited
Ability
Below
Average
Low
Ability
Orange = pretreatment
SLP &
Parent
Parent
4
5
6
7
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Green = Post Treatment
Blue = Follow-UP
Paediatric Symposium '11 Building a Brighter Future
132
Case Study # 2
Follow-Up Communication Skills
Parent Description of Communication Skills
• “May
not say much, but vocabulary has
definitely improved. Increased grammar
(possessive form).
• “This
is important because it helps him
communicate with his peers and allows him to
find new ways of expressing himself.”
Paediatric Symposium '11 Building a Brighter Future
133
Case Study # 2
Follow-Up Participation Skills
• Describe
your child’s participation skills.
Parent
SLP
SLP &
Parent
Parent
SLP
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Orange = pretreatment
Green = Post Treatment
Blue = Follow-Up
Paediatric Symposium '11 Building a Brighter Future
134
Case Study # 2
Follow-Up Participation Skills
Parent Description of Participation Skills
• “Great.
He is highly engaged in circle time. He still
needs help physically but is willing to participate.”
“This is great because he has initiative!”
Other Observations
“ He has better memory and is more curious about
world. He is showing likes/dislikes more and
starting to assert himself beyond food preference
(e.g. Dressing).”
Paediatric Symposium '11 Building a Brighter Future
135
Case Study # 3
Paediatric Symposium '11 Building a Brighter Future
136
Case Study # 3:
Child with Communication Disorder
•3
yrs 6 month old boy.
• Severe
speech and language disorder.
• Difficulties
with both receptive and
expressive language
• CFCS
Level = 4
• “Inconsistent
Sender and/or Receiver with
familiar partners”
Paediatric Symposium '11 Building a Brighter Future
137
Case Study # 3
Pretreatment Communication Skills
• Describe
your child’s communication abilities.
(e.g., listening and talking skills).
SLP
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Paediatric Symposium '11 Building a Brighter Future
138
Case Study # 3
Pretreatment Communication Skills
Parent Description of Communication Skills
“Poor clarity of speech. Delayed grammar.”
“Late talker.”
“People don’t understand him. He is limited in
expressing himself to others because they
don’t understand.”
Paediatric Symposium '11 Building a Brighter Future
139
Case Study # 3
Pretreatment Participation Skills
• Describe
your child’s participation skills.
(e.g., Does your child play at other children’s homes or
go to birthday parties or other social events?)
SLP &
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Paediatric Symposium '11 Building a Brighter Future
140
Case Study # 3
Pretreatment Participation Skills
Parent Description of Participation Skills
“He is very sociable and entertaining.”
“He makes friends easily.”
Paediatric Symposium '11 Building a Brighter Future
141
Case Study # 3
Treatment
•8
hours of group treatment
• Total
duration = 5 weeks
Treatment Goals
– Improve expressive language.
– Increase MLU / expand phrases.
– Teach vocabulary using themes.
Paediatric Symposium '11 Building a Brighter Future
142
Case Study # 3
Pre - Post Treatment Scores
Parent FOCUS
Change
– Pre
246
– Post
296
+ 50 points
VABS Communication
– Pre
94
– Post
113
+ 19 points
VABS Socialization
– Pre
117
– Post
134
+ 17 points
Paediatric Symposium '11 Building a Brighter Future
143
Case Study # 3
Post Treatment Communication Skills
• Describe
your child’s communication abilities.
(e.g., listening and talking skills).
SLP
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Orange = pretreatment
Green = Post Treatment
Paediatric Symposium '11 Building a Brighter Future
144
Case Study # 3
Post Treatment Communication
Parent Description of Communication Skills
“His speech still not clear.
“He is trying to make sentences.”
“His vocabulary has improved but he still has
difficulty with concepts such as
first/middle/last and with following
instructions.”
Paediatric Symposium '11 Building a Brighter Future
145
Case Study # 3
Post Treatment Participation
• Describe
your child’s participation skills.
(e.g., Does your child play at other children’s homes or
go to birthday parties or other social events?)
SLP &
Parent
Parent
1
2
3
4
5
6
7
Limited
Ability
Below
Average
Low
Ability
Average
Ability
Good Ability
Above
Average
Exceptional
Ability
Orange = pretreatment
Green = Post Treatment
Paediatric Symposium '11 Building a Brighter Future
146
Case Study # 3
Post Treatment Participation Skills
Parent Description of Participation Skills
“He is talking more. He finds games to play. He
initiates more.”
“He is more likeable and has more friends now.”
Other Observations
“He is more confident now.”
Paediatric Symposium '11 Building a Brighter Future
147
Case Study # 3
High Change FOCUS Items [>3]
• My
child’s communication skills get in the way of
learning.
[+5]
• My
child’s communication skills limit his
independence.
[+5]
• My
child waits for her/his turn to talk. [+4]
Paediatric Symposium '11 Building a Brighter Future
148
Case Study # 3
High Change FOCUS Items [=3]
• My
child’s is confident communicating with
adults who do not know my child well.
• My
child is understood the first time when s/he
is talking with other children.
• My
child takes turns.
• My
child can tell stories that make sense.
Paediatric Symposium '11 Building a Brighter Future
149
Case Study # 3
High Change FOCUS Items
• Even
though the parent did not rate the
communication skills as improved, she felt that
participation skills had improved a lot!
• She
was no longer concerned that
communication skills were interfering with
independence and learning.
Paediatric Symposium '11 Building a Brighter Future
150
FOCUS
Limitations:
• The
7 point rating scale may not have been
sensitive enough to capture communication
changes (change from low ability to average
ability).
Paediatric Symposium '11 Building a Brighter Future
151
Discussion
• Improvement
was measured by the FOCUS,
VABS communication and VABS socialization
domain scores.
• In
Case # 1 and # 2, the parent noted improved
communication skills. In Case # 3, the parent
did not rate communication skills as improved.
• For
all children, there were improvements in
participation scores!
Paediatric Symposium '11 Building a Brighter Future
152
FOCUS
• An
outcome measure that only measured
changes in specific communication skills (e.g.,
MLU, expressive grammar, articulation) would
have missed many of the positive changes
associated with treatment.
• Even
when there were no identified concerns
with participation pretreatment,
improvements were noted after treatment.
Paediatric Symposium '11 Building a Brighter Future
153
Summary
Paediatric Symposium '11 Building a Brighter Future
154
Summary
• The
use of a newly developed measure of
paediatric ‘participation’ outcomes, the FOCUS,
has provided evidentiary support that speech and
language intervention can have a broad and
positive effect on progress in participation skills!
Paediatric Symposium '11 Building ASHA
a Brighter
Convention
Future
November 18-20 2010
155
Examples of ‘Real Life’ Impact of
Speech-Language Therapy
• More
sociable.
• Understood
better by others.
• Improved
attention and listening skills.
• Improved
play with peers.
• Increased
• Less
communication with others.
frustration/improved confidence.
• Fewer
negative behaviors/temper tantrums.
Paediatric Symposium '11 Building a Brighter Future
156
Summary
• An
outcome measure that focuses solely on
speech and language skills (i.e. impairments)
would miss the large changes associated with
communicative participation.
Paediatric Symposium '11 Building a Brighter Future
157
Summary
Preliminary results suggest that…
The FOCUS is successfully measuring the
‘real world’ communication outcomes
corresponding at the ICF level of
participation.
Paediatric Symposium '11 Building a Brighter Future
158
FOCUS Journey continues…
• Evaluating
• Coding
• Seek
other outcomes measures
collected comments
development funds
1998
2002
2003
• FOCUS
development study
2005
• FOCUS
validation study
2009
• Dissemination
of the FOCUS
Paediatric Symposium '11 Building a Brighter Future
2011
159
Final Thoughts
• The
evaluation of outcomes in
the field of speech-language
pathology would benefit from
the development and
implementation of additional
measures of communicative
participation.
Paediatric Symposium '11 Building a Brighter Future
160
Acknowledgements
A special thank you to
all of the families and
clinicians who
participated in these
studies.
Paediatric Symposium '11 Building a Brighter Future
161
[email protected]
Paediatric Symposium '11 Building a Brighter Future
162