Transcript Effectiveness Measures for Telerehabilitation

Equivalence of Face-to-face and
Videoconference Administration of
Functional Communication
Measures for Post-Stroke Patients
Sue Palsbo, PhD
National Rehabilitation Hospital/George Mason University
Robert Mullen, MPH
Tobi Frymark, MA, CCC-SLP
American Speech-Language Hearing Association
Pamela G. Forducey, PhD, ABPP
INTEGRIS Neuroscience Institute and Telehealth
Funding


Robert Wood Johnson
Foundation,
Methodologies Grant,
#49143
US Department of
Education, National
Institute on Disability and
Rehabilitation Research
(NIDRR), Rehabilitation
Engineering and Research
Center (RERC) on
Telerehabilitation
#H133E990007-00C
RERC on
Telerehabilitation
Study Partners

Professional Association


American Speech-Hearing Language
Association (ASHA)
Rehabilitation Hospitals


INTEGRIS/Jim Thorpe (Oklahoma)
National Rehabilitation Hospital (Washington
DC)
Goal

Determine if a post-stroke
functional communication
assessment tendered by a
remote therapist is
equivalent to a face-to-face
assessment.
Criteria for Selecting Measures
(1) Be appropriate and relevant to people
with stroke.
(2) Have known psychometric properties
(validity and reliability) published in peerreviewed literature.
(3) Wide use in research and clinical practice.
(4) Be visually based (that is, the therapist
can measure using televideo without
touching the patient).
(5) All measures can be completed within 30
minutes.
Design Issues for Measuring
Equivalence

Serial correlation bias


Inter-rater reliability



Measure simultaneously, not serially
Use measurement tools with published
reliability values
Training
Bias in administration

Switch off the therapist conducting the
assessment
Randomization
Site #1
Site #2
Face-toface
admin
6
6
Remote
admin
6
6
Total: 24 paired observations
Functional Communication
Measures



Motor speech
Speech comprehension
Speech expression
FCM : Spoken Language Expression
Level 1: Individual attempts to speak, but verbalizations are not meaningful to
familiar or unfamiliar communication partners at any time.
Level 4: Individual successfully able to initiate communication using spoken
language in simple, structured conversations in routine daily
activities with familiar communication partners. Individual
occasionally requires moderate cueing, but is able to demonstrate
use of simple sentences (i.e., semantics, syntax, and morphology)
and rarely uses complex sentences/messages).
Level 7: Individual’s ability to successfully and independently participate in
vocational, avocational, and social activities is not limited by spoken
language skills. Independent functioning may occasionally include
use of self-cueing.
The Data
Speech Comprehension
On-site
Remote
On-site
Remote
On-site administration
NRH1
5
6
JTSLP1
5
6
NRH3
5
5
JTSLP2
6
6
NRH5
7
5
JTSLP3
5
5
NRH6
6
5
JTSLP4
6
7
NRH9
7
7
JTSLP5
6
5
NRH11
5
5
JTSLP6
7
7
Remote administration
NRH2
3
5
JTSLP7
6
5
NRH4
4
5
JTSLP8
6
7
NRH7
4
5
JTSLP9
7
6
NRH8
4
5
JTSLP10
4
3
NRH10
6
5
JTSLP11
6
5
NRH12
7
7
JTSLP12
4
3
ICC for Speech Assessment
Motor speech
Administration
Face-to-face
Remote
ICC
sig.
ICC
0.828
0.051
0.828
0.242
0.366
0.667
sig.
0.047
0.077
Speech Comprehension
Administration
Site
Face-to-face
Remote
ICC
sig.
ICC
NRH
0.545
0.214
0.737
Integris
0.762
0.084
0.872
sig.
0.062
0.010
Site
NRH
Integris
Speech Expression
Site
NRH
Integris
Administration
Face-to-face
Remote
ICC
sig.
ICC
0.733
0.080
0.844
0.906
0.015
0.837
sig.
0.042
0.025
Raw Agreement Index
Percent agreement
Exact
within 1
Motor speech
46%
96%
Speech comprehension
26%
91%
Speech expression
38%
96%
Conclusions and Improvements



Not clear why the percentage exact
agreement was low, and the agreement
within 1 score was high
Additional randomizing of SLPs would have
allowed us to control for one person who
appeared to consistently score lower
Scoring might have been more consistent
if we had selected activities that are a
better fit with the FCMs we selected
Future Research




Televideo assessments are worth pursuing
 Need to look at POTS videophones
Need to develop clinical protocols (positioning
camera; patient safety; appropriate candidates)
Need to identify more continuous measurement
scales and demonstrate equivalence
Maintain movement toward RCTs of teletherapy