Transcript Seminar on Speech and Language Processing for Augmentative

Seminar on Speech and Language Processing for
Augmentative and Alternative Communication
Class 10: More Symbol Systems
Presenter:
Masoud Rouhizadeh
 1) Transparency and ease of learning of symbols
represented by Blissymbolics, PCS, and Picsyms, Mizuko,
1987, Augmentative and Alternative Communication.
 2) Use of picture dictionaries to promote written
communication by students with hearing and cognitive
impairments, Cohen et al, 2001, Augmentative and
Alternative Communication.
 3) Generalization of a pictorial alternative
communication system across instructors and distance.
Ganz et al, 2008, Augmentative and Alternative
Communication.
Paper #1: Mizuko, 1987
Transparency and Ease of Learning of Symbols Represented
by Blissymbols, PCS, and Picsyms
 Determining whether significant differences existed among
normally developing 3-year-old children in transparency and in
the learning of symbols from three different graphic symbol
systems:
 Blissymbols
 Picture Communication System [PCS]
 Picsyms
Factors for selecting nonspeech symbol
systems
 Which nonspeech symbol system is the most appropriate
for a particular nonspeaking individual?
 What factors should be considered when selecting a nonspeech symbol system?
 Ease of acquisition (Clark, 1981 and others)
 Iconicity (which may facilitate symbol communication
development among nonspeaking individuals) (Fristoe
and Lloyd, 1979)
Iconicity
 The visual representation depicted by nonspeech symbols, in particular
their ability to visually resemble or suggest their referents.
 Bellugi and Klima (1976): the degree to which the elements of a sign or
symbol are related to the visual aspects of what is denoted.
Symbols which are highly suggestive of their referents. Their
meaning can be readily guessed by naive viewers.
(Transparent symbols)
.
.
Symbols which may not be readily guessable, but the viewer is
able to perceive a relationship between the symbol and its
meaning (Translucent symbols)
.
.
Abstract symbols with no apparent relationship to its referent,
which are not guessable. (Opaque symbols)
Transparency vs translucency
 Transparency: guessing the symbol when given the
gloss or guessing the gloss when shown the
symbol.
 Translucency: agreement regarding the
relationships between a symbol and its referent.
Iconicity in nonspeech symbols
 Nonspeech symbols with high iconicity were more easily
learned by individuals with mental retardation and/or
autism than nonspeech symbols with low iconicity.
 Clark (1984) suggested that nonspeaking individuals with
apparent cognitive delays may need a system that is iconic
and easily learned.
 Conversely, when little or no cognitive delay is present in a
nonspeaking individual, ease of learning and iconicity may
not be critical factors.
The purpose of the study
 To investigate transparency and the ease of learning of
referents represented by different graphic symbol
systems (Blissymbols, PCS, Picsyms).
 Also to investigate whether there are learning and
transparency differences across different graphic
symbol systems within three different word categories
(nouns, verbs, descriptors).
Blissymbols , Picsyms and PCS
 Blissymbols and Picsyms
 pictographic symbols (drawings that resemble their
referents),
 ideographic symbols (drawings that symbolize the idea
rather than the name of a referent)
 arbitrary symbols (drawings which do not have a
perceptual or conceptual relationship to its referent),
whereas
 PCS: mainly pictographic symbols.
Picture Communication Symbols
(PCS)
PCS (no word version)
Bliss
Picsyms
Subjects
36 subjects from a group of normally developing children, 29
to 44 months of age
1. English spoken as the primary language
2. no apparent handicaps
3. vision and hearing within normal limits
4. chronological age appropriate vocabulary recognition skills
5. lack of familiarity with any of the three symbol systems
Materials
 Three test booklets each containing 45 items from one of
the symbol systems
 15 nouns, 15 verbs and 15 descriptors were randomly
chosen for the learning tasks from a pool of referents
 The referent should have the following criteria:
1. listed in the PCS Dictionary, Picsyms Dictionary and
Blissymbolics for Use.
2. high frequency of occurrence
3. they could not be used as a noun and a verb (e.g., drink)
4. they could not be used as a body part or person
Procedures
 45 items of the booklet
 3 learning trials
 Transparency and learning tasks
 showing the subject a stimulus page with four different
symbols and asking the subject to select the symbol that
best matched the spoken label
Results

The Scheffe test for means revealed significant differences between Blisymbols and both
PCS and Picsyms.

Overall scores were significantly lower with Blissymbols than with the other two symbol
systems.

There were significant differences between PCS and Picsyms for descriptors and verbs.
Results
Discussion
 Fewer Blissymbols were correctly identified than either PCS or
Picsyms
 Picsyms and PCS were similar for nouns, while PCS was more
transparent than Picsyms for verbs and descriptors.
 Learning: more PCS symbols were learned over three trials than
either Picsyms or Blissymbols : symbols judged to be iconic are
learned more easily than symbols that are not iconic.
 PCS and Picsyms may be used with physically disabled children
with spoken language comprehension skills near 3 years, if they
need to acquire an immediate means of communication.
 Conversely, when selecting a system for older persons with
severe physical disabilities with little or no cognitive delay, ease
of acquisition may not be an important factor.
Paper #2: Cohen et al, 2001
Use of picture dictionaries to promote written
communication by students with hearing and cognitive
impairments
 Students with deafness and limited literacy skills at communitybased vocational sites
 Students were taught to use picture dictionaries—small
notebooks consisting of symbols with corresponding words—to
write printed messages to their communication partners.
Introduction
 Implementing other forms of communication can be
challenging in vocational settings such as community-based
vocational training (CBVT) sites.
 CBVT sites: businesses in the community, agreed to
provide work experience opportunities to high school
students
 Additional forms of communication should be available for
untrained coworkers, customers and students
 Interpreters? not always available in vocational settings
Communication partners
 Primary: teachers and family members
 Regular: immediate coworkers
 Primary and regular communication partners have the
opportunity to learn sign, gestures, and other effective
methods of interaction.
 Irregular: a coworker working in a different area
 Strangers: a customer
 Irregular communication partners and strangers have
little or no consistent contact with the student
Symbol-Based Forms of Communication
 Such as : PCS, Bliss, Picsyms
 Negative attitude:
 Symbol-based form of communication is different from
the gestures and written notes
 So they may be a visible sign of students' reduced
literacy capabilities
 Students may not use symbol-based systems because of
concerns that such systems make them appear “retarded”
to other people
Picture Dictionaries
 A series of pictures or symbols with printed labels,
placed in an alphabetical or categorical sequence
Picture Dictionaries
 Assuming that the student have sufficient writing skills
to look up and copy words, they can refer to a picture
dictionary for the desired picture, copy the associated
printed words onto a pad of paper, and give the
communication partner the written message.
 Less negative attitude:
 Because the picture symbols are hidden from the
communication partner and the product is a written note,
use of a picture dictionary should be less stigmatizing
and more motivating than a standard communication
board.
Participants
Materials
 Symbol Communication Boards
 multiple pages with 15 symbols per page of both site-
specific and job-related vocabulary, general vocabulary,
and social vocabulary.
 Picture Dictionaries
 Individualized picture dictionaries with pages of symbols
accompanied by written word(s) or short phrases
describing them.
Preintervention Assessment
 Students were assessed with regard to writing skills
 Copying the words
 Legibility of writing
 Copying speed
 Comprehension
 Eliminating the words that the students could spell and
sign
Picture Dictionaries Vocabulary Selection
 The investigator observed the students at each job site and
recorded the vocabulary words that were needed for them to
perform each task
 Interviews were conducted with the students and their job
coaches, coworkers, and site supervisors
 Target vocabulary words were selected out of a pool based on
their frequency of use, their potential usefulness for job
completion, and the likelihood that the student need that word
 Most of the selected words were job specific and consisted of
nouns representing the items necessary for job completion and
directive phrases such as I NEED, WHERE IS, or SHOW ME
 The social phrases PLEASE and THANK YOU and the names of
regular communication partners were also included
Instructional Procedure
 Teaching symbol meanings and the picture
dictionaries
 The distinction between the symbol-based boards and
picture dictionaries explained to minimize any
negative reactions
 Locating an appropriate symbol or word in response to
specific queries, copying the words associated with
each symbol ...
Design and Data Collection
 At baseline, data were collected on the student’s frequency of
board use at CBVT sites and their use of any other forms of
aided or unaided communication.
 During the first training phase, Treatment S (S = symbol),
students were taught the meanings of the symbols in their picture
dictionaries in a school setting.
 During the second training phase, Treatment M (M =
mechanism), students were taught the mechanics of using their
picture dictionaries (locating symbols, copying, ripping out the
note paper, handing it to a communication partner)
 During the generalization phase data were recorded on (a) the
forms of communication each student elected, (b) the content of
the message, and (c) whether the communication was an
initiation or a response.
Discussion
 Using picture dictionaries for note writing can be an effective form of
expressive communication at the job site for students with deafness and
mild to borderline mental retardation.
 The use of picture dictionaries resulted in fewer missed opportunities to
communicate.
 Increased ability to communicate on for specific work-related items
 Change in the content of messages with and without picture
dictionaries.
 One drawback is that such use assumes literacy skills on the part of the
communication partner, which may not always be the case.
 Another problem is that communication partners may assume that the
writer has a higher level of literacy than is actually the case.
Discussion
 This form of communication was readily
understandable by communication partners who had
literacy skills and enabled the students to communicate
effectively across a range of job-related and social
activities.
 The picture dictionaries also appeared to be more
motivating than symbol-based communication boards
for these students, which may have contributed to their
success.
Paper #3: Ganz et al, 2008
Generalization of a pictorial alternative communication
system across instructors and distance. Ganz et al, 2008,
Augmentative and Alternative Communication
 Investigated use of a modified Picture Exchange Communication
System (PECS) protocol to teach AAC-supported functional
communication skills to a 12-year-old boy with autism.
Introduction
 Nonverbal individuals with autism spectrum disorders (ASD)
often require the use of picture-based, aided augmentative and
alternative communication (AAC) systems.
 The Picture Exchange Communication System (PECS) (Bondy
& Frost, 1994) is a widely employed example of a picture-based,
aided AAC system.
 Phases I and II
 During Phase I, the child is taught to exchange a picture with an
adult to receive an item
 During Phase II, the child is taught to locate his or her
communication book, remove a picture from it, and walk to the
appropriate communicative partner.
 Improves functional communication skills, speech skills and play
How to increase the use and ease of use of
AAC systems
 (a) giving individuals choices about which AAC devices
(e.g., voice-output-communication-aids versus picture
selection methods) they prefer
 (b) organizing pictures according to taxonomic categories
 (c) decreasing the physical effort or response efficiency
required to use the AAC device
 (d) individualizing aspects and instruction of AAC use in
terms of vocabulary, cognitive, and communication
abilities; needs in varied environments; and motor skills
Generalization of AAC-supported
communication skills
 (a) generalization of AAC use into community settings
 (b) generalization of AAC use from clinical to natural
environments
 (c) use of video-based instruction to teach generalization of
AAC use (to fast-food restaurants)
 (d) generalization of writing skills to less structured
activities in children who use AAC systems
 Little research has investigated the generalization of these
devices to a variety of communicative partners and under a
variety of conditions
Research questions
 (a) How quickly would the participant (a Ryan, 12-year-old youth with
autism) generalize the use of a pictorial AAC system across three adult
instructors when those instructors were more than 10 feet away (Phase
1)
 (b) Would the participant spontaneously generalize the use of his AAC
system to a novel condition (communication binder 10 or more feet
away and reinforcer within reach) when taught in a more complex
condition (communication binder and reinforcer 10 or more feet away)
(Phase 2)
 (c) Once the participant had been instructed to use his communication
device in a number of conditions in which he was able to reach his
communication binder, would instruction be required or would the
participant be able to spontaneously determine a method of
communicating the need for help retrieving his communication binder
and reinforcers when they were placed out-of-reach (Phase 3)?
Participant
 Ryan was a 12-year-old boy with autism.
 His vision, hearing, and fine and his gross motor skills within the






normal range
Could play independently for 10– 20 min with a preferred toy or
activity
Ryan did not speak, but occasionally vocalized speech-like sounds.
He communicated primarily by pointing, using a few manual signs and
approximations (e.g., CRACKER, DRINK)
Ryan responded appropriately to several simple verbal and gestured
commands, including Sit down, and, Get your chair.
Ryan was able to use voice-output AAC devices
He mastered the skill of picture correspondence; that is, matching
numerous pictures with actual items.
Baseline: Phase 1
 Generalization across near and far instructors.
 Instructor A (the first author) stood near Ryan (i.e., within





arms-reach),
Instructor A stood far away from Ryan (10 feet or more
away)
Instructor B near
Instructor B far
Instructor C near (instructors varied),
Instructor C far.
Baseline: Phase 2
 Generalization of picture exchanges when the participant’s




AAC device and preferred items were far
Communication binder near Ryan and reinforcer near
Communication binder near and reinforcer far (10 or more
away)
Communication binder far but within-view and reinforcer
near
Communication binder far, but within-view and in-reach
and reinforcer far
Phase 3: Generalization
 To determine how the participant would spontaneously





communicate the need for help when his communication binder
and/or reinforcer was placed out-of-reach.
Communication binder near (within arms-reach) and reinforcer
far and out of reach
Communication binder far (approximately 10 feet or more away)
and reinforcer far and out-of-reach;
Communication binder far and out-of-reach and reinforcer near
Communication binder far, within-view but out-of-reach, and
reinforcer far
Communication binder far, however within-view and out-ofreach and reinforcer far, within- view but out-of-reach.
Results and Discussion
 Picture-based interventions provides a means for students with
ASD to spontaneously initiate their wants and needs while
utilizing a pictorial representation of a preferred item.
 This study demonstrated that a second instructor may be
excluded or used briefly, thus saving valuable resources.
 This study contradicts previous research that has found that
children with autism often have difficulty generalizing
communication skills to novel situations.
Effective instructional strategies in
producing generalization of skills
 (a) training of sufficient exemplars, in which generalization
instruction is planned in a variety of contexts with a variety of
materials to ensure all possible variations are addressed;
 (b) teaching skills that will naturally be reinforced by others in
community settings (e.g., the appropriate words to use to request
toys from peers);
 (c) using common stimuli across a variety of contexts or settings,
and
 (d) mediated generalization, in which instructors teach responses
that will likely be required in a variety of contexts, thus
promoting generalization in those other contexts
Two obstacle arise when using aided-AAC
 Learning to approach the appropriate listener:
 The appropriate listener was defined as the person holding the
preferred item.
 Retrieving an inaccessible AAC device:
 Indicate to listeners that they need to obtain their AAC devices when
they are not accessible. Ryan did this by using behavior indication
(e.g., reaching, leading)
Thank you.