Transcript Static Scene Cues

Use Of Electronic Media To Promote
Acquisition of Action Verbs
and Prepositions
Created in:
Autism Language Program,
Children’s Hospital Boston
Howard Shane, Ph.D.
[email protected]
www.aac-rerc.com
This work is funded by the National Institute on Disability and Rehabilitation
Research of the U.S. Department of Education, under grant number
H133E030018. The opinions contained in this presentation are those of
the grantee and do not necessarily reflect those of the U.S. Department of
Education.
Acknowledgement
Contributors to this presentation
• Patients and parents in the Autism Language Program
• Monarch School students, parents, and staff
• Model Autism Program, Boston Public Schools
• Sharon Shaham, James Sorce, Meghan O’Brien, Marie
Duggan & John Costello (Center for Communication
Enhancement, Children’s Hospital Boston)
Today’s Agenda
1. Overview of Teaching Language Concepts Program (TLC)
•
•
Why is learning language concepts difficult for some persons with
ASD?
Why are visuals needed?
2. Three Phases of Instruction
•
•
•
Dynamic Scene Cues
Static Scene Cues
Element Cues
3. Developing Effective and Efficient Implementation
Procedures for Teaching Element Cues
•
•
Using typically developing children
Adapting and applying findings to individuals with autism
Overview of VIP
Goal
Utilize visual supports to teach learners on the autism spectrum how to
better communicate during everyday home, school and community
interactions.
Key ingredients
1. Visual language for comprehension and expression
2. A ‘closed language’ with restricted pragmatic opportunity -- targeting
basic communicative operations
3. Mentors and learners use the same visual symbols
4. Use of advanced computer and video technologies
What is Visual Immersion?
A symbol-rich environment
An environment where visual symbols clarify spoken
language, support expressive communication, assist
with transitions, and facilitate learning.
• Used in home, school, and community
• Easy access to
– Non-electronic symbols
– Electronic Screen Media (Monitors and TVs)
What Does VIP Target?
•
Seven concrete Communicative Operations :
1. Protests/Refusals
2. Requests
3. Directives
4. Comments
5. Questions
6. Social conventions
7. Transitions
•
VIP is not intended to teach advanced communicative operations:
–
–
–
–
–
•
Abstract concepts: “…with liberty and justice for all.”
Passive voice: “The book was read by the boy.”
Complex syntactic structures: “If he hadn’t checked the weather in the
morning, then he would have forgotten to bring his umbrella.”
Figurative language: “She flew to the bookstore.”
Humor : “Why did the chicken cross the road?”
VIP provides foundation for more advanced language-based
intervention programs
VIP: Three Instructional Settings
1. Natural setting:
Familiar, everyday people,objects, actions, locations and events as
opportunities to overlay language. Takes advantage of adding language
to already learned behavioral skills:
1. Daily living/self-help routines
2. Operational knowledge
3. Preferred play routines
2. Tabletop setting:
A controlled environment involving one-to-one instruction
3. Virtual setting:
Electronic (television or computer) display for presentation of materials
Setting selection varies as a function of child and the skill to be learned.
Overview of Teaching Language
Concepts Program (TLC)
Why Teach Language Concepts?
Autism Language Program (ALP):
•
Population: Moderate to severe autism (from initial diagnosis to adulthood)
•
Core deficit: Difficulties with language comprehension and expression, in
addition to pragmatic language skills
– Comprehension:
• Children on the spectrum often experience difficulty comprehending
spoken language and/or auditory processing impairment (Novick et al.,
1980; Boddaert et al., 2003).
• Many children are able to follow routine-based, familiar directives, but
have not yet demonstrated comprehension of more abstract linguistic
concepts (e.g., verbs, prepositions, attributes)
– Expression:
• 50% of individuals with autism spectrum disorders do not use speech
functionally (Wetherby and Prizant, 2005; Lord & Paul, 1997; Rutter, 1978)
• Many children are able to reliably use PECS to request and label, but have
not yet developed the ability to describe, comment, or ask questions
Model of Language Comprehension
(Typical Development)
Spoken
Language
Short-Term
Memory
Language
Processor
Comprehension
Model of Language Comprehension
(Moderate-Severe ASD)
Verbal
Information
Short-Term
Memory
Language
Processor
Comprehension
= Clinical insight suggests these impairments are due to:
•Attention to auditory stimuli
•Not understanding language as meaningful/symbolic
•Fleeting nature of spoken language
•Language processor ‘broken’
•Combination of these
Language comprehension is often compromised
• Occasional comprehension of noun labels and familiar directives (in context)
• Impaired comprehension of relational linguistic concepts (e.g., verbs, prepositions,
attributes)
‘Teaching Language Concepts’ (TLC)
Program
• A visual instruction system for hard to learn language
concepts (e.g., verbs, prepositions and attributes).
• TLC is a closed visual language
• Instruction in the virtual environment (video clips), tabletop
environment (photographs, toy figurines and miniature
objects), and natural environments (home, school and
community)
Three Phases of TLC Instruction
TLC Instructional Phases:
•
Learner progresses through all/some of three phases of
visual language symbols, from concrete to abstract
representations:
1. Dynamic Scene Cues: full-motion video clips of
action scenes
Bypassing Spoken Language Comprehension
Using Dynamic Scene Cues
(Moderate-Severe ASD)
Verbal
Information
Short-Term
Memory
Imitation
(as inferred from
dynamic scene cues)
Visual
Information
(dynamic scene cues)
*Note: No involvement of language processor required
for comprehension of dynamic scene cues
TLC Instructional Phases:
•
Learner progresses through all/some of three phases of
visual language symbols, from concrete to abstract
representations:
1. Dynamic Scene Cues: full-motion video clips of
action scenes
2. Static Scene Cues: photographs that capture a
prototypical moment in the action scene
Bypassing Spoken Language
Comprehension Using Static Scene Cues
(Moderate-Severe ASD)
Verbal
Information
Short-Term
Memory
Imitation
(as inferred from
static scene cues)
Visual
Information
(static scene cues)
*Note: No involvement of language processor required
for comprehension of static scene cues
Implications of Static Scene Cue Mastery
Use of dynamic and static cues bypasses language
processor
– Due to load on language processor when using
elements, some children may use static scenes as their
communication system
Mastery of static scene cues is a significant
accomplishment
– Can be used to promote general understanding and
communication related to:
•
•
•
•
•
•
•
Daily Living Skills
Play
Transitions
Requesting
Commenting
Clarifying
Directives
TLC Instructional Phases:
•
Learner progresses through all/some of three phases of
visual language symbols, from concrete to abstract
representations:
1. Dynamic Scene Cues: full-motion video clips of
action scenes
2. Static Scene Cues: photographs that capture a
prototypical moment in the action scene
3. Language Element Cues: graphic icons
representing each of the individual linguistic
components that comprise an action scene (e.g.,
subject, object, verb, preposition, adjective, etc.)
TLC Phase 3: Element Cues
Symbolate:
Language Comprehension?
• Stringing symbols together does not automatically result in
comprehension.
• Learners must first have a knowledge of language
elements and semantic relationships, as provided in VIP.
• Paradoxically, stringing symbols together may actually
interfere with comprehension.
Model of Visual and Spoken Language
Comprehension
(Moderate-Severe ASD)
Comprehension of
Verbal
Representation
Verbal
Information
Short-Term
Memory
Visual
Information
(element cues)
Language
Processor
Comprehension of
Visual
Representation
(element cues)
Clinical Observations About
Element Understanding
• Some label elements without meaning attached (echolalic-like)
• Some comprehend agent and object elements -- struggle with relational
elements
• Some comprehend isolated elements -- difficulty interpreting element
strings
What improves element comprehension?
• Immersion in symbol-rich environment
• Mass trials across multiple settings and communication partners
• Computer-based instruction
• Persistence over extended period
Bottom line: A more effective, efficient way of teaching language
elements must be developed.
TLC 3 Phases: Clinical Observations
• Progressing from
– Dynamic Scene Cues to Static Scene Cues -- is easy
• No involvement of language processor required
– Static Scene Cues to Element Cues -- is difficult
• Requires involvement of language processor
But
• The payoff is worth it -- foundation for generative
language
– Goal: Combine language element cue vocabulary with
their understanding of semantic relationships to generate
novel sentences for expressing requests, comments,
replies, etc. in natural settings.
The Empirical Challenge
• Can we better understand the process?
• Can we expedite the process?
Developing Effective and Efficient
Implementation Procedures for
Teaching Element Cues
Top Down Approach
Example of Mixed Display - with Referencing
The TLC Approach
Strategy: First study typical children, then apply findings to children with
autism
Rationale:
– Not yet literate
– Able to verbalize thought process
– Compliance
Subjects: Ages 3 yrs, 11 mos to 6 yrs, 4 mos
Procedure: General task - sequence three linguistic elements: left-toright reading order
– agent + action + object
– agent + preposition + object
– Experimenter provides no spoken language to label elements
– Learner demonstrates comprehension of element string by acting
out directives
Multiple design-prototype-test cycles to refine TLC instructional
approach
Iteration 1: Top-Down Approach - Sequentially
Through Three Instructional Phases
Iteration 1 Procedure
• Learner presented with
– Dynamic scene cues; imitate action
– Static scene cues; imitate action
– Element cues; demonstrate comprehension by
acting out directives
•
For each instructional phase, learner viewed the
visual cues and was provided with spoken
prompt, “Now you do it!”
Iteration 1: Top-Down Approach Sequentially
Through Three Instructional Phases
Outcome:
– Able to accurately imitate dynamic and static cues
– Difficulty understanding task when presented with
element cues (often attempted to match objects to
elements rather than carry out directive)
– Difficulty comprehending left-to-right sequence of
‘reading’ elements
– Difficulty understanding meaning of verb/preposition
(abstract) symbols
– Difficulty comprehending de-contextualized element
strings.
– Tendency to act on objects in a familiar manner (e.g.,
‘man on ladder’ rather than ‘man climbs ladder’)
Iteration 2: Modified
Top-Down Approach
Modifications to Iteration 1 Procedure:
– Introduced visual template to focus attention on leftto-right order of elements, and presented elements
in left-to-right sequence
– Introduced Mixed Display (Static scene cue along
with Element cues) and demonstrated association
between scene cue and its elements
– To enhance understanding of task expectation:
• Presented task as ‘game’ to discover meaning of
the relational (verb or preposition) symbol
• Experimenter modeled the task
Iteration 2: Visual Template
Iteration 2: Modified
Top-Down Approach
Outcome:
– Initially learners did not appear to attend to the
relational element -- verb or preposition (e.g., “You
don’t need that one”)
– Still initially attempted to match the agent and
object elements to their physical objects
– Given repeated trials, demonstrated
comprehension of the individual elements, but
continued to experience difficulty with the left-toright ‘reading’ order for elements
Iteration 3: Bottom-Up Approach
Modifications to Iteration 2 Procedure
Instruction started at element rather than dynamic phase
• Presented elements on computer monitor
– Experimenter modeled the task
– Guided discovery to select combination of element
strings to play corresponding video clip
• Then moved to tabletop
– Presented non-electronic task with identical element
cues and physical objects;
– Experimenter modeled the task
– Learner encouraged to create element strings to direct
the actions of communication partners
Software Applications
Learning Together with Music
Puddingstone Place
Iteration 3: Bottom-Up Approach
Iteration 3: Bottom-Up Approach
Outcome:
– Older learners were able to execute each directive
presented and label each element
– Younger learners had difficulty attending to screen and
task; more likely to engage in their own independent
play with the materials
Iteration 4: Modified Bottom-Up
Approach with Compelling Scenes
• Modifications to Iteration 3 Procedure:
– Created ‘fun’ scenes depicting engaging, motivating
activities using human models and life-sized objects
– Clicked on message window to activate video clip of
corresponding element string (in hopes of enhancing
attention to relational element)
– Presented learner with an ‘agent’ element cue
representing him/herself
– Encouraged learner to use elements to direct his/her
own play and actions of communication partners
Iteration 4: Modified
Bottom-Up Approach
Iteration 4: Modified Bottom-Up
Approach with Compelling Scenes
Outcome:
– Actively engaged
– Successfully combined elements - although still initial
tendency not to attend to relational element (verb or
preposition)
– Prepositions more difficult than verbs
Iteration 5
Modifications to Iteration 4 Procedure:
– Focus attention to relational element (verb or
preposition):
• Animated relational element
• Click on relational element to play video
Key Ingredients to Teaching
Element Cues
• Present elements electronically
• Guided discovery with element combinations (to play
video)
• Provide children with symbols representing themselves
and familiar communication partners to allow them to
direct the activity
• Begin instruction with fun, motivating tasks. Gradually
expand to language-arts activities.
• Animate center elements to direct attention to element and
enhance likelihood of comprehension
Directions for future TLC research
• Improve symbols
– Animated symbols (verbs, preposition, attributes)
• Generalization of language skills targeted in TLC to natural
setting
• Facilitating generative language