Transcript Stimulus Control

Stimulus Control
Stimulus Control of Behavior
• Having stimulus control means that the probability of the
behavior varies depending upon the stimuli present.
• If a behavior is under stimulus control then
– The behavior happens when the stimulus is present
– The behavior doesn’t happen when the stimulus is absent
• Most of our behavior is under stimulus control
– A person that contributes to charity generously while in church
may watch every penny spent while at work
Discrimination
1.2
CS+
CS-
Response Strength
1
0.8
0.6
0.4
0.2
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Trials
Stimulus Control
• Stimulus Discrimination
– Definition: Degree to which antecedent stimuli set
the occasion for particular responses
• precise degree of stimulus control
• E.g., Man has beard and is Daddy; Man has beard and
is Uncle Eddie
– Stimulus discrimination is taught by using
discrimination training procedures such as
differential reinforcement
Stimulus Control
• Stimulus Generalization
– Definition: When a response is reinforced in the
presence of one stimulus there is a general tendency to
respond in the presence of new stimuli that have similar
physical properties ore have been associated with the
stimulus.
• loose degree of stimulus control
• E.g., All men with beards are Daddy
– Stimulus discrimination is taught by using
generalization training procedures, reinforcing
responses to many similar stimuli.
Development of Stimulus Control
• Stimulus discrimination training requires
– One behavior
– Two antecedent stimulus conditions (the SD and the S)
• Responses that occur in the presence of the SD are reinforced
(thus, the response increases in the presence of the SD)
• Responses that occur in the presence of the S are not
reinforced (this, the response decreases in the presence of the
S
– Can also result in a lesser amount or quality of reinforcement
Development of Stimulus Control
• Example: Reinforcing a child’s saying “red”
when someone asks “What’s your favorite
color?” and witholding reinforcement if
they said “red” when asked “ What’s your
name?”
SD:
“What’s your
favorite color?”
Response:
“red”
SΔ:
“What’s your
name?”
Reinforcer:
“Super! You
said red!”
No praise
Differential Responding
• When a child’s behavior comes under the control
of the SD,
– We say the SD has acquired stimulus control over the
child’s behavior
– So the verbal stimulus “What’s your favorite color?”
gains stimulus control over the response “red” – why?
– And the child is discriminating or responding
differentially
Stimulus Generalization
• When a response is reinforced in the
presence of one stimulus,
• But: that same type of behavior
tends to be evoked by stimuli that
share similar physical properties
with that controlling antecedent
stimulus
Stimulus Generalization
If you teach “green” using this color circle …
Student’s less likely
to say “green”
(discrimination)
Student’s
most likely
to say
“green”
Student’s less
likely to say
“green”
(discrimination)
Stimulus Generalization
• Stimulus Generalization is more likely
when:
– Stimuli that are highly similar to the original
stimulus are presented.
– E.g., similar people, materials, or settings
• The more similar the novel person,
materials, and setting are to the training
person, materials, and setting…
Discrimination vs. Generalization
• Essentially opposite processes
– As discrimination increases, generalization decreases
– As discrimination decreases, generalization increases
• Discrimination
– Responding differently to 2 or more stimuli
– Tight degree of stimulus control
• Generalization
– Responding similarly to 2 or more stimuli
– Loose degree of stimulus control
Stimulus Discrimination and Stimulus
Generalization are a Continuum
Stimulus
Discrimination
Stimulus
Generalization
•Generally, as the training and test stimuli become more different
responding will decline,
• Produces a generalization gradient
Generalization Gradient
Guttman & Kalish (1956)
– pigeons reinforced for pecking a
580 nm lit key (orange-yellow)
(S+) on a VI schedule
350
250
– A test session was then given
where many different colored key
lights were presented in
extinction
200
150
100
50
S+
0
63
0
61
0
59
0
57
0
55
0
53
0
0
51
Responses
300
Wavelength (nanometers)
Interpreting Generalization Gradients
Pigeons trained to
peck a moderately
bright light (S+) to get
food.
(S- = dim light)
After asymptote is
reached, present
occasional nonreinforced probe trials
at various wavelengths
or levels of brightness.
Excitatory and
inhibitory gradients
Excitatory gradients form
around the CS+; show
where responding is MOST
likely
Inhibitory gradients also
form; show where
responding is least likely
e.g., Pigeons trained to peck
at a 800 hz tone (S+), with a
500 nm light S-.
Peak Shift
• Phenomena where the peak of the
generalization curve shifts AWAY from the S– Means that the most responding does not occur
for the S+
– But slightly offset from the S+ and away from the
S– Question is WHY!
Peak Shift Effect – Hanson (1959)
Spence’s Theory
• Spence: Peak shift occurs because of the
summation of the excitatory and inhibitory
curves
– Result is that the most responding is slightly in
favor of the S+
– Not as strong of suppression for S– Thus, get shift away from S– Just a math phenomenon
Inhibitory or Excitatory Strength
Spence’s Theory to Account for Peak Shift
80
S+
Inhibitory
Excitatory
Difference
60
40
20
0
-20
S-
-40
-60
490 510 530 550 570 590 610 630 650 670
Wavelength (nanometers)
Interdimensional theory
• Kohler’s Transposition or Interdimensional theory
– The animal learns a conceptual rule
– “Not” the S- but is the S+
– Thus, when given generalization stimuli, take
whatever is the most “not” S– Shift away from S+ occurs
• Learning a concept: always choose larger or
greener, etc.
Interdimensional discrimination
700
Pseudodiscrim
Discrimination
# Responses
600
500
400
300
200
100
0
501
530
555
576
Wavelength
Discrimination: S+ = 555nm Light; S- = Tone
606
Test of Theories
• Choose between squares:
•
S+
100 cm2
250 cm2
500 cm2
1000 cm2
S60 cm2
150 cm2
250 cm2
500 cm2
Situation:
original training
Test 1: Spence’s test
Test 2” Kohler’s test
??????
• Spence's prediction: will choose closest to original S+
• Transposition or interdimensional: choose larger
Which is correct?
• tests for transposition yield evidence
supporting transposition
• tests for generalization yield evidence
supporting Spence
• probably: both effects working part of the
time
What is a stimulus
• Any sensory event that elicits a response
– Not necessarily the response you want
– Need to know that that rat perceives the stimulus
• Establishing a cue
– Classically condition (cue to cue)
– Operant conditioning: response to reinforcer, then
add predictive cue for contingency
Why Teach Discriminations?
• Discrimination is a fundamental process that
controls behavior.
• Discrimination allows us to differentiate when
reinforcement is available for specific
responses.
Why Teach Discriminations?
• Many essential tasks require discrimination
skills
– Reading.
– Labeling Objects.
– Following directions.
– Following activity schedules.
– Greeting people.
– Self-care skills.
Basic Problem of Discrimination
Training
• Discrimination training teaches learners to
perform a specific response in the presence
of a specific stimulus and not to perform that
response in the presence of other stimuli.
Discrimination Training
• Many individuals with developmental disabilities have
difficulty learning discriminations
• Stimuli are compound (they consist of different
elements)
– It may be difficult to control which element(s) of the
stimulus exert control over behavior
– We must be careful how we teach and what stimuli
we use!
Stimulus Compounding
• All stimuli are compound.
– They consist of many different elements.
• It is difficult to control which element or
elements of the stimulus exert control over
behavior.
– need to be careful how we teach and what stimuli
we use.
2 Types of Discriminations.
• Simple Discriminations.
– Non-Conditional Discrimination.
• Conditional Discriminations.
– Matching-to-sample.
– Arbitrary Discrimination.
Simple Discrimination
• Stimulus is present or not present
– One picture on the table that’s a “cat” Say touch
“cat” and they do
– Not very useful for children with autism
– That kind of trial becomes stimulus independent
which is what you DON”T want to happen
Conditional Discrimination
•
A response to a given stimulus is followed by a reinforcer if and only if another
stimulus is present
•
Said another way…a stimulus is discriminative for reinforcement or not, depending
on (conditional on) the presence of another particular antecedent
•
Often taught via Match-to-Sample procedures
– Many programs consist of Identity Matching
– matching identical stimuli:
• Objects
• Pictures
• Letters
• Shapes
• Colors
• Numbers
Match-to-Sample
SD/S
+
SΔ/S-
SΔ/S-
Then responds to one
of the comparison
stimuli
Learner 1st responds to
sample (conditional
stimulus)
Sample
Match-to-Sample
SD/S
SD/S+
+
Sample (Conditional
Stimulus)
Sample (Conditional
Stimulus)
Establishing New Forms of
Conditional Stimulus Control
• Identity matching (single mode)
– Visual: visual
– Auditory: auditory
• Arbitrary matching (multimodal)
– Visual: auditory
– Auditory: visual
– Auditory: tactile
Arbitrary Matching
• Stimuli are not physically identical
• Examples
– Match non-identical visual stimuli
• Object to pic, printed word to picture, object to printed
word
– Match auditory stimulus to visual stimulus
(“Receptive Identification”)
– Others?
Receptive Picture ID
SD/S
+
SD/S
+
“baby”
“flower”
Sample
Sample
Discrimination in the Laboratory.
• Many laboratory discrimination procedures have focused
on understanding exactly which stimuli or which aspects
of stimuli control responding.
• Several laboratory procedures have been developed that
allow us to isolate these stimulus control factors.
– Matching-to-Sample.
– Equivalence Class Formation.
Concept Formation
• Definition: Complex stimulus control that
results in generalization within a class of
stimuli and discrimination between classes of
stimuli (Keller & Schoenfeld, 1950).
Two Procedures to Teach Concept Formation:
• Simultaneous Discrimination:
– Definition: The concurrent presentation of multiple stimuli
• Successive Discrimination
– Definition: The successive presentation of stimuli
• In applied settings, simultaneous discrimination training
typically used
– The individual has access to all relevant stimulus cues at 1 time
– An on the spot comparison of choices is permitted
– Produces better generalization (Schroeder & Baer, 1972)
Real World Teaching Procedures.
• Real World teaching is more complex than the laboratory.
• Suggestions for improving discrimination training
– Evaluate stimuli carefully to promote control of relevant
antecedent cues
– Rotate stimulus placement randomly
– Change auditory stimuli frequently
– Be sure that the S+/S- functions of stimuli change
randomly
– Teach with a variety of stimuli to promote discrimination
between categories (and generalization within categories)
• Never, never teach in isolation!!
Stimulus Class
• Stimulus Class: set of stimuli that have a common
effect on behavior (evoke the same response
class)
These would probably be in the
same stimulus class for most people
Concept Formation
• Complex stimulus control that results in:
–Generalization within a class of stimuli and
–Discrimination between classes of stimuli
Stimulus Equivalence
• Emergence of accurate responding to
untrained and nonreinforced stimulusstimulus relations
– Following the reinforcement of responses to
some stimulus-stimulus relations
A
• 3 aspects
– Reflexivity
– Symmetry
– Transitivity
B
DO
G
C
“dog”
Reflexivity
• In the absence of training, person selects an
identical stimulus (AKA Generalized identity
matching)
• Logic is…A = A
• For example
– Without specific training, the person matches
Symmetry
• After being taught to match A to B, person can
match B to A (without training)
• Logic is…If A = B, then B = A
• For example
– After being taught to match
– Person can match
DOG
to
to
DOG
Transitivity
• Critical test for stimulus equivalence – if you get transitivity, the
stimuli are members of an equivalence class
• After being taught to match A to B and B to C, person can match
A to C (without training)
• Logic is…If A = B and B = C, then A = C
• For example…
– After being taught to match
•
to DOG AND DOG to “dog”
– Person can match to “dog” to
Factors Affecting the
Development of Stimulus Control
• Reinforcement
– Differential reinforcement with rich schedules of
reinforcement
– Reinforce stimuli that are part of concept
– Must learn to attend to specific elements of the stimuli, not
just the whole stimulus
• Preattending Skills
– Type of instructor or materials
– sitting up tall, ready to work (awake and functioning!)
– no stereotypy or other misbehavior
Factors Affecting the Development
of Stimulus Control
• Stimulus Presentation
– Specificity of directions
• Should relate to response definition
– Opportunity to response
• Frequent, active opportunities to respond (active student
learning) correctly leads to higher rates of academic
achievement
– Pacing of response opportunities
• Higher pace often leads to superior performance and less offtask behavior
Factors Affecting the Development
of Stimulus Control
• Salience of the stimuli
– Influences attention and ultimately the development of
stimulus control
– Dependent on the capabilities of an individual, the past history
of reinforcement, and the situation
– Multiple intelligences: some respond to instructional stimuli in
the visual modality more than the oral.
– This research has generated mixed results but it is important to
rule out sensory deficits and assess modality strength
Factors Affecting the Development
of Stimulus Control
• Masking and overshadowing
– Masking –even though one stimulus has control over behavior,
another stimulus blocks that control from being expressed
• E.g., Student knows answer but won’t answer in front of peers
– Overshadowing –the presence of one stimulus condition interferes
with the acquisition of control by another stimulus
• E.g., hallway may be more interesting that teacher’s presentation
in the front of the classroom
• Ways to overcome include to make changes to physical
environment, make instructional stimuli as intense and
centrally located as possible, reinforce behavior in the
presence of the instructionally relevant stimuli
What is a stimulus
• Any sensory event that elicits a response
– Not necessarily the response you want
– Need to know that that rat perceives the stimulus
• Establishing a cue
– Classically condition (cue to cue)
– Operant conditioning: response to reinforcer, then
add predictive cue for contingency
Shaping the cue
• Need to shape the response to the cue
• Introduce the stimulus cue right when the behavior
occurs
– Not as it is starting, not after
– E.g., for a dog to learn “sit”, not when it is walking, but
when it starts to sit.
– BUT: click once the behavior is underway, not when the
behavior is finished
– Then, only introduce the cue when the dog is NOT doing
the behavior
• Give the cue
• Get the behavior
• click
Four rules for stimulus control
• Behavior occurs immediately upon presentation
of the cue stimulus
• Behavior never occurs in absence of stimulus
• Behavior never occurs in response to another
stimulus
• No other behavior occurs in response to the
stimulus
Transferring stimuli
• Once establish one cue, can add a second cue
for the same behavior
– Again, use only for THAT behavior
– Keep stimuli separate.
– E.g., does down mean stop jumping on me or lie
down on the ground?
Prompting
• Prompts used to increase probability that
organism will engage in correct response
• Use during discrimination training
– SD or S+: in presence of this stimulus, do the behavior
– S or S-: in the presence of this stimulus, do NOT do
the behavior
• Function of prompts is to produce instance of
correct behavior so it can be reinforced
Types of Response prompts
• Response prompts: behavior of another evokes desired
response in presence of SD
• Verbal prompts: voice command
• Gestural prompts: physical movement or gesture of trainer
• Modeling prompts: observe model and imitate (not really used
with rats)
• Physical prompts: physically assisting rat to engage in behavior
Types of Stimulus prompts
• Stimulus prompts: change in stimulus or addition/removal of
stimulus increases likelihood of desired response in presence of SD
• Within-Stimulus prompts: change the salience of a prompt
– Bigger or smaller
– Louder or softer
– More or less
• Extra-stimulus prompts:
– Adding a stimulus to increase likelihood of correct discrimination
– Picture prompts
– Putting line of food pellets for rat to follow
Aversive cues
• Use aversive to STOP behavior
• E.g., “no”, a sharp noise, a table slap
correction
• Good for dangerous behaviors
• Potential downside:
– Squelch behaviors
– Organism is startled, now won’t move
– More likely to bite/nip/freeze
Limited holds and anticipation
• Limited hold:
– Reinforcer only available in a narrow window
• Either take it now, or lose it
• Often used to avoid stashing or lazy choice
– If do behavior, can get reinforcer within x seconds or
less, otherwise lose it.
• Anticipatory behaviors occur before the cue
– If occur, cancels the trial
– Withdraw target stick and even put rat in “time out”
Behavior chains
• One behavior can be a cue for the next response
• This is a behavior chain
– E.g., run through the tunnel, over the balance beam and then
through the weave poles
• Can teach forward chain:
– Start with first, then add second, third, etc.
– Reinforce after successful behavior for criterion
• Or backwards chain:
– Start with last response, then next to last, and so on
– Reinforce after successful behavior for criterion
Generalization
• May want organism to engage in behavior in many
settings/situations/cues
• E.g., go over balance beam whether it is on the desk or
in either of the two big boxes
• Stop at any red traffic light
• Must train so that the cues to which the subject
attends are the ones relevant to the task, not the
setting
Fading prompts
• Important to fade prompts
– Want behavior to occur to cue only
– Shows mastery of behavior
• Behavior flows well
• Behavior occurs at high rate (or low rate if that is goal)
• Occurs evenly and consistently
– Fading prompts allows fluency
• Behavior occurs when, where, how well you want it
• Can generalize to appropriate stimuli
• Can discriminate irrelevant stimuli
Fading prompts
• Several ways to fade prompts:
– Transferring stimulus control
• Remove one stimulus cue, then another
– Fading across different types of prompts or fading across
prompts
• Least to most: fade least invasive prompt first
• Most-to-least fading: fade most invasive prompt first
– Prompt delay
• Delay the prompt
• Increase delay until behavior occurs before prompt would have
been given
Herb Terrace: Errorless Learning
• instructional design introduced by B.F. Skinner
• Skinner said: "errors are not necessary for learning to occur.
Errors are not a function of learning or vice-versa nor are
they blamed on the learner. Errors are a function of poor
analysis of behavior, a poorly designed shaping program,
moving too fast from step to step in the program, and the
lack of the prerequisite behavior necessary for success in
the program."
• can also be understood at a synaptic level: principle of
Hebbian learning: "Neurons that fire together wire
together".
Traditional Procedure
• subject learns to associate one stimulus, S+ (positive stimulus), with
reinforcement (e.g. food) and another, S− (negative stimulus), with
extinction (e.g. absence of food).
• For example, red key (S+), green key (S−).
• Traditional procedures: pigeon trained to
– initially trained to peck a red key (S+).
– When responding consistently to the red key (S+), a green key (S−)
would be introduced.
– At first pigeon would also respond to the green key (S−)
– gradually responses to the S- would decrease, because they are not
followed by food, so that they occurred only a few times or even
never.
Terrace Method
• Terrace (1963): found that discrimination
learning could occur without errors:
– training begins early
– the S+ and S− were initially presented with
different brightness and duration,
• S+ would appear for 5 s and fully red
• S− would appear for 0.5 s and dark.
– duration of the S− and its brightness gradually
increased until the keylight was fully green for 5 s.
Example: teach
S+ = red; S- = green
• Teaching red vs. green discrimination
– S- presented for only 5 or so seconds
– S+ presented for 3 minutes
– S- begins as unlit (dark) key: pigeons much less likely to peck it
• S- gradually fades to green key
• human example: teaching MR children sight words
– word + picture on slide
– gradually fade the picture, leaving only the word
RAT
RAT
RAT
RAT
RAT
RAT
RAT
Does it work?
• highly effective in reducing the number of
responses to the S− during training.
• In Terrace's (1963) experiment:
– Ss trained with the conventional discrimination
procedure averaged over 3000 S− error responses
during 28 sessions of training
– Ss trained with the errorless procedure averaged
only 25 S− (errors) responses (same # sessions)
Additional Benefits (Terrace, 1972)
• Errorless learning procedure improves long-term discrimination
performance
• S− does not become aversive and so does not elicit "aggressive"
behaviors, as it often does with conventional training
• S− does not develop inhibitory properties;
• Positive behavioral contrast to S+ does not occur.
• “By-products" of conventional discrimination learning do not occur
with the errorless procedure.
Cognitive Research Supports!
• Studies of implicit memory and implicit learning provided
additional theoretical support for errorless learning
methods
– Brooks and Baddeley, 1976, Tulving and Schacter, 1990.
• Implicit memory
– known to be poor at eliminating errors
– can be used to compensate when explicit memory function is
impaired.
• In experiments on amnesiac patients:
– errorless implicit learning more effective
– reduced the possibility of errors "sticking" in amnesiacs'
memories
Feature Positive Effect
Feature Positive Effect
• Heart and Jenkins, 1974
• Asymmetry in the pigeon's learning of
discriminations based on the presence versus
absence of a single distinguishing feature.
Feature Positive Effect
• several limitations on effectiveness of discrimination
training:
– discrimination will not occur if animal cannot detect
difference between stimuli
– animal won't form discrimination if not attend to proper
stimuli
• the stimulus that an animal attends to is determined
by many factors, including:
• species differences
• salience of stimuli
• past experiences
Feature Positive effect:
Hearst and Jenkins (1974)
• basic experiment:
–
–
–
–
pigeons given 2 stimuli simultaneously
S+ = lighted key w/dark spot
S- = lighted key w/no dark spot
looked at discrimination ratio:
• key pecks to S+/ total key pecks
• thus: perfect dsicrimination ratio = 1.0
• no discrimination = 0.5
S+
S-
– results of initial experiment: pigeons quickly learned to
discriminate
Feature Negative Effect
then: switched stimuli around:
– S+ = lighted key w/NO dark spot
– S- = lighted key w/dark spot
• found NO discrimination in this case
• Why? Feature Negative effect!
Feature Positive Effect
S+
S-
S+
S-
S+
S-
S+
Feature Negative Effect
S+
S-
S+
S-
S+
S-
S+
One more try!
S-
S+
S-
S+
S-
S+
S-
S+
S-
S+
S-
S-
So, what is the concept?
An Unusual Paragraph!
• How quickly can you find out what is so unusual
about this paragraph? It looks so ordinary that you
would think that nothing is wrong with it at all, and,
in fact, nothing is. But it is unusual. Why? If you
study it and think about it, you may find out, but I
am not going to assist you in any way. You must do it
without coaching. No doubt, if you work at it for
long, it will dawn on you. Who knows? Go to work
and try your skill. Par is about half an hour.
A Clue
• The author, Eliot Hearst, could not add his
name to the paragraph and maintain the
concept!