Transcript Document

Extinction of Conditioned Behavior
Effects of Extinction
 the rate of responding decreases
 response variability increases
 experiment by Neuringer, Kornell, & Olufs (2001)
Extinction does not eliminate the original learning
 Spontaneous Recovery
 Renewal
 Reinstatement
 US-devaluation
What is learned during extinction?
The non-reinforcement of a response in the presence
of a specific stimulus produces an inhibitory S-R
association that suppresses that response whenever S is
present
Partial Reinforcement Extinction Effect
Extinction is slower if a partial reinforcement schedule,
rather than CRF, was in effect before extinction
Possible explanations:
 Discrimination hypothesis
 Frustration hypothesis
 Sequential memory hypothesis
Stimulus Control of Behavior
 Stimulus control
 Stimulus discrimination
 Generalization Gradient
Stimulus and Response Factors in Stimulus Control
 sensory capacity
 relative ease of conditioning various stimuli
• overshadowing
 type of reinforcement
• certain types of stimuli are more likely to gain
control over the instrumental response with positive
reinforcement than with negative reinforcement
• experiment by Foree & LoLordo (1973)
 type of instrumental response
• experiment by Dobrzecka, Szwejkowska, &
Konorski (1966)
Foree & LoLordo (1973)
2 groups of pigeons trained to press a foot treadle in the
presence of a LT compound stimulus
For 1 group of pigeons, reinforcement was food
For the other group of pigeons, reinforcement was the
avoidance of shock
Tone
Light
Tone + Light
15
Mean
10
responses
5
Food
reinforcement
Shock-avoidance
reinforcement
Foree & LoLordo (1973)
Pigeons trained with food reinforcement responded much
more when tested with the light alone than when tested
with the tone alone
Pigeons trained with shock-avoidance reinforcement
responded much more when tested with the tone alone
than when tested with the light alone
Selective association?
Dobrzecka, Szwejkowska, & Konorski (1966)
Group 1
(right/left discrimination)
Training
Group 2
(go/no go discrimination)
Buzzer
Metronome
Buzzer
Metronome
Raise
left leg
Raise
right leg
Raise leg
(go)
Do not
raise leg
(no go)
Metronome
Buzzer
Metronome
Buzzer
Raised
left leg
Raised
right leg
Did not
raise leg
Raised
leg
Testing
Dobrzecka, Szwejkowska, & Konorski (1966)
Two components to the auditory cues:
- quality
- location
Dogs trained on the left/right discrimination responded
mainly on the basis of the location of the auditory cues
Dogs trained on the go/no go discrimination responded
mainly on the basis of the quality of the auditory cues
Selective association?
Stimulus elements versus configural cues
So far, we’ve assumed that subjects treat stimulus
elements as distinct and separate features of the
environment (i.e., quality and location of sounds)
However, subjects could treat a complex stimulus
as an integral whole that is not divided into parts or
elements
Configural-cue approach
Stimulus-element approach
Learning Factors in Stimulus Control
Effects of Training on Generalization and Discrimination
 Nondifferential Training :
- S+ always present.
 Differential (or Discrimination) Training:
- Presence/Absence Training:
* reinforced in the presence of S+, not in its
absence.
- Intradimensional Training:
* reinforced in the presence of S+ and not
reinforced in the presence of S-
Non-Differential
Flat gradient
Presence/Absence
More peaked gradient
Less generalization;
more discrimination
Intradimensional
Most peaked gradient
Least generalization;
most discrimination
What is learned in discrimination training?
 respond during the S+
 do not respond during the S-
 or both (i.e., Spence)
Contextual Cues and Conditional Relations
Control of behavior by discrete stimuli (i.e., L or T)
- discrete cues are those which are present for a
brief time and have a clear beginning and end
Contextual cues are the visual, auditory, and olfactory
cues of the room or place where the discrete stimuli are
presented
Contextual cues can provide an additional source of
control of learned behavior
Control by Contextual Cues
Conditioned Place Preference
barrier
Rats given drug
and put in Black
side
Rats given saline
and put in White
side
On test:
Barrier
removed
Rats placed in the center
On test:
Rats spend more time in drug paired context
Contextual cues can control behavior if they serve as a
signal for a US or reinforcer
- experiment by Akins (1998) described on p. 243
Do contextual cues control behavior when they serve as
background rather than explicit signal for reinforcer?
Experiment by Thomas, McKelvie, & Mah (1985)
Trained pigeons on a line-orientation discrimination in context 1
Context 1:
S+ = vertical line (90º)
S- = horizontal line (0º)
After the discrimination was learned, the context was changed and
the discrimination training contingencies were reversed
Context 2:
S+ = horizontal line (0º)
S- = vertical line (90º)
After learning the second problem, pigeons given generalization
tests in which lines of orientation between 0º and 90º were presented
in the two contexts
Results:
Context 1
Context 2
S+
90
0
S0
90
30
Responses
20
10
0
15
30
45
Line angle
(degrees)
60
75
90
Thomas et al (1985) showed that:
 contextual cues can come to control instrumental
behavior
 contextual control can occur without one context
being more strongly associated with reinforcement
than the other
 direct associations between context and reinforcer
cannot explain the pattern of results
 each context associated with a different S+/Scontingency
 Conditional Relations
Control by Conditional Relations
Binary relations:
 S-R
 S-O
 R-O
In some cases the nature of a binary relation is
determined by a third event = modulator
 S-(R-O)
This is termed a conditional relation
S+/S- discriminations are conditional relations
Conditional Control in Pavlovian Conditioning
Rescorla, Durlach, & Grau (1985)
Nonreinforced trials
Reinforced trials
No Noise
Noise
Key light – no food
Key light - food
Element
Compound
80
Responses
per min
60
40
20
Trials
Conditional Control in Pavlovian Conditioning
In instrumental conditioning, modulators (i.e., S+ and S-)
are called discriminative stimuli
In Pavlovian conditioning, modulators are called
facilitators or occasion setters
 A positive occasion setter signals that a CS-US
contingency is in effect
 A negative occasion setter signals that a CS-US
contingency is not in effect
Positive Occasion Setting
(Facilitation)
Training
If Occasion Setter is present:
then CS 1 US
If Occasion Setter is absent:
then CS 1 no US
Result
Thus CS 1 CR only when Occasion
Setter is present
Distinction between excitation and
modulation
1. CS signals US; BUT occasion setter signals
the relationship (if - then) between CS and US.
2. CS elicits CR; BUT occasion setter does not
elicit a CR
3. CS does not facilitate responding to a new
CS+; BUT an occasion setter can.
4. CS can support second order conditioning,
BUT occasion setter does not.
5. CS presented alone results in extinction of CR;
BUT occasion setter presented alone, no
extinction of CS " CR
Negative Occasion Setting
Training
If Occasion Setter is present:
then CS 1 no US
If Occasion Setter is absent:
then CS 1 US
Result
Thus CS 1 CR only when Occasion Setter is
absent – is this familiar?
Conditioned Inhibition may be opposite of facilitation,
not excitation