Transcript Chapter 6

Chapter 6:
Operant Conditioning:
Introduction
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Thorndike
• The first experimental studies of operant conditioning
were undertaken by Edwin L. Thorndike in the 1890s.
• He believed that the intellectual ability of animals could
be properly assessed only through systematic
investigation.
• His most famous experiments involved hungry cats that
were enclosed in a puzzle box, while a dish of food was
placed outside.
• To reach the food, the cat had to learn how to escape
from the box.
• The response that worked was strengthened, while the
responses that did not work were weakened.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Law of Effect
• Behaviors leading to a satisfying state of
affairs are strengthened or “stamped in.”
• Behaviors leading to an unsatisfying or
annoying state of affairs are weakened or
“stamped out.”
• The consequences of a behavior
determine whether that behavior will be
repeated.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Skinner
• Skinner believed there were two types of behaviors,
reflexive-type behaviors and operant behaviors.
• Skinner devised his own procedure for the study of
behavior, the “Skinner Box.”
• The rat is able to earn food pellets by pressing a response
lever or bar.
• The experimenter controls the contingencies within the
operant chamber, but the animal is free to respond at any
time.
• Rate of behavior is controlled by the conditions in the box.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Operant Conditioning
• a type of learning in which the future probability
of a behavior is affected by its consequences.
• Behaviors that are influenced by their
consequences are called operant behaviors.
• The response operates on the environment to
produce a consequence.
• Behaviors that lead to favorable outcomes are
more likely to be repeated than those that do not
lead to favorable outcomes.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Operant Conditioning Process
1. a discriminative stimulus that occurs
before the behavior and signals that a
certain consequence is now available
2. a response that produces a certain
consequence
3. the consequence that serves to either
increase or decrease the probability of
the preceding behavior
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Example of the Operant Process
1. a tone signals that a lever press will now
produce food
2. lever pressing produces a food pellet
3. the consequence of a food pellet
increases the rat’s tendency to again
press the lever
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Operant Behavior
• a class of emitted responses that result in
certain consequences.
• These consequences affect the future probability
or strength of those responses.
• They are not elicited by stimuli like in classical
conditioning.
• Rather operant behaviors are emitted by the
organism.
• They are more flexible and voluntary behaviors.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Operant Behavior Examples
Lever press  Food pellet
The effect: The future probability of lever pressing
increases.
Tell a joke  Person frowns
The effect: The future probability of telling a joke
decreases.
Operant behaviors are classes of responses.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Do operant behaviors occur
because we choose to emit them?
• Not necessarily.
• The behavior comes to be controlled by
the contingencies of reinforcement and
punishment that follow the behavior.
• There is sometimes only an illusion of
voluntariness.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Reinforcers
• Consequences of behavior that:
1. follow a behavior, and
2. as a result, the future probability of that
behavior increases.
• Example:
Lever press  Food pellet
R
SR
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Punishers
• Consequences of behavior that:
1.follow a behavior, and
2.as a result, the future probability of that
behavior decreases.
• Example:
Tell a joke  Person frowns
R
SP
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Operant Conditioning Perspective
• It is the behavior that has been reinforced or
punished.
• It is NOT the person or animal that has been
reinforced or punished.
• Only the behavior increases or decreases in
frequency.
• Consequences are defined entirely by their
effect on behavior.
• Reinforcers increase the frequency of behavior,
while punishers decrease the frequency of
behavior.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Reinforcement & Punishment
• the process or procedure by which a certain
consequence changes the strength of a
behavior.
• Example:
– reinforcement = the use of food to increase the
strength of lever pressing
– reinforcer = the food itself
– punishment = the process of frowning to encourage
Jonathan to stop telling jokes
– punisher = the frown itself
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Means of Weakening Behaviors
• Punishment
• Withdrawal of reinforcement – extinction
• Example:
– Suppose a child has learned to whine for
candy in the supermarket.
– When behaving that way no longer results in
candy, the child will cease whining.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Discriminative Stimuli
• a stimulus in the presence of which responses
are either reinforced or punished.
• Example:
Susan: Jonathan tells her a joke  She laughs
SD
R
SR
Police car: You speed  You receive a ticket
SD
R
SP
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Discriminative Stimulus
for Extinction
• The stimulus signals the nonavailability of a
previously available reinforcer.
• Example:
– Lever pressing is followed by food only when a tone is
sounding and NOT when a buzzer is sounding.
Tone: Lever press  Food pellet
SD
R
SR
Buzzer: Lever press  No food
SD
R
--Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Three-Term Contingency (ABC)
• an Antecedent event
• a Behavior
• a Consequence
Antecedent
Susan:
SD
Tone:
SD
Behavior
Tell her a joke 
R
Lever press 
R
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Consequence
She laughs
SR
Food pellet
SR
Easy Ways to Remember
• ABC
• Notice something, do something, get
something
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Operant & Classical Conditioning
• A stimulus can act as both a SD and a CS.
• Example:
Tone: Lever press  Food
SD R
SR
Tone: Food  Salivation
NS
US
UR
Tone  Salivation
CS
CR
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Determining Type of Contingency
1. Does the consequence consist of
something being presented or
withdrawn?
2. Does the consequence serve to
strengthen or weaken the behavior?
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Negative vs. Positive
• Negative means that the behavior has
resulted in something being removed or
subtracted.
• Positive means that the behavior has
resulted in something being presented or
added.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Reinfocer vs. Punisher
• Reinforcer means the behavior has
strengthened as a result of the
consequence.
• Punisher means the behavior has
weakened as a result of the consequence.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Positive Reinforcement
• the presentation of a stimulus following a
response, which then leads to an increase in the
future strength of that response.
• Examples:
Smile at person  The person smiles at you
R
SR
Study diligently for quiz  Obtain an excellent mark
R
SR
Compliment partner  Receive a kiss
R
SR
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Negative Reinforcement
• the removal of a stimulus following a response,
which then leads to an increase in the future
strength of that response.
• Examples:
Claim illness  Avoid writing an exam
R
SR
Take aspirin  Eliminate headache
R
SR
Turn on the heater  Escape the cold
R
SR
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Heater Example
• Does the person turn on the heater
– to escape the cold (negative reinforcement) or
– to obtain warmth (positive reinforcement)?
• Either interpretation would be correct.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Escape Behavior
• results in the termination (stopping) of an
aversive stimulus.
• Examples:
– Opening an umbrella when it rains
– Turning on the heater when it is cold
– Taking an aspirin when you have a headache
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Avoidance Behavior
• occurs before the aversive stimulus is
presented and therefore prevents its
delivery.
• Examples:
– pretending to be ill to avoids having to write
an exam
– staying inside to avoid the rain
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Positive Punishment
• the presentation of a stimulus following a
response, which then leads to a decrease in the
future strength of that response.
• Examples:
Talk back to the boss  Get reprimanded
R
SP
Swat at the wasp  Get stung
R
SP
Meow constantly  Get sprayed with water
R
SP
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Negative Punishment
• the removal of a stimulus following a response,
which then leads to a decrease in the future
strength of that response.
• Examples:
Stay out past curfew  Lose car privileges
R
SP
Argue with boss  Lose job
R
SP
Play with food  Lose dessert
R
SP
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Punishment in Personal
Relationships
• A reduction in one person’s behavior as a
result of punishment can negatively
reinforce the behavior of the person who
implemented the punishment.
• Example:
GF ignores BF  BF stops talking to women
R
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
SR
Punishment in Personal
Relationships, continued
• Punishment is often successful in
immediately getting a person to stop.
• That success then reinforces our tendency
to use punishment in the future.
• Punishing our partners more can create
major problems for our relationships in the
long run.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
The Key to a Great Relationship
• the ratio of positive (pleasant) interactions
to negative (aversive) interactions.
• The positives need to outweigh the
negatives by a ratio of about five to one
(Gottman, 1994).
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Immediate vs. Delayed
Reinforcement
• The more immediate the reinforcer, the
stronger its effect on the behavior.
• Example:
– Give a treat to a child who’s playing quietly
while she is still playing quietly to reinforce
quiet playing.
– The benefits of exercise and proper eating are
delayed and therefore weak.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Primary Reinforcers
• A primary reinforcer is an event that is
innately reinforcing.
• Examples:
– food, water, proper temperature, sexual
contact.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Secondary Reinforcers
• A secondary reinforcer is an event that is reinforcing
because it has been associated with some other
reinforcer.
• Examples:
– good marks, fine clothes, a nice car
• Conditioned stimuli that have been associated with
appetitive unconditioned stimuli (USs) can also function
as secondary reinforcers.
• Example:
– A metronome that has been associated with food can be a
secondary reinforcer for the operant response of lever pressing.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Generalized Reinforcer
• a type of secondary reinforcer that has been
associated with several other reinforcers.
• Example:
– Money is associated with food, clothing, furnishings,
entertainment, and even dates.
– Social attention is associated with food, play, and
comfort.
– A “token economy” is when tokens are given as
reinforcers. The tokens can be traded for other
rewards later.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Intrinsic Reinforcement
• reinforcement provided by the mere act of
performing the behavior.
• Examples:
– Rollerblading is invigorating.
– Attending parties is fun.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Extrinsic Reinforcement
• the reinforcement provided by some
consequence that is external to the
behavior
• Examples:
– You drive to get somewhere.
– You work for money.
– You date an attractive individual merely to
enhance your prestige.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
The distinction is not always clear.
• Is candy an intrinsic or extrinsic reinforcer?
• What happens if you are given an extrinsic
reinforcer for an activity that is already
intrinsically reinforcing?
• The research findings are mixed.
• A meta-analysis found that external rewards can
undermine intrinsic motivation when:
– the reward is expected,
– the reward is tangible, and
– the reward is given for simply performing the activity.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Natural Reinforcers
• reinforcers that are naturally provided for a
certain behavior.
• They are a typical consequence of the
behavior within that setting.
• Examples:
– Money is a natural consequence of selling
merchandise.
– Gold medals are a natural consequence of
hard training and a great performance.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Contrived Reinforcers
• reinforcers that have been deliberately
arranged to modify a behavior.
• They are not a typical consequence of the
behavior in that setting.
• Examples:
– Turning on the television is a contrived
reinforcer for studying.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Intrinsic/Extrinsic –
Natural/Contrived
• Intrinsic reinforcers are always natural
reinforcers.
• Extrinsic reinforcers can be either natural
or contrived.
• Some extrinsic reinforcers are part of the
typical contingencies in our environment,
while others have been artificially imposed
to modify a particular behavior.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
In Behavioral Therapy
• When contrived reinforcers are used, the
ultimate intention is to let the “natural
contingencies” eventually take over if at all
possible.
• Example:
– We initially use praise to increase the frequency with
which a child reads.
– Hopefully, the natural (and intrinsic) reinforcers
associated with reading will take over.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Which is more important?
• Natural contingencies tend to produce
more efficient behavior patterns than do
contrived contingencies.
• Example:
– Throwing a ball where you want it to go is
more important than throwing a ball and
receiving praise.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Shaping
• the gradual creation of new operant
behavior through reinforcement of
successive approximations to that
behavior.
• Example:
– Shaping a rat to press the lever
– Teaching a dog to catch a frisbee
– Teaching animals to perform tricks
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Our Shaping
• Most of our behaviors have, to some
extent, been learned or modified through
shaping.
• Example:
– Learning to eat with utensils
– Learning to dress ourselves
– Learning to write
– Learning to speak politely
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Summary
• Operant behaviors are controlled by their
consequences.
• Reinforcers strengthen a response.
• Punishers weaken a response.
• Positive means the presentation of
something.
• Negative means the removal of
something.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Summary, continued
• A discriminative stimulus signals when a
consequence is available.
• Reinforcers can be described as:
– immediate or delayed
– primary or secondary
– generalized
– intrinsic or extrinsic
– natural or contrived
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.
Summary, continued
• Shaping is the creation of novel behavior
through the reinforcement of gradual
approximations to that behavior.
• Effective shaping is often carried out with
the use of a secondary reinforcer.
Introduction to Learning and Behavior, 3e
by Russell A. Powell, Diane G. Symbaluk, and P. Lynne Honey
Copyright © 2009 Wadsworth Publishing, a division of Cengage Learning. All rights reserved.