Transcript Slide 1

Thinking about Choice Behavior
• The problem of choice
– Organisms constantly bombarded with
alternatives
• What controls allocation of behavior?
Why Study Choice?
•
•
1. The empirical relation -- Herrnstein (1961)
2. Because all behavior is fundamentally
choice behavior (Herrnstein, 1971)
–
There is always a choice
•
•
–
–
Other responses that can be emitted
Other reinforcers that can be obtained -- even when you
are reinforcing just one response (e.g., in a single VI
schedule).
An animal or a human can always do something
else.
Understand choice behavior -> understand all
behavior
Experimental Analysis of Choice
• Methods: concurrent schedules, concurrent chains,
delay discounting, foraging contingencies, behavioral
economic contingencies.
• Models and Issues: matching/melioration,
maximizing/optimality, hyperbolic discounting, behavioral
economic/ecological models, behavior momentum, molar
versus molecular issue, concepts of response strength.
• Applications: self-control, drug abuse, gambling, risk,
economics, behavioral ecology, social/political decision
making.
Operant Conditioning
• How will behavior
allocate?
– When t’s are equal?
– Slightly different?
– Greatly different?
Measures of choice in concurrent schedules
1. Response allocation: The number of responses
emitted on each alternative
Used to be measured as relative responses (the number
of responses on one alternative divided by the total
responses
B1
B1  B 2
Proportional measure: ranges from 0 (all
responses to B2) to 1 (all responses on B1)
Measures of choice in concurrent schedules
1. Response allocation: The number of responses
emitted on each alternative
or, more recently, as response ratio (the number of
responses on one key divided by the number of
responses on the other key):
B1
B2
This measure can range from 0 to infinity
(and is not homoscedastic – it does not have
constant variance) so…
Measures of choice in concurrent schedules
1. Response allocation: The number of responses
emitted on each alternative
Often used:
 B1 
log 
 B2 
This measure can range from minus infinity to infinity
(and probably IS homoscedastic – it does have constant
variance as we change things that affect choice)
Measures of choice in concurrent schedules - continued
2. Time allocation: The amount of time spent on each
alternative.
This is measured from changeover response to
changeover response in the switching-key procedure,
or from the first response on one alternative to the first
response on the other alternative in the two-key
procedure.
As above, the measures used may be relative time, or the
time-allocation ratio.
T1
T1  T 2
T1
T2
 T1 
log 
T2 
Relative time allocation
Time-allocation ratio
Log time-allocation ratio *
Response and time measures are
usually very similar…
• Independent of measures of choice
– relative response-allocation and timeallocation ratios
• Or procedure
– Two-key or switching-key procedure
– Independent versus dependent scheduling
The initial empirical finding -- Herrnstein (1961) VI VI
1.0
HERRNSTEIN (1961)
BIRD 55
RELATIVE RESPONSES
0.8
BIRD 231
BIRD 641
0.6
0.4
0.2
STRICT MATCHING
0.0
0.0
0.2
0.4
0.6
RELATIVE REINFORCERS
0.8
1.0
So where do we start?: the strict matching law.
Approximately, the proportion of responses emitted to
one of the concurrent VI VI schedules equals the
proportion of reinforcers obtained at that alternative
1.0
HERRNSTEIN (1961)
BIRD 55
RELATIVE RESPONSES
0.8
BIRD 231
BIRD 641
0.6
0.4
0.2
STRICT MATCHING
0.0
0.0
0.2
0.4
0.6
RELATIVE REINFORCERS
0.8
1.0
The graph shows that, approximately, the proportion of
responses emitted to one of the concurrent VI VI
schedules equals the proportion of reinforcers obtained
at that alternative
This is the strict matching law, and its formula is:
B1
R1

B1  B 2 R1  R 2
B = responses, R = reinforcers, and the Subscripts 1 and
2 denote the alternatives.
Another way of writing the strict matching law:
B1
R1

B1  B 2 R1  R 2
Cross multiply:
B1( R1  R 2)  R1( B1  B 2)
B1R1  B1R 2  B1R1  B 2 R1
Subtract B1R1 from each side:
B1 R 2  B 2 R1
so:
B1 R1

B2 R2
B1
R1

B1  B 2 R1  R 2
is called the relative version of the strict matching law
B1 R1

B2 R2
is called the ratio version of the strict matching law
They both say exactly the same thing.
HERRNSTEIN’S HYPERBOLA:
ASSUMPTIONS
• 1. The matching law holds.
• 2. Every contingency involves choice (e.g.,
one can respond to a given alternative or
not).
• 3. Not responding to one alternative
means all others supply reinforcers too.
• 4. The total behavior in a situation is
constant.
B1
r1

B1  B 2 r1  r 2
Assume B1  B 2  k .
B1
r1

k
r1  r 2
Set B1= B, r1= r, and r2= ro, where ro
represents other sources of reinforcement.
Thus,
kr
B
r  ro
Herrnstein's Hyperbola
Operant Conditioning
Operant Conditioning
HERRNSTEIN’S HYPERBOLA
B  r krro
Modern Version
R
kr
a
a
o
r
r 
b
a
Operant Conditioning
Operant Conditioning
MATCHING LAW
Herrnstein
R1 / R1 = r1 / r2
Baum
R1 / R2 = b (r1 / r2)a
MATCHING LAW
• R1 / R2 = r1 / r2
Herrnstein
• R1 / R2 = b (r1 / r2)a
Baum
• V1 / V2 = b (r1 / r2)a1 (M1 / M2)a2 (D2 / D1)a3
• Rachlin’s “Value”
Operant Conditioning
Operant Conditioning
Melioration: A Theory of Matching
“To make better”: Behavior shifts to the
higher return (lower cost) or equal local
rates of reinforcement.
(R1 / R2) = (r1 / r2), or (r1 / R1) = (r2 / R2)
(reinforcers per response, i.e., return).
(T1 / T2) = (r1 / r2), or (r1 / T1) = (r2 / T2) (local
rate of reinforcement).
Example: Conc VI 30”VI 120”
Suppose in the first hour of exposure 1000 responses were
emitted to each alternative:
(VI 30”) r1 / R1 = (120 rfs / 1000 resps). Return = 0.120
(VI 120”) r2 / R2 = (30 rfs / 1000 resps). Return = 0.03
Ultimately behavior will shift toward the higher return. What
will be the result?
120 / (1000 + x) = 30 / (1000 – x); x = 600.
120 / 1600 = 30 / 400 i.e., matching (80% responses on VI
30” alternative).
Return = 0.075 rfs/resp on each alternative.
Problem: Conc VR 30 VR 120
ALL responses will ultimately be made to the VR
30 alternative. This is consistent with matching,
but same would be said if all the responses were
made to the VR 120 alternative. But melioration
can predict which alternative should receive all
the responses:
VR 30: r1/ R1 = 1/30; VR 120: r2/ R2 = 1/120.
These cannot change, so shifting to the higher
return means all the responses will go to VR 30
alternative.
Operant Conditioning
Operant Conditioning
FUNCTIONAL PROPERTIES AND CURVE
FITTING
• What is the “real” delay function?
Vt = V0 / (1 + Kt)
Vt = V0/(1 + Kt)s
Vt = V0/(M + Kts)
Vt = V0/(M + ts)
Vt = V0 exp(-Mt)
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning
Operant Conditioning