Transcript An Information Processing Perspective on Conditioning

Information in “Associative”
Learning
C. R. Gallistel
Rutgers Center for Cognitive Science
Temporal Pairing
• Thought to be essential for the formation of
associations
• Assumed to be the critical variable in work on
neurobiology of learning (LTP)
• Basis of unsupervised learning in neural net
models
Sloan-Swartz 7/22/08
1
But
• It’s never been objectively defined for any
paradigm: What is the critical interval?
• Neither necessary nor sufficient for
development of a conditioned response to
the CS (the warning signal)
Sloan-Swartz 7/22/08
2
Not Necessary
• Subjects develop a conditioned response to
a CS that is never paired with the US (the
predicted event)--conditioned inhibition
• Pavlov and Hull struggled with this problem
• It has not been solved
Sloan-Swartz 7/22/08
3
Not Sufficient
• The truly random control (Rescorla, 1968)
– It is the mutual information between CS & US that is
critical
– Not their temporal pairing
Sloan-Swartz 7/22/08
4
It’s Information!
• People believe in “temporal pairing”
because they are intuitively sensitive to the
fact that a relatively more proximal warning
gives more information
• It’s the information that matters, not the
temporal pairing
Sloan-Swartz 7/22/08
5
Information Derives From
Temporal Representation
• Information-theoretic analysis explains BOTH cue
competition AND the data on the temporal pairing
• Founded on the assumption that animals learn the
intervals
• AND, they represent the uncertainty with which
they can remember them (about +/- 15%)
Sloan-Swartz 7/22/08
6
Principles I
• Subjects respond only to stimuli (CSs) that
provide information about the timing of
future events (USs)
• CSs inform to the extent they change the
subject’s uncertainty about the time to the
next US
Sloan-Swartz 7/22/08
7
Principles II
• Bandwidth maximization by minimizing number
of information-carrying CSs attended to
• Information carried by intervals and numbers
• They are what is learned
• Weber’s law: uncertainty scales with delay:
=wT
Sloan-Swartz 7/22/08
8
Rate-Change Protocols
 e 
Ý
H   log 2 
 
 e 
H   log 2
 k  log 2 
 

1
H b  H cs  k  log 2 b  k  log 2 cs   log 2 cs  log 2 b

 cs 
 I us-us 
Information communicated by CS log 2
    log 2  I

b
us-us|cs

Sloan-Swartz 7/22/08
9
Delay Protocols
• Two sources of information:
1) The rate change
2) The fixed delay
• They are additive
• Only one depends on protocol parameters
 cs 
H  log 2    k
 
b
cs  1 T
 e 
1
k  log 2   log 2 w
2 
2
Sloan-Swartz 7/22/08
10
Gibbon & Balsam
• Reinforcements to
acquisition, as a
function of the
Ius-us/Ics-us ratio
• Slope (log-log) ~ -1
Sloan-Swartz 7/22/08
11
Trials Don’t Matter
• These two protocols are equi-effective!
• The number of trials is not in and of itself a
learning-relevant parameter of a training protocol
• Gottlieb (2008)
Sloan-Swartz 7/22/08
12
Associability
A  1 / N cs-us
• where Ncs-us = the number of CS reinforcements
required to produce an anticipatory response.
(The onset of conditioned responding is abrupt)
• Definition parallels definition of sensitivity
(1/Intensity) in sensory psychophysics
• Purely operational: no implication that
associations exist
Sloan-Swartz 7/22/08
13
Informativeness
• We define the ratio of the background rate
to the rate in presence of CS to be the
informativeness of the CS-US relation in an
associative learning protocol
• Thus, the information conveyed is the log of
the informativeness
Sloan-Swartz 7/22/08
14
A Simple Quantitative Law
Associabilty  Informativeness
cs
I US-US
A

b I US-US|CS
Sloan-Swartz 7/22/08
15
Why trials don’t matter
•
•
•
•
When there are 8 times fewer trials,
the trials are 8 times more informative
Provided one maintains total protocol duration
The only way to speed up learning is to increase
informativeness of the CS-US relation.
• Adding trials won’t do it!
Sloan-Swartz 7/22/08
16
Conclusion 1
• Temporal pairing is
–Undefinable
–Insufficient
–Unnecessary
• “Trials” are a pernicious fiction.
Banish them from your models
Sloan-Swartz 7/22/08
17
Conclusion 2
• What matters is the mutual information (between
CS and US), a component of which is the change
in US rate when the CS comes on
• The informativeness of the CS-US relation is the
factor by which CS onset changes the expected
time to the next US
• Associability is proportional to informativeness
• That’s why people believe in in temporal pairing
Sloan-Swartz 7/22/08
18
Conclusions 3
• Focus on mutual information gives an empirically
supported quantitative account of the notion of
temporal pairing
• And an account of “cue competition:” how the
system solves the multivariate prediction problem
(aka the assignment-of-credit problem; what is
predicting what), the other problem posed by
Rescorla’s experiment
Sloan-Swartz 7/22/08
19
Thank You
• Collaborators
–
–
–
–
The late John Gibbon
Peter Balsam
Stephen Fairhurst
Daniel Gottlieb
• Support
– RO1 MH68073 Time and Associative Learning
Sloan-Swartz 7/22/08
20