Transcript Neural Correlates of Variations in Event Processing during Learning

(2010)
Neural Correlates of Variations in
Event Processing during Learning in
Basolateral Amygdala
Matthew R. Roesch*, Donna J. Calu, Guillem R. Esber, and
Geoffrey Schoenbaum
* Department of Psychology and Program in Neuroscience
and Cognitive Science, University of Maryland College Park
Background…
• To optimize reward, animals must learn to
associate cues with rewards and recognize the
difference between the reward expected and
that which actually occurs to guide their
behaior
• The prediction error
• 2 categories of learning models:
Category 1: “Signed error” Models
• If a reward is larger than expected(+), the
association between the cue and reward will
be strengthened, whereas if the reward is
smaller than expected(-), the association will
be weakened.
• …predict that the sign of the prediction error
(i.e., whether the reward is bigger or smaller
than expected) will be encoded in neural
activity.
Category 1
• This correlate has been shown in midbrain
dopamine neurons (Rescorla and Wagner, 1972; Sutton and Barto, 1998;
Mirenowicz and Schultz, 1994; Montague et al., 1996; Schultz et al., 1997;
Hollerman and Schultz, 1998; Waelti et al., 2001; Bayer and Glimcher, 2005; Pan et
al., 2005; Bayer et al., 2007; D'Ardenne et al., 2008; Matsumoto and Hikosaka,
2009) .
• Firing in these neurons increases in the face of
unexpected reward (+) and is suppressed when
reward is unexpectedly omitted (-).
• Evidence also from other brain areas (Hong and Hikosaka,
2008; Matsumoto and Hikosaka, 2009) .
Category 2: “Unsigned error’ models
• Prediction errors tell an animal that it must
learn more about the cue–reward association
and therefore serve to drive attention.
• A cue should be more thoroughly processed
(and learned about) when it is a poor
predictor of reward. When the cue becomes a
more reliable predictor, processing (and
learning) should decline (Pearce–Hall model (1980, 1982)).
Category 2
• These models predict that neural activity
encoding prediction errors will be similar
regardless of the sign of the error(+/-).
• …lack of evidence for neural correlates of
unsigned prediction errors—e.g., increased
firing when reward is either better or worse
than expected.
What did this paper do?
• Basolateral Amygdalar (ABL) Neurons Encode
Unsigned Prediction Errors.
• This neural signal increased immediately after
a change in reward, and stronger firing was
evident whether the value of the reward
increased or decreased.
How did they do it
• Recording single unit activity in a behavioral
task in which rewards were unexpectedly
delivered or omitted.
• Basic paradigm is a choice task
Reward
well
Odor Port
Reward
well
3 different odor cues: one signaled reward on the right
(forced-choice), a second for left (forced-choice), and
a third for either well (free-choice).
• Trials and Blocks
-
+
-
>
<
>
<
Each Block consists of at least 60 trials; In between blocks,
rewarding value shifted (i.e. value of the port for rats
changed)
+
+
Results
• Performance and recording sites
• 70 reward-responsive ABL neurons recorded ;
• 58/70 exhibited differential firing base on
timing (short/long delay) or size of the
reward(large/small) after learning,  signed
coding theory; outcome-selective
• They also exhibited changes in reward-related
firing between the beginning and end trials of
each block, regardless reward upshift or
downshift, unsigned coding theory
• 2 factor ANOVA analysis in each neuron across
learning (early vs late) and shift type (upshift
vs downshift)
• 10 of the 58 neurons (17%) fired significantly
more early in a block(after a change in
reward), than later(after learning).
• Indices [(early – late)/(early + late)],
representing the difference in firing to reward delivery
(within 1 s) during trials 3–10 (early) and during the last
10 trials (late) after shifts .
Main contribution of this paper
• The activity in the outcome-selective ABL
neurons was higher at the start of a new
training block, whether reward was better or
worse than expected, and declined as the rats
learned to predict the value of reward.
• This pattern of firing is generally consistent
with the notion of an “unsigned error” models
such as that of Pearce and Hall (1980)
Another distinctive feature
• Their firing did not
immediately increase
at the start of a new
block, in response to a
change in reward, but
rather appeared to
gather momentum
and peak a few trials
into the block (3rd
trial).
• Given the remarkable fit provided by the
amended Pearce–Hall model (1982) and the role
attributed to unsigned errors within this
theoretical context, it seems natural to speculate
that this ABL signal may be related to variations
in event processing (title).
• Especially in view of the striking similarity
between changes in the ABL signal and changes
in the rats' latency to approach the odor port at
the start of each trial.
The close relationship between the ABL signal and
a behavioral measure “speed of orienting”
Increase in speed of
orienting to the odor port
Trial by trial analysis
Explanation
• Faster odor-port approach latencies may reflect
error-driven increases in the processing of trial
events (e.g., cues and/or reward), because rats
accelerate the reception of those events when
shifted contingencies need to be worked out.
• In this sense, approaching the odor port faster
can be looked upon as similar to conditioned
orienting. Conditioned orienting responses, also
known as investigatory reflexes, means to
recover from habituation when learned
contingencies are shifted.
• To further investigate the relationship
between the ABL signal and odor-port
approach latency, ABL was inactivated in some
rats during performance of the recording task.
• Inactivation of ABL disrupted the change in
orienting.
ns
• Inactivation of ABL also retarded learning in
response to changes in reward.
Block 1 well 1> well2, rats prefer well1
•
•
Inactivation of ABL with DNQX
Block 2, well 1< well 2, rats continue to approach well 1
Choice performance in vehicle versus NBQX sessions, plotted
according to whether the well values in a particular trial block
were similar to or opposite from those learned at the end of
the prior session.
Conclusions
• Basolateral Amygdalar Neurons Encode Unsigned
Prediction Errors ;
• This neural signal was correlated with faster
orienting to predictive cues after changes in
reward, and abolition of it disrupted this change
in orienting and retarded learning in response to
changes in reward.
• These results suggest that basolateral amygdala
serves a critical function in attention for learning.