Transcript Tori Collins - USD Biology

Wallis, JD
Helen Wills Neuroscience Institute
UC, Berkeley

Role of prefrontal cortex (PFC) in rewardguided choice behavior

2 Questions:
 Does PFC encode reward or behavioral sequelae of
reward?
 Is encoding specific to reward outcome or
reflective of abstract value signal?

Orbitofrontal cortex (OFC) is a key region in choice
behavior
 Has functions in
emotions and reward
• Thought to regulate planning
behavior associated with
sensitivity to reward and
punishment
Paul Wicks’ brain

Damage to the OFC leaves cognitive abilities
intact, but impairs ability to make decisions
 The cautionary tale of Elliot

OFC neurons encode expected rewards
 Neurons show response to delivery of juice
rewards predicted by a visual stimulus
 Useful for decision making

Reward encoding neurons are also found in
the dorsolateral PFC (DLPFC)
 Neurons showed a difference in firing rate
depending on large/small expected reward

However, are these neurons encoding the
value of the reward or a behavioral correlate
of the reward (tensed muscles, heightened
accuracy)?


Comparison of reward encoding in the DLPFC
and the OFC
2 monkeys choose between pictures
associated with small/large fruit juice rewards




Each picture associated with delivery of a
certain amount of juice
Subjects learn to maximize reward
Reward-picture contingencies reversed after
27 out of 30 successes
Most important neuronal activity after 2nd
picture appears

Activity was recorded simultaneously from
multiple electrodes in DLPFC and OFC
 167 DLPFC neurons
 134 OFC neurons





Spike density histograms from 2
representative OFC neurons
Activity recorded during predictive cue
One neuron showed higher firing rate when
the monkey expected 4 drops of juice
Another encoded the reward in parametric
fashion
Firing rate not affected by saccade
Figure 2



DLPFC neurons show responses related to
both reward and saccade
One neuron discriminated between different
amounts of juice only during right saccade
Another encoded reward in parametric
fashion (positive?) with a greater increase
during left saccade
Figure 3


2-way ANOVA on mean firing rate with factors
of Reward and Saccade
OFC:
 28% significant main effect of Reward
 19% significant interaction with Saccade

DLPFC
 13% significant main effect of Reward
 43% Reward-Saccade interaction

Some neurons in both areas had similar
properties:
 27% parametric increase with reward size
 15% parametric decrease with reward size
 59% encode specific reward

Sliding receiver operating characteristic (ROC)
analysis of the selectivity time-course
revealed differences in encoding of reward
between OFC and DLPFC
 Probability that an independent observer could
predict reward based on firing rate

Starting from 500ms prior to 2nd picture an
ROC curve was generated from 10ms
increments

Latency at which selectivity appeared was
computed as the point at which the curve
exceeded 0.6
 No difference between OFC (mean 426ms) and
DLPFC (mean 467ms)
▪ (t-test = 1, d.f. = 110, P > 0.1)

Selectivity rose more rapidly and peaked
earlier in OFC
 80 ms earlier on average

Short latency indicates OFC is encoding
reward’s value rather than correlated
behavioral/cognitive processes

Neurons sensitive to expected reward are
found in both the OFC and DLPFC

OFC neurons encoded only reward while
DLPFC neurons encoded reward and saccade

OFC neurons encoded reward earlier than
DLPFC

Therefore OFC is first prefrontal region
receiving reward information
 From basolateral amygdala
▪ Encodes immediate reward –Winstanley, CA et al
 From gustatory and olfactory cortices
Baxter, MG and Murray, EA The amygdala and reward

DLPFC is where reward value converges with
subjects actions: reward choice



OFC response indicates to the motor system
which action leads to largest reward
However, decision making needs to be more
complex
Reward value is determined by
 (Reward – Cost) x P of success
 To what extent does OFC encode these variables?

OFC may integrate all variables relevant to
decision making to derive an abstract value
signal
 Neuronal currency

Another study tested whether PFC neurons
were capable to responding to multiple
parameters

Monkeys were trained to choose between
pictures associated with particular rewards

Recordings from OFC, MPFC, and lateral PFC
taken simultaneously

3 variables were manipulated:
 Probability: Some pictures predict fixed amount of
juice on only a certain proportion of trials
 Reward: Some pictures were associated with
varying amounts of juice
 Effort: Monkey had to earn fixed amount of juice
by pressing a lever multiple times

1/3 of PFC neurons responded parametrically
to just 1 parameter
 Found in all 3 areas

Some neurons responded to a combination of
parameters
 Progressive increase from LPFC to OFC to MPFC
 16% -> 27% -> 48%

OFC and MPFC combine multiple variables in
order to make a decision by deriving abstract
value signals
 Too difficult to make direct comparisons between
all possible choices
 Each choice can be valued against a common
reference scale (currency)
 Example: how many bananas is your car worth?

Abstract value signals allow flexibility and
novelty
 Simplifies the task of the motor system
 Allows instantaneous choice

Patients with OFC and MPFC damage show
unusual patterns of decision making
 A>B,
B > C but A < C


Damage to the OFC impairs decision making
while leaving other cognitive abilities intact
OFC is implicated in reward info processing
 Must differentiate between reward and behavioral
sequelae of reward

Short latency of neuronal reward-related
responses indicates encoding of reward’s
value in OFC



In contrast, DLPFC encodes reward info in
relation to behavioral responses
PFC neurons also encode other variables
related to decision making, including
probability of success and effort required
OFC and MPFC neurons are responsible for
integrating these variables to derive an
abstract value signal