Transcript E(R) - Consciousness Online

Decision mechanisms for attention
Jacqueline Gottlieb
Department of Neuroscience
The Kavli Institute for Brain Research
Columbia University
Simon Kelly
Himanshu Mhatre
Nico Foley
“Attention is a gain modulation of a
sensory response, driven by top-down
feedback.”
The feedback comes from frontal and
parietal areas.
How is this feedback
generated?
Target selection
FEF
LIP
MT
V4
SC
Spatial receptive field.
Little feature selectivity.
Selective responses to
stimuli that are likely to
attract attention.
V1
Itti and Koch, 2000
How does this signal
arise?
Why 2 areas?
A possible answer: reward
High reward target
Low reward target
Sugrue et al., Science, 2005
Neurons encode
“action value”
The value is
determined by E(R)
Limited explanatory power
In natural behavior, attention and eye
movements are not directly rewarded
Attentional decisions are endogenous, even when
they support behavioral goals.
LIP/FEF neurons encode covertly attended objects.
We attend to salient distractors that can
only reduce our foreseeable reward.
LIP neurons respond strongly to unrewarded distractors.
The salience map in LIP selects stimuli, not actions
Gottlieb and Balan, TICS ,2010
What do we get from observing
a stimulus?
Pleasure (or displeasure)
Information
Pavlovian learning
Positive
Negative
Attention is
independent of
valence (“salience”)
OR
Attention depends
on valence
(emotional effect)
Attention depends on stimulus value
A cue announces the
reward of the trial
Good
news
attract
attention
RC+
(reward)
RC- (no reward)
Bad news
repel
attention
Peck, Jangraw et al., 2009
A saccade target appears
Peck, Jangraw et al., 2009
Impairment at the RC- location
Vertical eye position
RC+
RC-
More errors,
lower reward
Horizontal eye position
Peck, Jangraw et al., 2009
Increases with training!
RCIncongruent
Congruent
Peck, Jangraw et al, 2009
What do we get from observing
a stimulus?
Pleasure (or displeasure)
Stimulus (Pavlovian) value biases attention
Automatic, potentially maladaptive
Information
Reduction of uncertainty
I. Reliable predictors
II. Novel or uncertain predictors
Reducing uncertainty
2 antagonistic systems?
Two attentional systems?
Selects reliable cues
Frontal lobe
FEF?
Selects uncertain cues
Amygdala, substantia nigra,
parietal lobe
LIP?
Based on experiments in rats. Very different
definition of “attention”.
Still, could it be true in monkeys?
Two categories of salience?
Not purely sensory,
but obligatory
Novelty, uncertainty,
surprise, emotion (?)
Reward, goals (?)
FEF
LIP
MT
V4
V1
SC
Parietal neurons are sensitive to uncertainty
50% reward
Dual influence of
uncertainty and
reward
Reward probability
100% reward
(RC+)
0%
75
Firing rate (sp/s)
50%
100%
0% reward (RC-)
0
0
Cue on
300
Time (ms)
600
900
Parietal neurons are sensitive to novelty
Familiar vs. novel patterns
Large, early
effect of
novelty
Novel
RC+ (100%), RC- (0%)
Familiar
RC+ (100%), RC- (0%)
Reducing uncertainty
I. Reliable predictors
Predictive stimuli reduce
uncertainty
Increase the
reward of a future
action.
! Is it simply
expected reward...
…or sensitive to new
information?
The value of information
Black cue: 100% validity
Green cue: 80% validity
reward
Blue cue:
reward
55% validity
3 Informative
cues with
different
validities
Uninformative cues
Uninformative stimuli are necessary
to obtain the reward.
They bring no new information; the
monkey has already obtained all the
information through the first cue.
Black
>> yellow >> 100%
Green >>
Blue
>>
Informative
red
>> 80%
cyan
>> 55%
Redundant
E(R)
Consistent pairing: matched for E(R)
with informative cues
Reward modulates only the informative cues
Informative
Normalized response
100%
Uninformative
100%
80%
80%
Cue on
Go
77% significant
55%
Cue on
Go
14% significant
55%
Is it D(E(R))?
Informative
Redundant
D(E(R))
+0.22
fixation
E(R)=1.0
+0.02
0.00
p(R)=1.0
E(R)=1.0
0.00
p(R)=0.8
E(R)=0.8
E(R)=0.8
E(R)=0.78
-0.23
E(R)=0.55
0.00
E(R)=0.55
p(R)=0.55
Normalized firing rate
100%
55%
80%
Uninformative
Possibly D(E(R))
A modulation!
Neurons DO respond to the
uninformative cues. This may indicate
that neurons encode a “visual
prediction error” that occurs by
default and is modulated by expected
reward.
NOT entropy!
0.000 bits
0.072 bits
DE(R)
Uninformative stimuli bring less
information but elicit a higher
response than the 55% cues. It is not
the information per se! It is its utility
to the organism. After all, attention
MUST be selective!
Attention is the system that satisfies
the brain’s demand for information.
It is controlled by decision variables related
to active learning. These variables prioritize
*reliable predictors that we can learn from
*novel/uncertain/surprising stimuli that we
can learn about.
Two computations may identify such stimuli:
*prediction errors (reward and sensorimotor)
*direct Pavlovian associations (fast but fallible)
Eyes are windows to the soul.
Let’s give them due respect!
Identifying predictors is fiendishly
difficult.
Pavlovian mechanisms may provide a
fast (and fallible) heuristic.
William James,
Lectures to teachers, 1899
To keep [your students] where you have called
them, you must make the subject too interesting
for them to wander again. And for that there is
one prescription […]
[…]the subject must be made to show new
aspects of itself; to prompt new questions; in a
word, to change. From an unchanging subject the
attention inevitably wanders away.