Transcript katherine_slides

Does the brain compute confidence estimates about decisions?
EXPERIMENT SETTING AND BEHAVIORAL DATA
-Two choice odor mixture
categorization
- Randomized delays:
between entering the odor
port and odor delivery;
and between entering the
choice port and reward
delivery (for correct choices)
- Reward contingencies
deterministic
-Decision uncertainty varies
with stimulus difficulty due to
imperfect perception of
stimuli and knowledge of
category boundary
NEURAL DATA
- Single neuron activity in the orbitofrontal cortex (OFC) was recorded during
the delay period between choice and reward
- Firing rates of many OFC neurons during this period were modulated by
stimulus difficulty:
A large fraction of OFC neurons fired more
intensely for more difficult stimuli; smaller fraction
showed opposite tuning. This is consistent with
previous findings that OFC neurons activity
correlates with the expected values of reward.
MORE NEURAL DATA
Is the firing rate modulated by anything else except stimulus difficulty?
Many neurons show different firing rates for correct and error choices, and
this difference is observed before the outcome delivery:
Again, a large fraction of
neurons fired more for
incorrect trials; smaller
fraction had opposite
tuning.
Notice that the difference
in firing rates is larger for
easier stimuli which
paradoxical if the firing
rate is driven by stimulus
difficulty!
SUMMARIZING THE NEURAL DATA
Explanation?
DYNAMIC LEARNING ?
Maybe reward predictions are based on recent reinforcement history?
In this scenario, outcome selectivity would arise because the present trial’s
expected outcome is correlated with recent trials’ outcomes.
Linear regression: firing rate depends linearly on stimulus, choice, and history
of previous outcomes.
It turns out that although some OFC neurons do carry information about past
trials, including the history of recent outcomes does not improve the model fit
significantly.
Dynamic learning? No…
INTERPRETATION OF NEGATIVE OUTCOME SELECTIVITY: ERROR ?
Maybe the negative outcome selective population of neurons signals error rather
than uncertainty?
Negative outcome selectivity might arise if, after executing a choice, extra sensory
or memory information enters the decision-making circuits and causes realization
that an error occurred (even before obtaining feedback).
This is tested by separating the highest firing rate trials. It turns out that they are
associated not with errors but rather with near-chance performance. Also, this does
not explain the V-shaped curves.
Signaling error?
No…
OUTCOME SELECTIVITY ANALYSIS ?
OFC is known to signal outcome expectations. Outcome prediction might arise
from a combination of stimulus and side selectivity.
If firing rates are driven by stimulus difficulty and choice side:
Dashed arrows signify the distance
between error and correct choices of a
given difficulty.
The direction of solid arrows signifies
whether error or correct choices have
higher rates for a given stimulus.
This is not what we see in the data…
Outcome selectivity? No…
CONFIDENCE MODEL?
The probability of a correct trial outcome could be estimated based on a
subjective measure of confidence about the decision.
Model: measure of confidence = comparison of the perceived stimulus value
and the recalled category boundary.
Note that the model (and the subject) only has access to a stimulus sample and not
the stimulus type. Errors only occur where distributions overlap, which is smaller than
the entire range; so the maximal distance between samples will be smaller for error
trials, thus the uncertainty will be higher. For easy stimuli errors are rare because the
overlap is small, and the samples in this case will be close (high uncertainty).
CONFIDENCE MODEL CONTINUED
Dashed arrows signify the distance between error and correct
choices of a given difficulty.
The direction of solid arrows signifies whether error or correct
choices have higher rates for a given stimulus.
CONFIDENCE MODEL CONTINUED
Model
Data
ALTERNATIVE WAY TO DEFINE CONFIDENCE: ‘RACE’ MODEL
Decision confidence can be calculated from the difference between two decision
variables at the time a decision is reached.
CAN CONFIDENCE GUIDE ADAPTIVE BEHAVIOR ?
Confidence estimates derived solely from the decision variables in the
current trial can provide good estimates of the expected decision outcome across
trials. This indicates that confidence estimates are readily available in the rodent
brain. Can rats use this information behaviorally?
REINITIATION EXPERIMENT
CONCLUSIONS
- Firing rates of many neurons in the OFC match closely to the predictions of
confidence model.
- These firing rates cannot be readily explained by alternative mechanisms.
- Thus, confidence estimates are likely to be readily available even in the
rodent brain. When a decision is made, the brain not only makes a choice but
also generates an evaluation of uncertainty of that decision.
- Confidence estimates can drive adaptive behavior.
- Computation of subjective confidence may be a core component in
decision-making.
FURTHER QUESTIONS
- Two different classes of decision model yielded similar results. Other
methods for estimating confidence?
- OFC has been suggested to generate reward prediction and to signal
outcome risk. The data is consistent with both functions. Further experiments
will be needed to distinguish between these two alternatives.
-Do OFC neurons drive the reinitiation behavior directly, or through other
functions (controlling exploration, focusing attention etc.)?