Transcript Powerpoint Presentation for "The Fussy Brain"

The fussy brain:
What makes one option more attractive than another?
Steve Fleming and Louise Whiteley
Value
Some decisions are about information gathering, where
what matters is being accurate. Many everyday decisions
are about what is valuable to us now, and in the future…
OR
?
Predicting the future
What is a decision?
Prior Beliefs
Information
gathering
Short- vs
Long-term
gain
Value
Decision
Risk
Context
Bentham and probability
“Nature has placed mankind under
the guidance of two sovereign
masters; pain and pleasure. It is for
them alone to point out what we
ought to do, as well as to determine
what we shall do”
Jeremy Bentham, 1748-1832
Jeremy Bentham believed that
using “felicific calculus” it was
possible to work out the best
action to take
Darling’s investment – predicting the future
Up 10%
p = 0.2
p = 0.8
The value of the share can rise or fall…
Down
5%
Darling’s investment – predicting the future
Up 10%
p = 0.2
p = 0.8
Expected value of share = weight each outcome by its
probability, then add them all up
Down
5%
Darling’s investment – predicting the future
Up 10%
p = 0.2
p = 0.8
Expected value of share = weight each outcome by its
probability, then add them all up
EV(share) = S outcomes p(outcome) x r(outcome)
= (0.2 x 10) + (0.8 x -5) = -2
Down
5%
Darling’s investment – discounting the future
Share 1
p = 0.2
Up 25% In six months…
Value
Time
p = 0.8
Down
20%
OR
Share 2
p = 0.2
Up 15% In six weeks…
Value
Time
p = 0.8
Down
20%
Darling’s investment – discounting the future
Measuring impulsivity…
Value
Value
Time
Time
EV(share) = S outcomes p(outcome) x r(outcome)
EV(share) = S outcomes λ x p(outcome) x r(outcome)
How are these values learnt?
What is a decision?
Prior Beliefs
Information
gathering
Short- vs
Long-term
gain
Value
Decision
Risk
Context
+ Learning
Investigating value in the brain
1. Find neurons that signal our preferences
2. Work out how these neurons learn from
experience to predict future values
3. See how these neurons are affected by
probability
4. See how these neurons are affected by when
you get the reward
1. Find neurons that signal our preferences
LIP
MT
OFC
OFC – Orbitofrontal cortex
Neurons representing value of choice…
vs.
V(pineapple)
V(orange)
We want to know if OFC neurons can keep track of different
preferences
Preferences in the OFC
pineapple
orange
during
instruction
just before
reward
• Different groups of neurons within OFC are associated with different types of
reward (e.g. orange vs. pineapple)
• OFC neurons also know how much reward is on offer - e.g. six apples vs. one
piece of cake
Padoa-Schioppa & Assad (2006)
2. Work out how neurons predict future values
• Learn from the past!
New value = prediction + new information
= difference between prediction and what
happened…
So:
New value = prediction + α(outcome – prediction)
How does the brain predict future values?
Reward
unpredicted, reward
occurs
(outcome – prediction)
Reward predicted,
reward occurs
Reward
predicted,
reward absent
Schultz et al. (1997) Science
Changing our predictions with new
information
It’s corked
Time
Read label
Taste…
New information in the brain…
Basal ganglia
Seymour et al. (2004)
3. See how these neurons are affected by
probability
p = 0.8
p = 0.2
How does the brain respond to probability?
LIP
Platt & Glimcher (1997)
How does the brain respond to probability?
OFC
Basal ganglia
EV = S outcomes p(outcome) x r(outcome)
Knutson et al. (2005)
4. See how these neurons are affected by
when you get a reward
Would you like a) £900 now or b)
£1000 in one month’s time?
Short- and long-term gain in the brain
OFC
Kable & Glimcher (2007)
Brain data help us refine our theory
• Two theories:
–a) brain region knows about “absolute” value,
communicates it to somewhere else which knows about
how far away it is in time
–b) discounting the future is inherent to our value system
OFC
Kable & Glimcher (2007)
What is involved in making a decision?
Prior Beliefs
Information
gathering
Value
Short- vs
Long-term
gain
Decision
Risk
Context
What happens when things get more complicated…?
Many decision systems in parallel
We’ve been focusing on how the brain learns values from
experience, building up habits that can be used again
Many decision systems in parallel
Sometimes, we can’t learn habits, and need to look ahead in a
more sophisticated way…
Complicated or one-off decisions…
Many decision systems in parallel
And sometimes we don’t need to bother - we have innate
values attached to things like food and shelter
Complicated value
Bentham again…
“the game of push pin is of
equal value with poetry”
vs. J.S. Mill…
“it is better to be … Socrates
dissatisfied than a fool
satisfied”
Many decision systems in parallel
In the next talk we hear more about these three systems,
about how the brain chooses which system to use, and how
this can lead us astray…
Any questions...?
What is a decision?
Prior Beliefs
Information
gathering
Value
Short- vs
Long-term
gain
Decision
Risk
Context
Neurons representing value of choice…
Padoa-Schioppa & Assad (2006)