Transcript Mental time travel and temporal discounting: The

Intertemporal choice, hyperbolic
discounting, and mental time travel:
A comparative and evolutionary discussion
Stephen Lea
Psychology, University of Exeter
UK
Aims of this paper
• To account for the key facts about human
inter-temporal choice within a framework of
evolutionary psychology
• To do that we must
– Know what the key facts are, perhaps beyond
the “stylised” level
– Go beyond “armchair adaptationism”
– Use modern data on both human and nonhuman preferences and cognition
ICABEEP Summer
School, 2010
Stephen Lea
2
A simple experiment
(which might not work)
€100
Now
ICABEEP Summer
School, 2010
€200
Later
Stephen Lea
3
The stylised fact about inter-temporal
choice
Humans are terrible at making choices between outcomes that
appear at different times in the future, for example:
• One marshmallow now vs. two marshmallows in 5 minutes’
time
• Extra sleep now vs. getting to work punctually in an hour’s
time
• Having a washing machine now vs. being debt-free next year
• Luxury expenditure now vs. enough income in retirement
• A cigarette now vs. living 20 years longer
Experimental data support this. It is by far the biggest deviation
from rational choice in the whole of economic behaviour
ICABEEP Summer
School, 2010
Stephen Lea
4
...aka
•
•
•
•
•
•
•
•
•
•
•
Failure to delay gratification
Lack of self control
Weakness of will
Yielding to temptation
Impulsiveness
Myopia
Short time-horizon
Inability to consider future consequences
[excessive] Delay discounting
Failure of self-regulation
...etc
ICABEEP Summer
School, 2010
Stephen Lea
5
In what ways are we bad at
intertemporal choice?
1. Impatience (Fisher, 1930): We choose the outcome
that will arrive sooner even when it is much less
valuable than the more delayed one, beyond any
point that could be justified by a rational analysis
2. Inconsistency (Ainslie, 1974): We change our
preferences between immediate and delayed
outcomes as they (both) come closer in time –
though this cannot happen if delay is discounted
exponentially, as rationally it should be
ICABEEP Summer
School, 2010
Stephen Lea
6
In terms of discount rates:
1. We have to set a “subjective discount
rate” at an absurdly high level (Friedman:
33%) to account for the (econometric)
facts of human saving (and even higher
for experimental data)
2. No single discount rate will account for all
the inter-temporal choices that an
individual makes even in one situation, let
alone across situations
ICABEEP Summer
School, 2010
Stephen Lea
7
A positive alternative
• These problems had been known for
decades – centuries even – but had had
little influence
• What made them bite was the proposal of
a positive alternative theory of
intertemporal (and other) choice:
– Positive in that it accounts for the anomalous
data
– Positive in that it forms part of a data-driven
science
ICABEEP Summer
School, 2010
Stephen Lea
8
Hyperbolic discounting
The non-exponential shape of
the hyperbolic discount curve...
...generates a discount rate that
is higher for outcomes further
into the future
But why pick that particular functional form?
ICABEEP Summer
School, 2010
Stephen Lea
9
The matching law
Picture of
Herrnstein
• Herrnstein’s matching law, formulated to
describe the behaviour of pigeons working on
concurrent variable interval schedules of
reinforcement:
R1/R2 = Rf1/Rf2
• Here Ri is a rate of responding (pecks/unit time)
and Rfi is a rate of reinforcement (food
deliveries/unit time)
• This is consistent with optimal foraging theory,
which stresses the rate of prey acquisition per
unit time
• In its original context (pigeons, conc VI VI
schedules, food reward) this is an immensely
robust relationship. Beyond that, it becomes
more open to question
ICABEEP Summer
School, 2010
Stephen Lea
10
Picture of Ainslie
Operant choice in
discrete trials
Picture of Rachlin
• In concurrent schedules, the two options are
continuously available
• However, many researchers (including Ainslie,
Rachlin, Killeen, Fantino, Davison, Mazur) wanted to
apply the matching law to discrete trial choice
• In a single trial, there is exactly one reinforcement,
after a delay Di ; and the natural analogue of
reinforcement rate is then simply 1/Di – leading to the
prediction that relative subjective value will be
discounted hyperbolically
ICABEEP Summer
School, 2010
Stephen Lea
11
Producing inconsistent choices
experimentally: Ainslie & Rachlin’s
“commitment” task for pigeons
A2
no choice
2nd
A2
choice point
If t0 is long enough,
birds that would make
the impulsive choice at
the 2nd choice point
will nonetheless make
the commitment choice
at the 1st choice point
1st choice point
A1
t0
ICABEEP Summer
School, 2010
t1
Stephen Lea
t2
12
Commitment and hyperbolic
discounting
Picture of
Mazur
• Ainslie and Rachlin’s commitment
result follows directly from the discretetrial version of the matching law
• Mazur shows that the following form is
more precise:
Vi = Ai/(1+KDi)
Where Vi is “subjective value”, Ai is reward
size, Di is delay, and K is a fitting constant
• Vi values can be used to predict choice
ICABEEP Summer
School, 2010
Stephen Lea
13
First steps in an evolutionary
analysis
• Matching is a consequence of basic laws of
operant conditioning / associative learning
• These processes are available in all vertebrates
that have been tested, and at least some
invertebrates, e.g. bees, molluscs
• This is what we would expect: these mechanisms
are evolutionarily conservative because they make
(nearly) optimal foraging possible, and almost all
animals need to forage
• But they will only work over the time and quantity
scales that are relevant to natural foraging
• Humans are no exception to the need to forage…
but…
ICABEEP Summer
School, 2010
Stephen Lea
14
Note!
• We have pulled off a significant trick here
• We have used a mechanism (matching) that
produces an explanation of a “near enough” and
optimal outcome (in foraging) to explain a
massive irrationality in intertemporal choice
• Yet foraging is inherently concerned with choices
in time – choices between streams of prey
availability
• The point is that foraging is concerned with rates
of prey capture / reward
• But also the time scales are different…
ICABEEP Summer
School, 2010
Stephen Lea
15
The methodological leap
• Mazur’s formula comes from detailed work with rats and
pigeons; we are interested in humans
• The delays concerned are a few seconds, the rewards a small
fraction of a daily food requirement; whereas in human
applications we may be interested in large sums of money over
periods of years or even decades
• Because of the short time intervals, in the animal experiments,
subjects can experience the choices trial after trial, for an hour or
more each day, for days and weeks on end
• In most human experiments, we ask “which of these would you
choose?” or at best “which of these will you have?” – once only
(though there are human operant discrete choice experiments)
• In the big applications, e.g. savings/pensions choices, none of us
can go back and try again having experienced the consequences
of our choices
ICABEEP Summer
School, 2010
Stephen Lea
16
Long-term choices in other animals
• Examples:
– birds gaining weight before migration
– hibernators gaining weight in the autumn
– hoarders (both scatter hoarders e.g. some corvids,
some squirrels, and larder hoarders e.g. hamsters)
spending much time and energy creating caches
in the food-rich season
• These behaviours are subserved by specific
physiological or instinctual mechanisms
and/or specific cognitive capacities (e.g.
seasonal change in size of stomach or
hippocampus)
• Such mechanisms have evolved because they
enable animals to exploit particular niches
(in these examples, seasonal variation)
ICABEEP Summer
School, 2010
Stephen Lea
17
Could we ever ask other animals to do the
delayed gratification tasks humans do?
• Whether the contingencies are hypothetical
or real, we ask people to imagine the future
• Lea (2001), Clayton et al (2003) point out
that this is closely linked to recollecting the
past – episodic memory (what happened,
where it happened, when it happened)
• Suddendorf & Corballis (1997, 2007) call
these capacities “Mental time travel” and
claim that they constitute a uniquely human
cognitive capacity
• Other names include “episodic future
thought” (Szpunar, 2010)
• Roberts (2002): “Are animals stuck in time?”
ICABEEP Summer
School, 2010
Stephen Lea
Picture of
Suddendorf
Picture of
Roberts
18
What, when and where?
“Rhymes with Orange” cartoon for 7th January 2010
ICABEEP Summer
School, 2010
Stephen Lea
19
Picture of
Clayton
“Episodic-like” memory
in non-humans
• Clayton, Dickinson and colleagues have
claimed to demonstrate what-where-when
memory in: scrub jays (scatter hoarders)
• Similar claims for other scatter hoarders e.g.
magpies (Smulders & colleagues) &
chickadees (Sherry & colleagues)
• Also male meadow voles, who have to keep
track of the location and status of multiple
females (Ferkin et al., 2008)
• However, also accumulating evidence for
what-where-when memory in rats (facultative
hoarders), but also pigeons and rhesus
monkeys (no known hoarding propensities)
ICABEEP Summer
School, 2010
Stephen Lea
Picture of a
meadow vole
20
Evolutionary analysis, Step 2
• In response to specific niche demands, some species have
evolved specific mechanisms that enable adaptive longterm choices
– Perhaps a capacity for mental time travel is one such mechanism, if
it turns out most species really don’t have it
• However, the only niches we know of that make that
demand are those with big annual variations in food
availability (or mate availability as in voles)
• This isn’t obviously a property of the East African
hunter/gatherer niche (though it could be)
• So why would our “adaptive toolbox” (Tooby &
Cosmides, Gigerenzer & Todd) include anything for
coping with it?
• Perhaps human mental time travel has a different origin?
– If so it might not be very accurate for making long-term choices
26th January
ICABEEP
Summer
2010
School, 2010
Stephen
Stephen
Lea:
Lea
Köln
21
Making the methodological leap
from the opposite direction
• Even if (some) animals do have (something like)
episodic memory, we don’t yet know how to ask
them questions about anticipation - so we still
can’t put animals into the situations we use with
humans
• But can we put humans into the situations we use
with animals?
• There have been many experiments testing the
matching law with humans
• Results are mixed…
ICABEEP Summer
School, 2010
Stephen Lea
22
Testing the matching law in humans
(samples from a large literature)
Picture
of Lowe
• Conger & Killeen (1974): “conversation”
situation, reinforcement by agreement – quite
good matching obtained
• Bradshaw et al. (e.g. 1979): concurrent schedule
for points reward (later exchanged for cash),
matching law generally upheld
• Horne & Lowe (1993): concurrent schedules for
points reward, only a minority of participants
match – and whether they do or not is closely
correlated with their understanding of the situation
as gleaned from post-experimental questionnaires
ICABEEP Summer
School, 2010
Stephen Lea
23
The human difference
• Humans have more than one mode of responding
in matching experiments
• If conditions favour “mindless” performance, the
laws of operant conditioning will hold
• But when people approach a task with a
verbalising, problem-solving mindset, quite
different results, with no parallel in animal studies,
may be seen: “rule-governed” rather than
“contingency-governed” behaviour”
ICABEEP Summer
School, 2010
Stephen Lea
24
Similar ideas…
• Opposition of reason to emotion and motivation
dating back to classical times and continuing to
William James
• Freud’s id vs. ego/superego
• Rules vs. associations in concept acquisition (e.g.
Ashby)
• Automatic vs controlled processing (Schneider &
Shiffrin, 1977 et seq.)
• Neurological theories (see Frank et al., 2009)
• Thaler’s planner vs. doer
• Habitual buying vs problem solving (Katona,
reviewed 1975)
Picture of
Katona
…and many others
ICABEEP Summer
School, 2010
Stephen Lea
25
Evolutionary analysis, Step 3
• For whatever reason (there are many hypotheses),
humans have evolved the capacity for reflective
thought.
• This gives us a consciousness of the self (Tulving
e.g. 2001: autonoetic consciousness)– and thereby
makes mental time travel over long periods
possible, in a new way
• However that is probably not its original or main
function
• So when we are comparing an imagined future
with a present reality, we don’t have anything in
our adaptive toolbox that will ensure we make a
wise choice…
ICABEEP Summer
School, 2010
Stephen Lea
26
Self-control and intra-psychic conflict
• Because we have two mechanisms available
for making many kinds of decisions, they
will sometimes lead in opposite directions
• When we use reflective thought to override
the response that would be made on the
basis of the laws of conditioning, we call
that self-control or self-regulation (though it
would better be called control by the self)
• This “mental tool” is used in many
situations – including inter-temporal choice
ICABEEP Summer
School, 2010
Stephen Lea
27
Picture of
Baumeister
Self-control as a limited
resource
• Baumeister and others have shown that
the capacity of self-regulation is a
limited resource
• It is required for many situations where
the “instinctual” response is not the
optimal one, e.g. moral behaviour
• It is easily reduced by use, but also by
fatigue, frustration, etc (“ego depletion”)
• The less we have available, the more
likely we are to make maladaptively
impulsive choices (among other choices
our “better selves” would prefer not to
make)
ICABEEP Summer
School, 2010
Stephen Lea
28
The origins of hyperbolic
discounting
• However, myopia in general and hyperbolic
discounting in particular are not the result of
self-control breaking down and operant
conditioning taking over
• If they were, the degree of myopia would be
even worse than it typically is
• But nothing in human evolution adapts us to
make accurate value comparisons over
delays of years: we are not (unlike squirrels
or stilts) natural inter-temporal choosers
ICABEEP Summer
School, 2010
Stephen Lea
29
So how do we discount?
• To assess the value of far distant rewards, we have to rely
on:
– Calculation, which depends on theoretical knowledge and
computational ability
– Intuition, which is likely to be guided by the laws of operant
conditioning, which will tell us how good an amount-delay
combination “feels”
• This means we are likely to observe:
– Substantial individual differences in both discount rates and
discount functions
– Substantial intra-individual inconsistencies
– Little correspondence between actual and rational discounting,
either in terms of discount functions or discount rates
• And there will always be a push towards the outcome
predicted by pure conditioning – a stronger push the more
ego-depleted we are
ICABEEP Summer
School, 2010
Stephen Lea
30
Evolutionary analysis, final step
• Humans are not natural inter-temporal choosers
• We get by, by using two tools that derive from our
capacity for imagination:
– Mental time travel
– Self control
• Because neither evolved under the pressure to
make good inter-temporal choices, they only do a
moderately good job at it.
• And operant conditioning mechanisms are always
pushing us towards myopia and hyperbolic
discounting.
ICABEEP Summer
School, 2010
Stephen Lea
31
Picture credits
Copyright photos (or those whose copyright status could not be
determined) are available as links to their original sites. The following
materials are used by permission of the photographers; they should not
be re-used without similar acknowledgement
• Grey Squirrel: photographs by SEGL
• Hyperbolic discounting graphs by Wikimedia Commons user Moxfyre,
used with permission
• Red knot photograph by Jan van de Kam from PLoS, used with
permission
• Western scrub jay photo by Jarek Tuszynski from Wikimedia
Commons, used with permission
• Polar bear photo by Ansgar Walk from Wikimedia Commons, used
with permission
• Swiss Army knife photo by Jonas Bergstein from Wikimedia
Commons, used with permission
ICABEEP Summer
School, 2010
Stephen Lea
32
References
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Ainslie, G. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21, 485-489.
Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological Review,
105, 442-481.
Babb, S. J., & Crystal, J. D. (2006). Episodic-like memory in the rat. Current Biology, 16, 1317-1321.
Baumeister, R. F., Bratslavsky, E., Muraven, M., & Tice, D. M. (1998). Ego depletion: Is the active self a limited resource? Journal of Personality and Social Psychology, 74,
1252-1265.
Bradshaw, C. M., Szabadi, E., Bevan, P., & Ruddle, H. V. (1979). The effects of signaled reinforcement availability on concurrent performances in humans. Journal of the
Experimental Analysis of Behavior, 32, 65-74.
Clayton, N. S., Bussey, T. J., & Dickinson, A. (2003). Can animals recall the past and plan for the future? Nature Reviews Neuroscience, 4, 685-691.
Clayton, N. S., & Dickinson, A. (1998). Episodic-like memory during cache recovery by scrub jays. Nature, 395.
Conger, R., & Killeen, P. (1974). Use of concurrent operants in small group research - demonstration. Pacific Sociological Review, 17, 399-416.
Feeney, M. C., Roberts, W. A., & Sherry, D. F. (2009). Memory for what, where, and when in the black-capped chickadee (Poecile atricapillus). Animal Cognition, 12, 767777.
Ferkin, M. H., Combs, A., del Barco-Trillo, J., Pierce, A. A., & Franklin, S. (2008). Meadow voles, Microtus pennsylvanicus, have the capacity to recall the "what", "where",
and "when" of a single past event. Animal Cognition, 11, 147-159.
Fisher, I. (1930). The theory of interest. New York: Macmillan.
Frank, M. J., Cohen, M. X., & Sanfey, A. G. (2009). Multiple systems in decision making: A neurocomputational perspective. Current Directions in Psychological Science,
18, 73-77.
Friedman, M. (1953). The methodology of positive economics. In M. Friedman (Eds.), Essays in Positive Economics, pp. 3-43. Chicago: University of Chicago Press.
Gigerenzer, G., & Todd, P. M. (1999). Fast and frugal heuristics: The adaptive toolbox. In G. Gigerenzer, P. M. Todd, and the ABC Research Group (Eds.), Simple heuristics
that make us smart. New York: Oxford University Press.
Herrnstein, R. J. (1961). Relative and absolute strengths of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267272.
Horne, P. J., & Lowe, C. F. (1993). Determinants of human performance on concurrent schedules. Journal of the Experimental Analysis of Behavior, 59, 29-60.
Katona, G. (1975). Psychological economics. New York: Elsevier.
Lea, S. E. G. (2001). Anticipation and memory as criteria for special welfare consideration. Animal Welfare, 10, S195-S208.
Mazur, J. E. (1984). Tests of an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology: Animal Behavior Processes, 10, 426-436.
Piersma, T., Dietz, M. W., Dekinga, A., Nebel, S., van Gils, J. A., Battley, P. F. & Spaans, B. (1999). Reversible size-changes in stomachs of shorebirds: when, to what extent,
and why? Acta Ornithologia, 34, 175-181.
Rachlin, H., & Green, L. (1972). Commitment, choice, and self-control. Journal of the Experimental Analysis of Behavior, 17, 15-22.
Roberts, W. A. (2002). Are animals stuck in time? Psychological Bulletin, 128, 473-489.
Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information-processing: 1: Detection, search, and attention. Psychological Review, 84, 1-66.
Suddendorf, T., & Corballis, M. C. (1997). Mental time travel and the evolution of the human mind. Genetic Social and General Psychology Monographs, 123, 133-167.
Suddendorf, T., & Corballis, M. C. (2007). The evolution of foresight: What is mental time travel, and is it unique to humans? Behavioral and Brain Sciences, 30, 299-351.
Szpunar, K. K (2010). Episodic future thought: An emerging concept. Perspectives on Psychological Science, 5, 142-162.
Thaler, R. H., & Shefrin, H. M. (1981). An economic theory of self-control. Journal of Political Economy, 89, 392-406.
Tooby, J., & Cosmides, L. (1992). The psychological foundations of culture. In J. Barkow, L. Cosmides & J. Tooby (Eds.) (Eds.), The adapted mind, pp. 19-136. New York:
Oxford University Press.
Tulving, E (2001). Episodic memory and common sense: how far apart? Proceedings of the Royal Society of London Series B, 356, 1505-1515.
Zentalll, T. R., Singer, R. A., & Stagner, J. R. (2008). Episodic-like memory: Pigeons can report location pecked when unexpectedly asked. Behavioural Processes, 79, 9398.
Zinkivskay, A., Nazir, F., & Smulders, T. V. (2009). What-Where-When memory in magpies (Pica pica). Animal Cognition, 12, 119-125.
ICABEEP Summer
School, 2010
Stephen Lea
33