Transcript Lea for EPS April 2013 on mental time travelx

Experimental Psychology Society
Lancaster Conference, April 2013
Symposium on Mental Time Travel
organised by Tony Dickinson
Intertemporal choice, hyperbolic
discounting, and mental time travel:
A comparative and evolutionary
discussion
Stephen E. G. Lea, University of Exeter
The Economic Problem:
The stylised facts about inter-temporal choice
Humans are terrible at making choices between
outcomes that appear at different times in the future
Everyday examples:
• Extra sleep now vs. getting to work punctually in an
hour’s time
• Having a washing machine now vs. being debt-free
next year
• Luxury expenditure now vs. enough income in
retirement
• A cigarette now vs. living 20 years longer
This is by far the biggest deviation from rational choice in
the whole of economic behaviour
EPS Lancaster, 11th April 2013
S. E. G. Lea
2
Experimental data support this, under
many different names:
•
•
•
•
•
•
•
•
•
•
Failure to delay gratification
Lack of self control
Weakness of will
Impulsiveness
Myopia
Short time-horizon
Inability to consider future consequences
[excessive] Delay discounting
Failure of self-regulation
...etc
EPS Lancaster, 11th April 2013
S. E. G. Lea
3
In what ways are we bad at
intertemporal choice?
1. Impatience (Fisher, 1930): We choose the
outcome that will arrive sooner even when it is
much less valuable than the more delayed one,
beyond any point that could be justified by a
rational analysis
2. Inconsistency (Ainslie, 1974): We change our
preferences between immediate and delayed
outcomes as they (both) come closer in time –
though this cannot happen if delay is discounted
exponentially, as rationally it should be
EPS Lancaster, 11th April 2013
S. E. G. Lea
4
In terms of discount rates:
1. We have to set a “subjective discount rate”
at an absurdly high level (Friedman: 33%) to
account for the (econometric) facts of
human saving (and even higher for
experimental data)
2. No single discount rate will account for all
the inter-temporal choices that an individual
makes even in one situation, let alone across
situations
EPS Lancaster, 11th April 2013
S. E. G. Lea
5
The psychological solution
• Suppose that the value of a future outcome
declines according a hyperbolic rather than an
exponential function of the time until it will be
realised
• This automatically generates a discount rate that
is higher for outcomes further in the future
• Hyperbolic discounting is now widely accepted by
(behavioural) economists. It is arguably the most
important concept to have been imported into
economics from psychology
EPS Lancaster, 11th April 2013
S. E. G. Lea
6
What is the psychological evidence for
hyperbolic discounting?
• Herrnstein’s matching law, formulated to describe the
behaviour of pigeons working on concurrent variable
interval schedules of reinforcement asserts that:
R1/R2 = Rf1/Rf2
• In this free-operant situation, Ri is a rate of responding
(pecks/unit time) and Rfi is a rate of reinforcement
(food deliveries/unit time)
• In discrete trial choice, there is exactly one
reinforcement, after a delay Di ; and the natural
analogue of reinforcement rate is then simply 1/Di –
leading to the prediction of hyperbolic discounting
EPS Lancaster, 11th April 2013
S. E. G. Lea
7
Commitment
• Ainslie and Rachlin used this analysis to design
experiments on pigeons, in which the birds showed
both impatience and preference reversal, and
choices were qualitatively consistent with the
discrete-trial version of the matching law
• Mazur shows that the following form is more precise:
Vi = Ai/(1+KDi)
Where Vi is “subjective value”, Ai is reward size, Di is delay,
and K is a fitting constant
• Vi values can be used to predict choice across a wide
range of animal choice experiments
EPS Lancaster, 11th April 2013
S. E. G. Lea
8
Evolutionary argument No. 1
• Matching is a consequence of basic processes of
operant conditioning / associative learning, common to
all vertebrates, and at least some invertebrates, e.g.
bees, molluscs
• It predicts “near enough” optimal behaviour in
standard foraging situations, e.g. the optimal diet
problem, faced by organisms of virtually all taxa
• We therefore predict that the laws of associative
learning, in general, and matching in particular, will be
evolutionarily conservative
• Humans should be no exception to any of this
EPS Lancaster, 11th April 2013
S. E. G. Lea
9
If you aren’t getting worried, you
haven’t been paying attention…
1. We are explaining life-time, one-off, deliberative,
human intertemporal choices by means of
phenomena demonstrated in short-term, highly
repetitive, contingency-governed, choices made by
rats and pigeons.
2. A key step in predicting the phenomena is the use of
the matching law, which in general does not hold for
human deliberative choices (Horne and Lowe)
3. The discount rates required to account for
commitment choices in the laboratory would be
grotesque if extrapolated to the life-time scale – even
by the standards of human intertemporal irrationality
EPS Lancaster, 11th April 2013
S. E. G. Lea
10
…and how reliable is hyperbolic
discounting anyway?
• Not all direct experimental tests of discount functions
show a hyperbolic form in all participants
• Not all market data are consistent with hyperbolic
rather than exponential discounting
• Not all demonstrations of preference reversal show
reversal in the direction predicted by hyperbolic
discounting (Read et al 2012)
• Intertemporal choices can be radically affected by
framing of the question (Read et al 2013)…
• …or by the comparison set made available (Vlaev et al
2007)
EPS Lancaster, 11th April 2013
S. E. G. Lea
11
Paradox
As psychologists, what we have to
explain is not why humans are so
bad at intertemporal choice, but
why we are so (relatively) good at
it… and not why hyperbolic
discounting occurs, but why it
sometimes doesn’t
EPS Lancaster, 11th April 2013
S. E. G. Lea
12
How, other than from the matching law,
might we get hyperbolic discounting (or
something like it)?
Two basic kinds of possibility
1. Humans have some mechanism, either
specific instincts or a specific cognitive
capacity, arising from our evolutionary past,
that deals with the far future (i.e. more than
a few seconds/minutes ahead)
2. Decisions about the far future are outside
our evolutionary experience, so we deal with
them as best we can using general-purpose
cognitive mechanisms
EPS Lancaster, 11th April 2013
S. E. G. Lea
13
Adaptations for long-term choices
• Examples:
– birds gaining weight before migration
– hibernators gaining weight in the autumn
– hoarders (both scatter hoarders e.g. some corvids,
some squirrels, and larder hoarders e.g. hamsters)
spending much time and energy creating caches in
the food-rich season
Red knot. Photograph by Jan van de
Kam from PLoS, used with permission
• These behaviours are subserved by specific
physiological or instinctual mechanisms and/or
specific cognitive capacities (e.g. seasonal
change in size of stomach or hippocampus)
EPS Lancaster, 11th April 2013
S. E. G. Lea
14
A common selective pressure:
Seasonal variation
•
•
Such mechanisms have evolved because
they enable animals to exploit particular
niches, and in all these examples, it is
seasonal variation in food supply
It’s conceivable that
1. During human evolution we had to cope with
such seasonal variations
2. The mechanism for doing so is capable of being
extended beyond its natural one-year limitation
EPS Lancaster, 11th April 2013
S. E. G. Lea
15
Life history strategies
• However many animals also make “choices” between
patterns of behaviour that affect the entire course of
their lives, not just survival in the coming season
• In evolutionary theory these patterns are referred to
as life-history strategies
• A very common field for such choices is the timing of
reproduction (the r/K decision axis)
• Although these differences in strategy are clearly
facultative, to what extent can we regard them as
being chosen? It’s plausible that they are subserved
by evolved instincts
EPS Lancaster, 11th April 2013
S. E. G. Lea
16
Human reproductive choice
• Humans are long-lived, reproduce slowly, and
occupy many different ecological niches: these
all predict multiple life-history strategies
• In existing hunter-gatherer societies, birth rate
is well below the maximum achievable,
implying some kind of trade-off to achieve
maximum fitness (e.g. Kaplan, 1996)
• In all modern societies, birth rate has crashed
(the demographic transition), implying that
the trade-off is labile
EPS Lancaster, 11th April 2013
S. E. G. Lea
17
Reproductive choice and economic
choices
• Modern humans vary in speed of reproduction strategies
• These variations are correlated with social class, economic
status and other indicators of environmental harshness and
unpredictability (e.g. Quinlan, 2008; Ellis et al., 2009;
Nettle, 2010)
• Increased use of credit and reduced saving are associated
with the fast-reproduction (r) strategy
• However direct experimental tests or a correlation between
life history strategies and high discount rates do not reliably
support predictions (e.g. Wilson & Daly, 2006)
EPS Lancaster, 11th April 2013
S. E. G. Lea
18
What about general cognitive
mechanisms?
• In delayed gratification tasks, whether the
contingencies are hypothetical or real, people
need to imagine the future
• This is closely linked to recollecting the past –
episodic memory
• Suddendorf & Corballis (1997, 2007) call these
two capacities “Mental time travel” and claim that
they constitute a uniquely human cognitive
capacity (autonoetic consciousness: Tulving, 1985)
• The forward looking version of it is referred to as
anticipation or episodic future thought
EPS Lancaster, 11th April 2013
S. E. G. Lea
19
Temporal discounting and mental
time travel
• Stewart, Chater & Brown (2003) propose Decision by
Sampling (DbS) theory
• According to DbS, we compare a proposed delay with
a sample of delays drawn from memory
• Because long delays are experienced less often than
short delays, the distribution in the sample follows a
power law (cf. Zipf’s law, etc)
• This yields (sub)hyperbolic discounting even with
random sampling from memory, without any
underlying psychoeconomic scale
EPS Lancaster, 11th April 2013
S. E. G. Lea
20
Evolutionary argument No. 2
• For whatever reason (there are many hypotheses),
humans, and perhaps a few other species, have
evolved the capacity for reflective thought.
• This gives us a consciousness of the self – and
thereby makes mental time travel over long periods
possible
• This allows us to compare an imagined future with
present reality; this might be the selective pressure
that caused the capacity to evolve
• But whatever pressures produced this capacity, they
did not involve modern, finance-based, choices, so
there is no reason to expect these to be made
optimally or accurately
EPS Lancaster, 11th April 2013
S. E. G. Lea
21
An inevitable complication: the need
for self-control
• Associative learning still works in humans, even if we
have other cognitive processes available in addition
• So we have two mechanisms available for making
many kinds of decisions – and they will often lead in
opposite directions
• When we use reflective thought to override the
response that would be made on the basis of the
laws of conditioning, we call that self-control or selfregulation (though it would be better to call it control
by the self)
• This “mental tool” is used in many situations –
including inter-temporal choice
• There is much evidence that it is cognitively and even
physiologically expensive to use
EPS Lancaster, 11th April 2013
S. E. G. Lea
22
But (lack of) self-control is not the core
problem
• Myopia and preference reversals occur even when
self-control has not broken down. When operant
conditioning completely takes over, the degree of
myopia is even worse
• There are plausible evolutionary accounts of mental
time travel, and its use to make value comparisons
over months or years
• Hyperbolic discounting can be predicted by a general
cognitive mechanisms, Decision by Sampling theory
• No mechanism has been proposed that would lead
to the “economically rational” solution, exponential
discounting
EPS Lancaster, 11th April 2013
S. E. G. Lea
23
So how do humans discount?
To assess the value of far distant rewards, we have
to rely on:
• Self control, to avoid the hyper-myopia
predicted by the matching law
• Episodic future thought, to imagine the future
situation
• And to assess its value
– Either Decision by Sampling (or something like it)
– Or rational calculation, if we have the theoretical knowledge
and computational ability , not to mention the required faith
in economists
EPS Lancaster, 11th April 2013
S. E. G. Lea
24
This means we are likely to
observe:
• Substantial individual differences
• Substantial intra-individual
inconsistencies
• Varying forms of the discount function
• Little correspondence between actual
and rational discounting, either in terms
of discount functions or discount rates
QED
EPS Lancaster, 11th April 2013
S. E. G. Lea
25
Thanks to…
• Detlef Fetchenhauer, Erik Hölzl and their
students at Köln, and Stephanie Preston and
all at the Michingan Meeting on the
Interdisciplinary Science of Consumption, for
discussion of earlier versions of this paper
• Paul Webley, Lesley Newson, Avril Mewse,
Wendy Wrapson, Gordon Brown, Peter Fischer
and David Crelley for discussion of essential
ideas
EPS Lancaster, 11th April 2013
S. E. G. Lea
26
References
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Ainslie, G. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21, 485-489.
Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological
Review, 105, 442-481.
Barron, G., & Erev, I (2003). Small feedback-based decisions and their limited correspondence to description-based decisions. Journal of Behavioral Decision Making,
16, 215-233.
Baumeister, R. F., Bratslavsky, E., Muraven, M., & Tice, D. M. (1998). Ego depletion: Is the active self a limited resource? Journal of Personality and Social Psychology,
74, 1252-1265.
Clayton, N. S., Bussey, T. J., & Dickinson, A. (2003). Can animals recall the past and plan for the future? Nature Reviews Neuroscience, 4, 685-691.
Conger, R., & Killeen, P. (1974). Use of concurrent operants in small group research - demonstration. Pacific Sociological Review, 17, 399-416.
Fisher, I. (1930). The theory of interest. New York: Macmillan.
Frank, M. J., Cohen, M. X., & Sanfey, A. G. (2009). Multiple systems in decision making: A neurocomputational perspective. Current Directions in Psychological
Science, 18, 73-77.
Friedman, M. (1953). The methodology of positive economics. In M. Friedman (Eds.), Essays in Positive Economics, pp. 3-43. Chicago: University of Chicago Press.
Gigerenzer, G., & Todd, P. M. (1999). Fast and frugal heuristics: The adaptive toolbox. In G. Gigerenzer, P. M. Todd, and the ABC Research Group (Eds.), Simple
heuristics that make us smart. New York: Oxford University Press.
Herrnstein, R. J. (1961). Relative and absolute strengths of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4,
267-272.
Horne, P. J., & Lowe, C. F. (1993). Determinants of human performance on concurrent schedules. Journal of the Experimental Analysis of Behavior, 59, 29-60.
Kaplan, H (1996). A theory of fertility and parental investment in traditional and modern human societies. Yearbook of Physical Anthropology, 39, 91-135.
Katona, G. (1975). Psychological economics. New York: Elsevier.
Lea, S. E. G (1979). Foraging and reinforcement schedules in the pigeon: Optimal and non-optimal aspects of choice. Animal Behaviour, 27, 875-886.
Lea, S. E. G. (2001). Anticipation and memory as criteria for special welfare consideration. Animal Welfare, 10, S195-S208.
Nettle, D. (2010). Dying young and living fast: variation in life history across English neighborhoods. Behavioral Ecology, 21, 387-395.
Mazur, J. E. (1984). Tests of an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology: Animal Behavior Processes, 10, 426-436.
Piersma, T., Dietz, M. W., Dekinga, A., Nebel, S., van Gils, J. A., Battley, P. F. & Spaans, B. (1999). Reversible size-changes in stomachs of shorebirds: when, to what
extent, and why? Acta Ornithologia, 34, 175-181.
Quinlan, R. J. (2008). Human Pair-Bonds: Evolutionary Functions, Ecological Variation, and Adaptive Development. Evolutionary Anthropology, 17, 227-238.
Rachlin, H., & Green, L. (1972). Commitment, choice, and self-control. Journal of the Experimental Analysis of Behavior, 17, 15-22.
Roberts, W. A. (2002). Are animals stuck in time? Psychological Bulletin, 128, 473-489.
Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information-processing: 1: Detection, search, and attention. Psychological Review, 84, 1-66.
Suddendorf, T., & Corballis, M. C. (1997). Mental time travel and the evolution of the human mind. Genetic Social and General Psychology Monographs, 123, 133167.
Suddendorf, T., & Corballis, M. C. (2007). The evolution of foresight: What is mental time travel, and is it unique to humans? Behavioral and Brain Sciences, 30, 299351.
Thaler, R. H., & Shefrin, H. M. (1981). An economic theory of self-control. Journal of Political Economy, 89, 392-406.
Wilson, M., & Daly, M (2006). Are juvenile offenders extreme future discounters? Psychological Science, 17, 989-994.
EPS Lancaster, 11th April 2013
S. E. G. Lea
27