Transcript 04_West_SAP08

UNCLASSIFIED / FOR OFFICIAL USE ONLY
Decision Making and Stochastic
Delay
at
Workshop on Social Computing, Behavior Modeling
and Prediction
1 April 2008
Dr. Bruce J. West
Chief Scientist
Mathematical & Information Science Directorate
Army Research Office
[email protected]
919-549-4257
UNCLASSIFIED / FOR OFFICIAL USE ONLY
Decision Making and Delay
Outline of talk
•
•
Discounted Utility Model & intertemporal choice
Anomalies from discounted utility theory
– irrationality
– hyperbolic discounting
•
Objective and subjective time
– entropy and the direction of time
– time as a stochastic variable
•
Individuality and paternalism
– some experiments
– fit of theory to data
•
Conclusions
Decision Making and Delay
Discounted Utility Model (DUM)
T
U t u t , u t 1 ,..., uT      t u
 1
• Discount factor δ compresses many mechanisms
• mortality, uncertainty, time compression,…
• Accepted as both normative (how things should be) and
descriptive (how things are)…..but was initially arbitrary
Samuelson (1937).
• Exponential form implies time consistency (rationality)
Decision Making and Delay
Anomalies from DUM
• Time inconsistency
– empirical discount factor is not constant
• over time
• across type of intertemporal choices
•
•
•
•
•
•
Delay effect (hyperbolic discounting)
Interval effect (non-stationarity)
Sign effect (gains vs. loses)
Magnitude effect (small vs. large)
Direction effect
Sequence effects (ordered set vs. single)
Decision Making and Delay
Model comparison
• Exponential delay model
– monotonic decrease in value
with objective time
– constant rate results in time
consistency
– rationality
• Hyperbolic delay model
– decreasing rate results in time
inconsistency
– irrationality (preference reversal)
hyperbolic
exponential
exponential
hyperbolic
hyperbolic
exponential
Decision Making and Delay
Objective vs. subjective time
• Hyperbolic models
– Objective time
• clockwork universe
• entropy and the direction of time
– Subjective time
• unidirectional
• probability and statistics
• Motivate decision-making using
ensemble distributions
– subjective time
– stochastic delays
Decision Making and Delay
Delay and uncertainty
•
Decision-making models of intertemporal choice can be extended
to incorporate probabilistic choice U(t)  U( 0 )F(p) where p is the
probability of reward at time t and F is an unspecified function.
discrete
No reward before delay time t

t      d
t
•
ψ τ dτ

p
•
continuous

Delay-time probability density
d t 
 t   
dt
U(t)=U( 0 )Ψ t 
Decision Making and Delay
Stochastic rate
• Deterministic discount rate r     F '   is replaced with a conditional
F  
probability per unit time
1
 t 
r (t )  lim
Pr obt    t  t t    
t 0 t
 t 
• The ratio of the delay time distribution function to the survival probability
density, integrates to
t

 t   exp   r t 'dt '
 0

• The utility function in terms of subjective time is therefore
t

U t   U 0t   U 0 exp   r t 'dt '
 0

Decision Making and Delay
Example rate
• Rate of reward production suggested by hyperbolic
distribution
r0
r (t ) 
•
1  r1t
Probability of no reward before time t is
 t r0 dt ' 
1
 T 
t   exp  



r0


 0 1  r1t '  1  r1t  r1  T  t 

so that the utility function is inverse power law

 T 
U (t )  U (0) 
;

T  t 
•
T=
1
;  = r0T
r1
T measures response time and α measures irrationality
Decision Making and Delay
Experimental data
• Students (20) asked to make decisions about hypothetical
money to be received immediately or at a later time, concerning
the subjects themselves or another person not known to them.
Takahashi, Physica A (2007).
self
T=31
α=0.28
other
T=1.85
α=0.11
Decision Making and Delay
Implications from experiments
• The response times could describe paternalistic policy
making government officials, where irrationality is
enhanced.
• Irrationality is nowhere more significant than in the
military where choices may determine whether others
live or die.
Decision Making and Delay
• Nonlinear dynamic equation
(0,1).
dq
 aq z
dt
solved on the interval
q 0  1  1  z a 
1
1 z
• define a delay-time distribution density
p(q0 )dq0    d
• assume a uniform distribution of initial conditions to obtain
      1
T  1
T   

;   1
• delay-time distribution density is non-Poisson, renewal and
non-ergodic
Decision Making and Delay
Measured discount rates
•
Higher discount rates compared with controls* ( smaller T
and α in stochastic intertemporal model)
–
–
–
–
–
–
•
smoking
excessive alcohol consumption
illicit drug use (cocaine, crack-cocaine and heroin)
pathological gambling
age
cognitive ability (negative correlation with intellectual
achievement)
Consistent with neuroeconomic hypothesis that prefrontal
cortex is essential for patient (forward looking) decision
making.
* Chabris, Laibson & Schuldt, The New Palgrave Dictionary of Economics
(2007).
Decision Making and Delay
Brain Activity
β network”: midbrain dopamine network; reward processing
(ventral striatum V.Str. and medial prefrontal cortex mPFC.)
δ network: cognition; dorsolateral prefrontal cortex (diPFC)
and right posterior parletal cortex (R.Par.)
Sanfey, Loewenstein, McClure and Cohen, TRENDS in Cognitive Science (2006).
Decision Making and Delay
More brain activity
• Two discounting slopes
• < one year
• > one year
 00.5.5
  0.15
  0.5
• Different parts of the brain light
up under fMRI
• short-term
• long-term
Wittmann & Paulus (2007)
Decision Making and Delay
Conclusions and Speculations
• decision-making is not always rational
• irrationality in intertemporal choice models take a hyperbolic
form
• inverse power laws or hyperbolic utility functions can be
generated by stochastic delay times
• different parts of the brain control decisions associated with
long and short delay times
• the complexity of the brain produces the subjective nature of
biological time