Transcript Foraging, Learning & Genes

Foraging, Learning & Genes
“Optimal Foraging”
Behavior adjusts to environment
Diet: Density of preferred prey
Risk-sensitivity: (Expected – Required) Intake
Seek general predictions/understanding
Foraging, Learning & Genes
Mechanisms: Intrinsic Constraints on Behavior
Genes, nervous system, physiology
Diverse among species (vs generality)
Initial integration
Foraging, Learning & Genes
Positive & Negative Reinforcement
Reinforcement: value, penalty
Currency of fitness (?)
Positive: Increases frequency of behavior
Negative: Decreases frequency of behavior
“Once burned, twice shy”
Some people slow to learn from penalty
Klein, T.A. et al. (2007) Genetically determined
differences in learning from errors. Science
318: 1642-1645.
Klein et al. (2007)
D2 receptor
Protein on surface of human brain cells
Activate by neurotransmitter dopamine
A1: allelic variant of gene
Single base-pair difference from “normal”
Klein et al. (2007)
A1: allelic variant of gene
Reduces D2 density by 30%
Linked to “insensitivity of consequences of selfdestructive behavior” (?)
Learning from errors
Klein et al. (2007)
Learning: reinforcement outcomes
Performance-monitoring system
posterior medial frontal cortex
has rostral cingulate zone (RCZ)
RCZ: learning form errors
Hypothesis: midbrain sends dopamine signal to RCZ
Outcome better or worse than expected
Enables associative learning
Klein et al. (2007)
26 healthy German males
12:  A1 allele (reduced D2 density)
Chose between pairs of ideograms
“Reward or punishment” resulted
Functional magnetic resonance imaging
RCZ & hippocampus
Klein et al. (2007)
Learned to choose high Pr[reward] over low Pr
Post-test: Avoid lower of low Pr[reward]
A1: Same response to positive reinforcement
Reduced learning via negative reinforcement
Klein et al. (2007)
Klein et al. (2007)
Klein et al. (2007)
See genetic difference associated with
difference brain activity & avoidance learning
D2 receptor important “reward learning”
Genetic variation for choice behavior
Risk-sensitivity
Reward amounts, scaled to requirements
Effects of mean and variance on choice
Variance in delay (time) to obtain given reward
Different problem; more complex
Behavioral Ecology & Microeconomics Intersect
Neuroeconomics, Risk & Brain
Objective value of reward (x)
Subjective value of reward, “utility” (U)
U(x) nonlinear; necessary for risk-sensitivity
Neuroeconomics, Risk & Brain
Neuroeconomics, Risk & Brain
McCoy, A.N. & Platt, M.L. 2005. Risk-sensitive
neurons in macaque posterior cingulate cortex.
Nature Neuroscience 8:1220-1227.
Posterior cingulate cortex (CGp)
Limbic area: integrates
Link reward to with spatial attention (visual)
McCoy, A.N. & Platt, M.L. (2005)
2 Adult rhesus macaques
Choose between 2 visual targets
Fruit juice reward; certain and variable
Ecological validity?
Recorded activity single neurons in CGp
McCoy, A.N. & Platt, M.L. (2005)
McCoy, A.N. & Platt, M.L. (2005)
Risk-prone for reward amounts, fruit juice
U(x) = Mean reward + c Risk
Risk: Coefficient of variation
McCoy, A.N. & Platt, M.L. (2005)
Posterior cingulate
neurons Risk-prone
Strongest signals when
eye movements go to
symbol for variable
reward