Transcript Modeling working memory and decision making using generic

Modeling working memory and
decision making using generic
neural microcircuits
Prashant Joshi,
Institute for Theoretical Computer Science,
Technische Universität Graz,
Austria
Email: [email protected] | Web: www.igi.tugraz.at/joshi
Synopsis

Decision making is a recurring event in our day to day lives, and involves
three phases:
L – Initial loading of stimulus into working memory (WM),
 M – Maintaining the stimulus in WM for a couple of seconds,
 D – Making a binary decision on the arrival of second stimulus


A neurocomputational model using generic neural microcircuits with
feedback can integrate these three phases into a single unified framework!
Outline

Background & Methods

Computational principles
 Generic neural microcircuits
 What is a liquid state and who are the “neural users”?

Interval discrimination tasks



Two-interval discrimination (TID)
Delayed match-to-sample (DMS)
A glimpse of biology – the role of pre-frontal cortex in decision making

Modeling Results

Additional Results

Conclusions
Computational Principles
Dilemma – Analog fading memory has an upper limit on the
order of tens of msec whereas working memory holds
information on the order of seconds

New theoretical results [Maass, Joshi, Sontag, 2005] imply that generic neural
microcircuit with feedback has unexpected computational capabilities:



It can emulate any dynamical system, in particular any analog computer
Induces multiple co-existing “partial-attractor” states in the circuit dynamics
This work demonstrates that even in presence of feedback noise, such “partial
attractor” states can be held by generic neural microcircuits on the time-scales of
several seconds, which is obviously a requirement for tasks involving working
memory
Generic Cortical Microcircuits


Neurons – leaky integrate-and-fire neurons, 20% of them inhibitory, neuron a
is connected to neuron b with probability C.exp(-D2(a,b)/λ2)
Synapses – dynamic synapses with fixed parameters w, U, D, F chosen from
distributions based on empirical data from Henry Markram’s lab
What is a liquid state and who
are the “neural users”?
Each readout neuron receives as input a vector
x(t), which has as many components as it has the
pre-synaptic neurons in the circuit
The ith component of x(t) results from the spiketrain of the ith pre-synaptic neuron by applying a
low-pass filter, which models the low-pass
filtering properties of receptors and membrane of
the readout neuron
We assume that a readout neuron has at time t a
firing rate proportional to w.x(t)
Remark: Experimental results from the labs of
Nicolelis and Poggio show that such weighted
sums (from neurons in visual or motor cortex)
contain behavior-relevant information
Interval Discrimination Task





Classical experimental paradigm to study working
memory and decision making
Subject has to compare two stimuli (tactile, visual,
auditory etc.)
The experiment starts with presentation of the first
stimulus
Second stimulus is presented after a temporal delay
Subject needs to make a binary (yes/no) decision
after the presentation of the second stimuli, usually
by pressing a corresponding switch
Two-Interval-Discrimination

Subject receives two frequencies f1 and f2 and has to
decide if f1 > f2?
Delayed-Match-to-Sample



Cue stimulus (a color) is presented initially on the screen
Subsequently two probe stimuli (one identical to cue stimulus) are presented
Subject has to decide which probe stimuli has the same color as the cue
stimulus
A glimpse of biology – the role of
pre-frontal cortex in decision
making
f1 > f2?



Two kinds of neurons (+ and “-”) have
been observed in PFC which show
opposite activities is response to the above
question
The “+” neurons show an increase in their
activity during the decision phase when the
answer to the above question is “yes”
The “– “neurons show an increase when
this answer is “no”
Figure from Machens et. Al. Science, 307:1121-1124
Modeling Results

The generic neural microcircuit is used as a model for PFC

Inputs are fed to the circuit with a simple form of spatial coding (moving
Gaussian window)

Each input is fed to a layer of 100 neurons in the circuit

At each time step t, a signal-dependent noise was added to the input signal
(0.0001. ρ . f(t))
ρ is a random number drawn from a Gaussian distribution with mean 0 and SD
1
 f(t) is the current value of the input signal


Simple linear readouts trained by linear regression are used to model the
“+” or “-” neurons, and send feedback of their activity to the neural circuit
Modeling Two-intervaldiscrimination

400 neurons, arranged of a 20 x 5 x 4
grid

Circuit receives two external inputs (f1
and f2), and feedback from the readouts
modeling the “+” and “-” neurons

Total simulation time for one trial = 3.5
sec

Simulation time-step = 10 msec

10 pairs of input frequencies

f1 and f2 are presented for 0.5 sec each,
during the L and D phases respectively
Results - TID

f1 = 18 Hz, f2 = 26 Hz

Readouts perform
exceptionally well even in the
presence of feedback noise
(correlation of 0.9 and above)

Robustness – Setup is robust to
synaptic pruning – control
values (no pruning) for 10
validation runs, mean =
0.9956, SD = 0.0206

Interesting as traditional
models of attractor based
neural computation fail to
demonstrate robustness
Additional Results

Decision making followed by action selection is one of the most challenging events of our
daily lives

Involves 4 phases – 3 for decision making (L,M, and D), and one for acting on the decision
(called the A phase)

Neurocomputational models of working memory (including this talk till now) fail to address
how the decisions made by neurons in PFC are converted into motor commands, which are
executed by the sensori-motor system?

Similarly, models for computational motor control ignore the first three phases (L, M and D)

A neurocomputational architecture is proposed that integrates the four phases (LMDA)
involved in the process of action selection in presence of a decision

Essentially this model integrates two different cortical modalities – decision making carried
on by PFC, and subsequent action selection carried out by sensori-motor system

A spiking neural network model is presented for the DMS task followed by an arm movement
to the decided goal position
Modeling DMS followed by arm
movement
PFC Circuit:
•500 neurons, arranged of a 20 x 5 x 5 grid
•Circuit receives three external inputs (Csample, Cleft,
and Cright), and feedback from the readouts
modeling the “left” and “right” neurons
M1 Circuit:
• 1000 neurons (20 x 5 x 10)
• 4 external inputs (The x and y coordinates of the
center of left and right probe regions)
• the output of the “left” and “right” readouts from
PFC circuit
• feedback from readouts that predict joint angles
• feedback from readouts that compute joint torques
Arm Model:
• Standard model of a 2-joint robot arm
• Trajectory followed by the tip of the arm is
generated using the minimum jerk model
Simulation:
• Total simulation time for one trial = 2.5 sec
• Simulation time-step = 10 msec
• 5 triplets of input colors
Csample is presented for 0.32 seconds during the L
phase, and Cleft, and Cright are presented
simultaneously for 0.6 seconds, during the D phase
Results – DMS +
Arm Movement

Movement occurs during the A
phase (last 500 msec)

Biologically realistic bell-shaped
velocity profile observed

Performance – tested over 100
validation runs:






Left – mean = 0.9962, SD =
0.0018
Right – mean = 0.9942, SD =
0.0018
τ1 – mean = 0.9246, SD = 0.0247
τ2 – mean = 0.9571, SD = 0.0019
θ1 – mean = 0.9733, SD = 0.0282
θ2 – mean = 0.9887, SD = 0.0035
Results – DMS +
Arm Movement
Conclusions

A new neurocomputational paradigm is described that uses synaptic
learning mechanisms and is able to integrate the L,M,D and A phases
involved in decision making followed by action selection

Task independent and biologically realistic

Demonstrates the ability of generic neural microcircuit models to hold
“partial attractor” states

Robustness to factors such as synaptic pruning and feedback noise

Works for significant long time-scales in presence of noise because:


Generic neural microcircuits are inherently endowed with fading memory
Feedback enhances this
 Adding noise to teacher-feedback helps in making the target an attractor

Reflects the “internal model” hypothesis
References
1.
Machens , C.K., Romo, R., and Brody, C.D., 2005, Flexible control of mutual
inhibition: a neural model of two-interval discrimination, Science, 307:11211124
2.
W. Maass, P. Joshi, and E. D. Sontag. Computational aspects of feedback in
neural circuits. submitted for publication, 2006. (PDF, 1154 KB)
3.
W. Maass, P. Joshi, and E. D. Sontag. Principles of real-time computing with
feedback applied to cortical microcircuit models. In Advances in Neural
Information Processing Systems, volume 18. MIT Press, 2006. in press. (PDF,
806 KB)
4.
P.Joshi. Modeling working memory and decision making using generic neural
microcircuits. In Proc. of the International Conference on Artificial Neural
Networks, ICANN, 2006. in press
5.
P. Joshi. From memory based decisions to decision based movements: A model
of interval discrimination followed by action selection. submitted for
publication, 2006