Transcript STDP

Learning crossmodal
spatial transformations
through STDP
Gerhard Neumann
Seminar B, SS 06
Overview






Network Model
Hebbian Learning and STDP
Properties of STDP
Cortical Maps
Learning Spatial Transformations
Papers:
 [Song99] : S. Song, L. Abbott, Competitive Hebbian Learning
through Spike-Timing-Dependent Plasticity
 [Song00] : S. Song, L. Abbott, Cortical Development and
Remapping through Spike Timing-Dependent Plasticity
 [Davison06]: A. Davison and Y.Fregnac, „Learning Crossmodal
spatial transformations through spike-timing-dependent plasticity.
Leaky Integrate + Fire Model

Membrane Potential Vj of neuron j:


Input



consists of:
Background noise
Excitatory Input (added):
Inhibatory Input (substracted):
Neuron Model

Excitatory Input:


Incremented by
following each spike
Inhibatory Input:



incremented by
by every spike
Simplified Version:


Direct change in synaptic current
More Complex Version
 Conductance Based IF Neurons used by Abbott
 Basically the same results
Hebbian Learning

Donald Hebb:

When an axon of cell A is near enough to excite cell B or
repeatedly or consistently takes part in firing it, some growth or
metabolic change takes place in one or both cells such that A’s
efficiency, as one of the cells firing B, is increased.

Correlation based learning

Not a stable rule:
Wij (t )  Oi (t )O j (t )


Weight normalization needed
No Competition

Usually we need a „global competition signal“


Not biologically realistic
Only for feed forward networks: No recurrent connections
possible
Spike-timing-dependent
plasticity


Synaptic plasticity is sensitive to the temporal oder
of the presynaptic and postsynaptic spike
Long Term Potentiation (LTP)



Long Term Depression (LTD)



If pre synaptic spike before post synaptic spike
Correlated Input
If post synaptic spike before pre synaptic spike
Random Input
Experiments with culture of rat Hippocampal cells
STDP: Time Window

Time Window: ~ 20 ms

Hard Bounds or Soft Bounds Model

w… either models the conductance for the synaptic input or
directly the change in the synaptic current



Area of Depression must be larger
than area of potentiation
Typical Values for:


: 20 ms
: 20 ms – 100 ms
STDP: Other Models

Other Models:

Symmetric STDP:



Inverse STDP



Short Time Intervalls: LTP
Long Time Intervalls: LTD
Reversed LTP/LTD
Also recurrent loops are possible
 Surpresses recurrent loops leading to a stable network
Mean Input to a Neuron should only be sufficient to charge the membrane
to a point below or only slightly above the treshold
 Postsynaptic Neuron fires primarily in response to statistical fluctuations
in the input
Basic Experiments with STDP

For a single Post Synaptic Neuron


STDP tends to segragate synaptic weights into
strong and weak groups (~ 50 %).
Competitive Nature of STDP
Basic Experiments with STDP

Effects of Different Correlation Times

- Dots…
- Triangles…

also works for larger Correlation times

Network Model:

1-D stimulus
 E.g. location of a touch stimulus
 Encoded in 1000 Input Neurons


Grid over the input stimulus
Firing rate: Gaussian Firing Curve



Population Coding
Use periodic boundary conditions
200 network neurons
 Sparse connectivity to the input neurons (20 %, random)
Learning procedure
 Input: Brief presentations of the stimulus at a random input
location
 Lasts ~ 20 ms (exponential distribution)


Maximum at the prefered stimulus location of the cell
Experiments:

Without any recurrent connections:


Each Neuron develops a random input selectivity
=> Nearby input neurons are correlated


Strengthing of synapses in one group of corr. inputs
Surpresses other input (competitive)
Experiments

Add all-to-all recurrent excitatory connections to the output
neurons



Initiliaze weights with zero
Selectivity and Column Structure, all neurons are selective in the
same neighborhood
Recurrent Connections are quite weak after convergence
Experiments

Seeding the network






Network neurons 81 – 120 were given initial input weights
for stimulus locations from 401 to 600
Recurrent synapses are strengthened before ff synapses
 Only one strongly correlated group of network neurons,
many correlated groups of input neurons
Seeded network neurons begin to drive unseeded network
Synapse from input neurons 401-600 to unseeded network
become strong
Recurrent synapses are weakened again
 FF synapses compete with recurrent synapses because of
shorter latency.
Seeded network units can be seen as sort of teacher signal
Experiments

Seeding the network
Cortical Map

Until now:



Competitive nature of STDP leads to a winner
take it all situation
Single column structure forms
We want: A Continuous Map



Restrict the spread of selectivity from one neuron
to another
Limit the recurrent excitatory connections
between network neurons to local neighborhoods
Add an initial seed to the FF connections
Experiment: Refinement of Maps

Seeded Cortical Map


Initialize FF connections with a coarse map
Map is tightened and refined.
Experiment:


Unseeded Case
Without initial seeding, a single column structure forms.
Map can also arise from random initial conditions if
inhibitatory connections are introduced

All to all uniform connections of fixed strength between network
neurons



Different neurons in the network tend to develop different location
preferences
Local excitatory connections favor similar preferences
=> formation of a smoothly changing cortical map
Experiment:


Unseeded Case
A Map like structure is formed
The Map can be arranged in either direction at any point in
the network (random initial conditions)
Learning Spatial
Transformations


Learn the transformation of a 1 or 2 DoF simulated
arm from proprioceptive input to the visual location
of the hand (end effector)
Network structure:


Three populations, each consisting of a one (or 2)
dimensional array of cells
 Input Population: Proprioceptive Stimulus
 Training Population: Visual Stimulus (Location)
 Output Population: Prediction
Connectivity:
 Input -> Output: All to All, learned with STDP
 Training -> Output: topographical and local, fixed
Learning Procedure

A random location is choosen,
the training reference
calculated



Use for the input population,
for the training population
E.g.
Again use Gaussian Firing
Curve
Experiment:

1 DoF arm: Evolution of weights

S-Shaped Band becomes visible in the matrix of
synaptic weights
Prediction:

Training Layer removed, sweep through the input space


Weight pattern from the input to the output layer remains stable
Good approximation found, extreme values are
underestimated
Experiment: Other Functions

Linear Functions
Non-Linear Functions
Experiment: 2 DoF

Also for the 2-D case, longer learning time
Influence of the population tuning curve shape

Poor accuracy of
trigonometric
functions near the
extremes
 Multiple input values
giving a single
output value
 Input values
compete with one
another
 => Make width of
the gaussian
population tuning
curve smaller
Influence of spatio temporal
learning

Drawbacks of the learning procedure:




Assumes that all point in space is visited with equal
frequency
 Tested with normal distribution
 Still works, slower convergence
Requires dwell times (time the input stays in a particular
location)
 Has to be approximately the same as the STDP time
constants
Ignores travel time between input locations, input location
changed smoothly
 Works if motion is fast enough
Inputs must fire together to produce potentiation, but
then quickly stop firing to avoid depression
Influence of the correlation time

Increase
or use symmetric STDP
Influence of signal latency


Latency
difference between
the input and the
training signal
For negative
latencies (training
before input), the
negative weight
pattern can emerge
Influence of Training Pathways


Precise range and strength of the connections to the training
population is not critical
Plasticity also for the training connections


Pattern is stable for prespecified initial connections
Similar to Map refinement from Song
Learning the training pathways

Start with random training weights



Recurrent (local) excitatory and inhibitory connections are needed
Similar to Map Learning with random initial conditions from Song
Connections develop with an arbitrary phase

Tends to wander as training continues
Experiments

Influence of plastic rules:




Symmetric: More robust to temporal structure
Asymmetric: Less dependent on the population
timing curve
Soft weight Boundaries: Modulation of the weight
pattern by correlation is weaker, but still learnt
Influence of network size and connectivity

Behavior of the network is the same by adjusting
other parameters
Summary

STDP: Biologically Inspired Correlation Based
learning


Population coding + STDP:



Competitive rule which strengthens connections to
correlated input, weakens connections to random input
Can be used to calculate cortical mappings
Can also learn (simple) spatial transformations
Quite complex model


Learning can also be done by a simple offline-linear
regression
Sensitive to a lot of parameters