Transcript The basic Hebb rule

LECTURE 10
Plasticity and Learning
I. Introduction
II. Synaptic placticity rules
− The basic Hebb rule
− The covariance rule
− BCM Rule
− Non-Hebbian rules
− Anti-hebbian rules
− Timing-based Rules
Introduction
Hebb’s postulate
“When an axon of cell A is near enough to excite cell B
or repeatedly or persistently takes part in firing it, some growth process
or metabolic change takes place in one or both cells such that A’s
efficiency, as one of the cells firing B, is increased”.
Donald O. Hebb (1949)
The theory is often summarized as "cells that fire together,
wire together".
• Conditioning:
－ The first attempt to model conditioning in terms of
synaptic change.
－ Behavior ---?--- neural mechanisms
• Development:
－ The formation and refinement of
neural circuits need synaptic
elimination.
－ Axonal or synaptic competition in
neuromuscular junctions and visual
system (Consumptive and interference
competition)
• Long term potentiation (LTP) - Long term depression （LTD）
－ Changes that persist for tens of minutes or longer are
generally called LTP and LTD. It lasts for hours in vitro
and days and weeks in vivo
－ The longest-lasting forms appear to require protein
synthesis.
－ First found in Hippocampus
－ The physiological basis of Hebbian learning
－ Properties and mechanisms of long-term synaptic
plasticity in the mammalian brain may relate to learning
and memory.
－ Inhibitory synapses can also display plasticity, but this
has been less thoroughly investigated both
experimentally and theoretically
I. Introduction
II. Synaptic placticity rules
− The basic Hebb rule
− The covariance rule
− BCM Rule
− Non-Hebbian rules
− Anti-hebbian rules
− Timing-based Rules
The basic Hebb rule
dwi (t )
w
 axi (t ) y (t )
dt
pre i
(a  0 )
wi
learning rate
xi and y : the firing rates of the pre- and postsynaptic neurons
1. Local mechanism
2. Interactive mechanism
3. Time-dependent mechanism
post
The basic Hebb rule is unstable
pre
dw
w
 xy
dt
2
dw
dw
2
w
 2 w w
 2w  xy  2 y  0
dt
dt
w
post
1. The processes of synaptic plasticity are typically much slower than the
neural activity dynamics.
2. If, in addition, the stimuli are presented slowly enough to allow the
network to attain its steady-state activity during training,

dy
  y   wj  f (x j )
dt
j
Assume f ( x j )  x j , we have y   w j  x j w  x
j
Theoretically, an upper saturation constraint must be imposed to avoid
unbounded growth. But experimentally,
LTP and LTD at the Schaffer collateral inputs to the CA1
region of a rat hippocampal slice
Is it due to that the basic Hebb rule has no LTD? Let’s add LTD by
introducing the covariance rule
I. Introduction
II. Synaptic placticity rules
− The basic Hebb rule
− The covariance rule
− BCM Rule
− Non-Hebbian rules
− Anti-hebbian rules
− Timing-based Rules
The covariance rule
dw
w
 x( y   y )
dt
postsynaptic threshold, e.g.  y 
dw
w
 (x   x ) y
dt
presynaptic threshold, e.g.
x
dw
w
 (x  x ) y 
dt
 (x  x )( x  x )  w
the input covariance matrix,
dw
w
 x( y   y )
dt
(homosynaptic depression)
dw
w
 ( x   x ) y
dt
(heterosynaptic depression)
• By the basic Hebb rule, synapses are modified whenever correlated preand postsynaptic activity occurs. Such correlated activity can occur purely
by chance, rather than reflecting a causal relationship that should be
learned. To correct for this, the covariance rather than correlation-based
rule is often used by network models
• Although the covariance rule allows LTD and reflects a causal preand postsynaptic relationship it is still unstable due to positive
feedback
The covariance rules, like the Hebb rule, are
unstable and non-competitive
dw
w
 ( x   x ) y
dt
w
dw
2
dw
 2w  w
 2w  ( x   x ) y
dt
dt
 2( y  w  x ) y
pre
w
post
Average above equation over the training period:
w
dw
dt
2
 2( y 2    y  y )  0
Competition can be introduced to allowing threshold to slide as follows
I. Introduction
II. Synaptic placticity rules
− The basic Hebb rule
− The covariance rule
− BCM Rule
− Non-Hebbian rules
− Anti-hebbian rules
− Timing-based Rules
BCM Rule
Bienenstock, Cooper and Munro (1982) proposed an alternative for
which there is experimental evidence where the postsynaptic threshold is
dynamic
dw
w
 xy ( y   y )
dt
Hebb rule
covariance rule
One example:

d y
dt
 y  y
Usually set:
   w
LTP
2
0
LTD
y
BCM rule
Postsynaptic activity
－ This is again unstable if  is fixed.
－ However, if the threshold is allowed to grow faster than v we get
stability.
－  depends on postsynaptic activity. For instance, the threshold
for LTP decreases when postsynaptic activity is low (y ↓
↓)
－ Here competition
between synapses appears
since strengthening some
synapses results in threshold
increasing meaning that it is
harder for others to be
strengthened

LTP
0
LTD
Postsynaptic activity
Synaptic weight normalization
w
j
 constant
j
2
w
 j  constant
j
－It is a more direct way of enforcing competition
－Idea is that postsynaptic neuron can only support a certain amount
of total synaptic weight so strengthening one leads to weakening
others
－2 types: subtractive normalisation and multiplicative
normalisation
Subtractive normalisation
dw
y
w
 xy  n
dt
Nx
x
j
(n  x   x j ; n  w   w j )
j
j
j
or :
dwi
y
w
 xi y 
dt
Nx
x
j
j
w
j
 constant
j
－It is easy to prove that the total increase in the weights is 0.
Evidences for
BCM rule
Evidence for a sliding
threshold:
It is easier to obtain LTP
in the cortex of darkreared animals and it is
harder to induced LTD in
these cortices
－ The field potentials evoked in layer III by layer IV stimulation in slices
of visual cortex prepared for light-deprived and control rats 4-6 weeks
old
－The effects can be reserved by as little as two days of light exposure
before slice preparation
Experimental evidences for constant total
synaptic weights
－ Low- and high-frequency BLA stimuli (LFS, HFS) are known
to, respectively, produce homosynaptic NMDA dependent LTD
and LTP in ITC cells.
－ Whether LFS and HFS also produce inverse heterosynaptic
modifications is unclear.
(Royer and
Pare 2003,
Nature)
• slices of the amygdala
• guinea-pigs (3–5 weeks old)
• intercalated (ITC) neurons of the amygdala: 中间神经元
• the basolateral amygdala (BLA): 基底外侧杏仁核
• an array of closely spaced (~150 μm) stimulating electrodes
• Homosynaptic LTP was induced with
HFS paired to postsynaptic depolarization.
Postsynaptic depolarization was achieved
by applying short (2ms) depolarizing
current pulses (0.2 nA) timed so that BLAevoked EPSPs would occur just before or
during current-evoked spikes
(Royer and Pare 2003, Nature)
Plot of EPSP amplitude and rise
time versus stimulation site
LTD
induction
produces
heterosyna
ptic LTP
(Royer and
Pare 2003,
Nature)
Left: Difference between pre- and post-LFS response profiles
(EPSP amplitudes) for one cell (top) and average of all cells
Right:Time course of changes in response amplitude
Result is
similar
with high
frequency
stimuli
(Royer and Pare 2003, Nature),
－ Their results showed that the activity-dependent
enhancement or depression of particular inputs to intercalated
neurons is accompanied by inverse modifications at
heterosynaptic sites, which contributes to total synaptic weight
stabilization
－ The inverse homo- versus heterosynaptic plasticity seems to
be a cell- wide event, which needs an intracellular signaling
system that can render synapses ‘aware’ of each other or of the
mean neuronal activity.
－ How do unstimulated inputs detect the stimulation frequency
at the stimulated pathway?
I. Introduction
II. Synaptic placticity rules
− The basic Hebb rule
− The covariance rule
− BCM Rule
− Non-Hebbian rules
− Anti-hebbian rules
− Timing-based Rules
Non-Hebbian forms of synaptic plasticity
• They modify synaptic strengths solely on the basis of pre- or
postsynaptic firing, are likely to play important roles in
homeostatic, developmental, and learning processes
• Homeostatic plasticity
－It allows neurons to sense how active they are and to adjust
their properties to maintain stable function
－Loosely defined, a homeostatic form of plasticity is one that
acts to stabilize the activity of a neuron or neuronal circuit
in the face of perturbations, such as changes in cell size or
in synapse number or strength, that alter excitability.
－ A large number of plasticity phenomena have now been
identified (e.g., synaptic scaling and homeostasis of
intrinsic excitability of neurons)
Synaptic scaling
− A form of synaptic plasticity that adjusts the strength of all of a
neuron's excitatory synapses up or down to stabilize firing,
avoiding quiescence and hyper-excitation at the level of individual
neurons.
− Current evidence suggests that neurons detect changes in their
own firing rates through a set of calcium-dependent sensors
− Review paper:
Gina G. Turrigiano. The Self-Tuning Neuron: Synaptic Scaling
of Excitatory Synapses. Cell 135: 422-435, October 31, 2008
A model of multiplicative scaling through the
removal of AMPA receptors
(Turrigiano 1999, TINS)
Homeostasis of intrinsic excitability of neurons
• Activity can also modify the intrinsic excitability and
response properties of neurons
• Models of such intrinsic plasticity show that neurons can be
remarkably robust to external perturbations if they adjust their
conductances to maintain specified functional characteristics
• Intrinsic and synaptic plasticity can interact in interesting
ways. For example, shifts in intrinsic excitability can
compensate for changes in the level of input to a neuron caused
by synaptic plasticity.
Homeostasis of intrinsic excitability of neurons
Theoretical and experimental work suggests that intracellular Ca2+
concentration might regulate the balance of inward and outward currents
generated by a neuron
(Turrigiano 1999, TINS)
Anti-Hebbian plasticity
• It causes synapses to decrease (rather than increase) in strength
when there is simultaneous pre- and postsynaptic activity.
• It is believed to be the predominant form of plasticity at
synapses in mormyrid electric fish and those from parallel fibers
to Purkinje cells in the cerebellum
• Anti-Hebbian modification tends to make weights decrease
without bound
dwi (t )
w
 axi (t ) y (t )
dt
(a  0 )
I. Introduction
II. Synaptic placticity rules
− The basic Hebb rule
− The covariance rule
− BCM Rule
− Non-Hebbian rules
− Anti-hebbian rules
− Timing-based Rules
Timing-Based Rules
LTP is induced by repetitive
stimulation with positively
correlated spike times of post and
pre-synaptic neuron
LDP is induced by repetitive
stimulation with negatively
correlated spike times of post and
pre-synaptic neuron
Spike Timing Dependent Plasticity (STDP)
An intracellular
recording of a pair of
cortical pyramidal
cells in a slice
experiment
(Markram et al., 1997)
－LTP and LTD of
retinotectal synapses
recorded in vivo in
Xenopus tadpoles
(Zhang et al., 1998)
• Simulating the spike-timing dependence of synaptic plasticity
requires a spiking model (e.g. Integrate-and-Fire Models).
However, an approximate model can be constructed on the
basis of firing rates

dwi
w
  d [ H ( ) y (t ) xi (t   )  H ( ) y (t   ) xi (t )]
0
dt
where
LTP
LTD
H ( )   H ( )
• Note above equation is based on a Hebbian rule
• The STDP rule describes an asymmetric learning rule
• About H(τ): a function like the solid line in previous figure.
 A e
, if t  0
H (t )    t / 
 A e
, if t  0.
  t /  
Sequence learning based on STDP
• The Timing-Based plasticity rule is applied throughout a training
period during which the stimulus being presented moves to the right
and excites the different neurons in the network sequentially
• After the training period, the neuron with sa = 0 receives strengthened
input from the sa =−2 neuron and weakened input from the neuron with
sa = 2
• If the same time-dependent stimulus is presented again after
training, the neuron with sa = 0 will respond earlier than it did
prior to training
• The training experience causes neurons to learn a time
sequence
Another example on time sequence learning in place
fields
• Place field is negatively skewed after experience
(Mehta et al. 1997; 2000)
A variety in plasticity
• Different cortical regions, such as
hippocampus and visual cortex have
somewhat different forms of synaptic
plasticity.
(Abbott and Nelson 2000, Nature)
A few properties of LTP and LTD
Long-term plastic changes can be induced in about 1 s or less
(i.e. within a rather short period, similar to short-term
plasticity)
The induced change in synaptic weight typically lasts for
hours (if no further changes are induced)
The longest-lasting forms appear to require protein synthesis
Three types of training procedures
• Unsupervised (or sometimes self-supervised) learning.
－ A network responds to a series of inputs during training
solely on the basis of its intrinsic connections and
dynamics
• Supervised learning
－A desired set of input-output relationships is imposed on
the network by a ‘teacher’ during training.
－ Networks that perform particular tasks can be
constructed in this way
• Reinforcement learning
－It is somewhat intermediate between these cases.
－The network output is not constrained by a
teacher, but evaluative feedback on network
performance is provided in the form of reward or
punishment
Homework
1. 与基本 Hebb 学习律比较, BCM 学习律在哪些方面做了
改进? 意义何在?
2. 举例说明稳态可塑性(Homeostatic plasticity)。
3. 如果要实现神经网络的时间序列学习，需要采用那种学
习律？为什么？