Transcript Document
Learning and Stability Learning and Memory Ramón y Cajal, 19th century Learning and Memory Ramón y Cajal, 19th century Learning and Memory axon neuron dendrite synapse Ramón y Cajal, 19th century Learning and Memory axon dendrite Neuron doctrine von Waldeyer-Hartz (1891) neuron synapse Ramón y Cajal, 19th century Input Environmental Learning and Memory active neuron neuron synapse James, 1890; Hebb, 1949; Martin et al., 2000; Martin and Morris, 2002; … i Input Environmental Learning and Memory wij j Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases. Thus, the influence of neuron j on the firing of neuron i is enhanced. neuron synapse James, 1890; Hebb, 1949; Martin et al., 2000; Martin and Morris, 2002; … Recall Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases. Cell Assembly: A later presented (partial) recall signal of the original learned input has to activate the same group of neurons. A group of strongly connected neurons which tend to fire together can represent a memory item. James, 1890; Hebb, 1949; Martin et al., 2000; Martin and Morris, 2002; … Learning and Memory Synaptic Plasticity: If the neurons i and j are active together the synaptic efficiency wij increases. Cell Assembly: Synaptic Plasticity and Memory A group of strongly connected neurons which Hypothesis tend to fire together can represent a memory item. Synaptic Plasticity ≈ Learning James, 1890; Hebb, 1949; Martin et al., 2000; Martin and Morris, 2002; … Synaptic Plasticity: Hebbian Neuron A Neuron B Synapse u Classical Hebbian plasticity: ω ω=μ u v v Long-Term Potentiation (LTP) u: input v: output ω: weight μ : learning rate What are the long-term dynamics of Hebbian Plasticity? Learning and Stability Synaptic Plasticity: Hebbian Neuron A Neuron B Long-Term Potentiation (LTP) Synapse u linear neuron model: Classical Hebbian plasticity: ω v v=u ω u: input v: output ω: weight μ : learning rate ω=μ u v stability analysis Gradient of ω positive The weights follow divergent dynamics 0 negative Mechanisms of stabilization Sliding threshold: Subtractive normalization: Multiplicative normalization: dw = m vu (v - Q) dt m << 1 dQ = n (v2 - Q) dt n<1 1 dw ö dt = vu à v( n áu ) n N dw = m (vu – a v2w), a>0 dt Sliding Threshold BCM- Rule dw dt = m vu (v - Q) m << 1 As such this rule is again unstable, but BCM introduces a sliding threshold dQ dt = n (v2 - Q) n<1 Note the rate of threshold change n should be faster than then weight changes (m), but slower than the presentation of the individual input patterns. This way the weight growth will be over-dampened relative to the (weight – induced) activity increase. less input leads to shift of threshold to enable more LTP Time scales between learning rule and experiment are different Kirkwood et al., 1996 open: control condition filled: light-deprived Mechanisms of stabilization Sliding threshold: Subtractive normalization: Multiplicative normalization: dw = m vu (v - Q) dt m << 1 dQ = n (v2 - Q) dt n<1 1 dw ö dt = vu à v( n áu ) n N dw = m (vu – a v2w), a>0 dt biological unrealistic Subtractive normalization Subtractive: 1 dw ö dt = vu à v( n áu ) n N v <u> With N, number of inputs and n a unit vector (all “1”). This yields that n.u is just the sum over all inputs. This normalization is rigidly apply at each learning step. It requires global information (info about ALL inputs), which is biologically unrealistic. Mechanisms of stabilization Sliding threshold: Subtractive normalization: Multiplicative normalization: dw = m vu (v - Q) dt m << 1 dQ = n (v2 - Q) dt n<1 1 dw ö dt = vu à v( n áu ) n N dw = m (vu – a v2w), a>0 dt biological unrealistic biological unrealistic Multiplicative normalization: Multiplicative: dw dt = m (vu – a v2w), a>0 (Oja’s rule, 1982) This normalization leads to an asymptotic convergence of |w|2 to 1/a. It requires only local information (pre-, post-syn. activity and the local synaptic weight). Mechanisms of stabilization Sliding threshold: Subtractive normalization: Multiplicative normalization: dw = m vu (v - Q) dt m << 1 dQ = n (v2 - Q) dt n<1 1 dw ö dt = vu à v( n áu ) n N dw = m (vu – a v2w), a>0 dt biological unrealistic biological unrealistic no biological evidence Synaptic Scaling (Turrigiano et al., 1998) could be a candidate to stabilize plasticity Synaptic Scaling EPSPs block raise Activity changes compared to control Number of AMPA receptors change Turrigiano and Co-workers, 1998, 2004, 2008, … Synaptic Scaling EPSPs Turrigiano and Co-workers, 1998, 2004, 2008, … firing rate [v] Synaptic Scaling weight [ω] Turrigiano and Co-workers, 1998, 2004, 2008, … Synaptic Scaling firing rate [v] ω = γ [vT – v] weight [ω] Synaptic scaling guarantees stable weight dynamics v: output ω: weight vT: target firing rate γ : learning rate Hebbian Plasticity and Scaling Hebbian Plasticity Hebbian Plasticity + Synaptic scaling ω=μ u v ω = μ u v + γ [vT – v] ω2 unstable stable Synaptic Plasticity: STDP LTP Neuron A Synapse u ω Neuron B v LTD Makram et al., 1997 Bi and Poo, 2001 Synaptic Plasticity: Diversity Kampa et al., 2007 Synaptic plasticity together with synaptic scaling is globally stable regardless of learning rule and neuron model. Synaptic scaling is an appropriate candidate to guarantee stability. Another learning mechanism Structural Plasticity Structural plasticity of axon terminals In-vivo 2 Photon-Laser Imaging from the cortex of living mice reveals a permanent axonal remodelling even in the adult brain leading to synaptic rewiring Top: Axonal outgrowth/retraction Red arrow: Axonal outgrowth Yellow: Remains Blue: Retracts Bottom: Rewiring Blue: Retracts and looses synapse Red: Grows and creates new syn. Structural plasticity of dendritic spines Dendritic spines 2μm Dendritic spines on an appical dendrite on a LIV-neuron in V1 of the rat Spine growth precedes synapse formation Arellano et al. (2007) Neurosci. Spine formation via filopodia-shaped spines (see arrow, top figure) precedes synapse formation. Spines in synapses are rather mushroom-shaped and carry receptor plates (active zones, red, top figure). Spines contact axonal terminals or axonal varicosities in reach and form synapses (left). Knott et al., 2006 Structural plasticity of dendritic spines Stable and transient spines In-vivo imaging of dendritic trees within the barrel cortex of living rats Trachtenberg et al. (2002) Nature Spines are highly flexible structures that are responsible (together with axonal varicosities) for synaptic rewiring. Only one third of all spines are stable for more than a month. Another third is semistable, meaning that it is present for a couple of days. Transient spines appear and disappear within a day. Structural Plasticity: More abstract Structural plasticity creates and deletes synapses 33 Structural Plasticity and Learning before training after camera poor mice structural changes Xu et al., 2009; Yang et al., 2009; Ziv and Ahissar 2009 Structural Plasticity and Learning new spine formation old spine elimination training duration new training task Xu et al., 2009; Yang et al., 2009; Ziv and Ahissar 2009 Structural plasticity monocular deprivation Dendrite of an adult mouse in the visual cortex Hofer et al., 2009 low activity Dendritic Outgrowth high activity Dendritic Regression Kater et al., 1989; Mattson and Kater, 1989 Activity shapes neuronal form and connectivity Neuronal activity Neuronal morphology Morphology Connectivity Network connectivity . Neuronal activity changes the intracellular calcium. Via changes in intra-cellular calcium, neurons change their morphology with respect to their axonal and dendritic shape. This leads to changes in neuronal connectivity which, in turn, adapts neuronal activity. The goal is that by these changes neurons achieve a homeostatic equilibrium of their activity. Circular axonal and dendritic probability spaces overlap ≈ # synapses Overlap determines the synaptic connectivity. Individual shrinkage and growth following a homeostatic principle. # dendrites: # axons: Calcium dependent dendritic and axonal growth or shrinkage for synapse generation or deletion The model (without the equations) Normal Input Less Input Less Synapses at THIS neuron Homeostatic Process High Calcium Input Axonal Growth & Dendr. Shrinkage Memory strong, interconnected cluster representing memory Input time neuron Plasticity+Scaling synapse Structural Plasticity Memory Relations between biological processes and learning/memory Short-term memory msec Physiology Long-term memory Working memory sec min Activity Short-term plasticity hrs days years Hebbian plasticity Synaptic Scaling Structural plasticity Tetzlaff et al. (2012). Biol. Cybern.