Transcript Lecture 16
CAP6938 Neuroevolution and Developmental Encoding Leaky Integrator Neurons and CTRNNs Dr. Kenneth Stanley October 25, 2006 Artificial Neurons are a Model n • Standard activation model H j xi wij i 1 • But a real neuron doesn’t have an activation level – Real neurons fire in spike trains Wolfgang Maass, http://www.tu-graz.ac.at/igi/maass – Spikes/second is a rate – Therefore, standard activation can be thought of as outputting a firing rate at discrete timesteps (i.e. rate encoding) What is Lost in Rate Encoding? • Timing information • Synchronization • Activity between discrete timesteps 30 Neurons Firing in a monkey’s striate cortex From Krüger and Aiple [Krüger and Aiple, 1988]. Reprinted from www.igi.tugraz.at/ maass/123/node2.html Spikes Can Be Encoded Explicitly • • • • • Leaky integrate and fire neurons Encode each individual spike Time is represented exactly Each spike has an associated time The timing of recent incoming spikes determines whether a neuron will fire • Computationally expensive • Can we do almost as well without encoding every single spike? Yes: Leaky Integrator Neurons (CTRNNS; Continuous Time Recurrent Neural Networks) • Idea: Calculate activation at discrete steps but describe rate of change on a continuous scale • Instead of activating only based on input, include a temporal component of activation that controls the rate at which activation goes up or down • Then the neuron can react to changes in a temporal manner, like spikes Activation Rate Builds and Decays Input to neuron Activation Level (i.e. spike rate) Output over time time • Incoming activation causes the output level to climb over time • We can sample the rate at any discrete granularity desired • A view is created of temporal dynamics without full spike-event simulation What is Leaking In a Leaky Integrator? Leaking activation level (membrane potential) Activation Level (i.e. spike rate) time • The neuron loses potential at a defined rate • Each neuron leaks at its own constant rate • Each neuron integrates at the same constant rate as well Leaky Integrator Equations Leak • Expressing rate of change of activation level: • Apply Euler Integration to derive discretetime equivalent • Expressing current activation in terms of activation on previous discrete timestep: Real time Between steps Equations from: Blynel, J., and Floreano, D. (2002). Levels of dynamics neural controllers. In Proceedings of the Seventh International Behavior on From Animals to Animats, 272–281. What Can a CTRNN Do? • With the right time constants for each neuron, complex temporal patterns can be generated • That is, the time constants are a new parameter (inside nodes) that can evolve • More powerful than a regular RNN • Capable of generating complex temporal patterns with no input and no clock Pattern Generation for What? • Walking gaits with no input! Evolution of central pattern generators for bipedal walking in a real-time physics environment T Reil, P Husbands - Evolutionary Computation, IEEE Transactions on, 2002 Reil and Husbands Went on to Found the Company NaturalMotion Pattern Generation for What? • Salamander walking gait Ijspeert A.J.: A connectionist central pattern generator for the aquatic and terrestrial gaits of a simulated salamander, Biological Cybernetics, Vol. 84:5, 2001, pp 331-348. • Wing flapping Evolution of neuro-controllers for flapping-wing animats - group of 2 » JB Mouret, S Doncieux, L Muratet, T Druot, JA … - Proceedings of the Journees MicroDrones, Toulouse, 2004 Maybe Good for Other Things with Temporal Patterning • Music? • Tasks that we typically do not conceive in terms of patterns? • Learning tasks (better than a simple RNN?; Blynel and Floreano 2002 paper) • Largely unexplored • How far away from the benefits of a true spiking model? Leaky NEAT • There is a rough, largely untested leakyNEAT at the NEAT Users Group files section: – http://groups.yahoo.com/group/neat/files/ – Introduces a new activation function and new time constant parameter in the nodes • A new leaky-rtNEAT will soon be available too • The topology of most CTRNNs in the past was determined completely by the researcher Next Topic: Non-neural NEAT, Closing Remarks on Survey Portion of Class • Complexification and protection of innovation in non-neural structures • Example: Cellular Automata neighborhood functions • What have we learned, what is its significance, and where does the field stand? Reading: Mitchell Textbook pp. 44-55 (Evolving Cellular Automata) think about: How would NEAT apply to this task?