Transcript learn
AI23 – 2004/05 – demo 2 Learning School of Computing, University of Leeds, UK part 1: what is learning? • what would you say learning is? Part 1 : what is learning? • meaning of learning is subject to discussion • recap some ideas • high-level: “experience alters behaviour” • low-level: “weights (on neuron connections) change” example 1: Yamauchi/Beer’s alternate worlds • one agent, one goal, one landmark • two kinds of world: landmark-far/near a/b: landmark opposite to goal c/d: landmark between agent and goal • agent’s task: reach goal (how? what if it knows the type of world it is in?) example 1 [cont.] • so, if world is known, a fixed strategy can be applied • now, suppose a coin is tossed every 10 trials, and the kind of world is changed accordingly • how can the problem be solved? The agent must learn to detect the kind of world it is in • Yamauchi/Beer’s solution • separately obtained (through artificial evolution) 3 distinct networks that solve subtasks: world detection, LF and LN goal-finding • integrated networks: agent uses first trial in the 10-trial sequence to “learn” what world he is in; with that knowledge, he then switches to the right strategy for that world, for the next 9 trials. On average, 95% success example 1 [cont.] • is that learning??? • can be seen as “experience altering behaviour”? • no “weights changing”; rather, internal state of the agent is changed (by setting a world-type flag) – does it matter? • the network is only learning one thing (the world the agent is in); can that still be called learning? example 2: c.elegans 1-mm worm example 2: c.elegans • no evidence of synaptic plasticity in c.elegans, i.e. no mechanisms for changing the weights between neurons • however, c.elegans exhibits various kinds of learning capabilities (behavioural plasticity) • habituation / sensitisation, associative learning • this would mean that changing weights on neuron connections is not the only way in which learning occurs in nature • lots to discover and understand yet! Part 2: different forms of learning • activity: recall different forms of learning forms of learning • neural networks • Gradient-descent algorithms for the McCulloch and Pitts neuron and for Feed-Forward Neural networks • delta rule • backpropagation • feed-forward nets used in some demos in BEAST forms of learning • reinforcement learning • agent interacts with environment and receives • rewards (positive reinforcement) • punishments (negative reinforcement) • different to delta rule / backprop • the agent is not given the correct answer, but only a good/bad signal; “quantitative v. qualitative” • only desired results are needed to specify the problem, rather than intermediate actions; think of riding a bike, mazes, tic-tac-toe, backgammon [see demo, pendulum] forms of learning • conditioning • Pavlov’s experiments • repeated pairing of two stimuli so that a previously neutral (conditioned) stimulus eventually elicits a response (conditioned response) similar to that originally elicited by nonneutral (unconditioned) stimulus • notion of reward for artificial purposes forms of learning • Hebbian Learning • form of learning in natural and artificial neural networks • potentiation of effective synaptic connections and decay / depression of ineffective ones • concept of simultaneous / concurrent / correlated activation forms of learning • winner takes all • a form of competitive learning in natural and artificial neural networks • neurons compete on activation over an input • winner neuron gets reinforced • Hebbian-like rule • will be seen in this module forms of learning • evolutionary algorithms • search algorithms inspired by natural evolution: population evolves, improving its “fitness” • concepts of assessment (of an individual), selection, variation (of population’s individuals over time) • can be used as optimisation tools, even to "train" neural networks • Yamauchi/Beer • also the way we use them in BEAST • will be seen in this module forms of learning • imitation • a form of learning in nature and (recently) in robotics • individuals learn by replication and repetition of behaviour observed in others [see demo, CogVis] work by CogVis lab @ SOC • behaviour is adapted to their particulars [see demo, tennis] forms of learning • mimicry • a form of evolutionary learning: species / groups learn by mimicking desirable genetic traits from other species / groups • “wasp-like” insects work by J.Noble / D. Franks, SOC • social learning • learning is achieved via the communication of information within a social structure • schools, books; birds, mammals learning • activity: where are the above used in nature and in bio-inspired algorithms? thank you!