Transcript learn

AI23 – 2004/05 – demo 2
School of Computing,
University of Leeds, UK
part 1: what is learning?
• what would you say learning is?
Part 1 : what is learning?
• meaning of learning is subject to
• recap some ideas
• high-level: “experience alters behaviour”
• low-level: “weights (on neuron connections)
example 1: Yamauchi/Beer’s alternate worlds
• one agent, one goal, one landmark
• two kinds of world: landmark-far/near
a/b: landmark opposite to goal
c/d: landmark between agent and goal
• agent’s task: reach goal (how? what if it knows
the type of world it is in?)
example 1 [cont.]
• so, if world is known, a fixed strategy can be applied
• now, suppose a coin is tossed every 10 trials, and the kind of
world is changed accordingly
• how can the problem be solved? The agent must learn to detect
the kind of world it is in
• Yamauchi/Beer’s solution
• separately obtained (through artificial evolution) 3 distinct networks
that solve subtasks: world detection, LF and LN goal-finding
• integrated networks: agent uses first trial in the 10-trial sequence to
“learn” what world he is in; with that knowledge, he then switches to
the right strategy for that world, for the next 9 trials. On average,
95% success
example 1 [cont.]
• is that learning???
• can be seen as “experience altering behaviour”?
• no “weights changing”; rather, internal state of the
agent is changed (by setting a world-type flag) – does
it matter?
• the network is only learning one thing (the world the
agent is in); can that still be called learning?
example 2: c.elegans
1-mm worm
example 2: c.elegans
• no evidence of synaptic plasticity in c.elegans,
i.e. no mechanisms for changing the weights
between neurons
• however, c.elegans exhibits various kinds of
learning capabilities (behavioural plasticity)
• habituation / sensitisation, associative learning
• this would mean that changing weights on
neuron connections is not the only way in
which learning occurs in nature
• lots to discover and understand yet!
Part 2: different forms of learning
• activity: recall different forms of learning
forms of learning
• neural networks
• Gradient-descent algorithms for the McCulloch
and Pitts neuron and for Feed-Forward Neural
• delta rule
• backpropagation
• feed-forward nets used in some demos in
forms of learning
• reinforcement learning
• agent interacts with environment and receives
• rewards (positive reinforcement)
• punishments (negative reinforcement)
• different to delta rule / backprop
• the agent is not given the correct answer, but only a
good/bad signal; “quantitative v. qualitative”
• only desired results are needed to specify the problem,
rather than intermediate actions; think of riding a bike,
mazes, tic-tac-toe, backgammon
[see demo, pendulum]
forms of learning
• conditioning
• Pavlov’s experiments
• repeated pairing of two stimuli so that a
previously neutral (conditioned) stimulus
eventually elicits a response (conditioned
response) similar to that originally elicited by
nonneutral (unconditioned) stimulus
• notion of reward for artificial purposes
forms of learning
• Hebbian Learning
• form of learning in natural and artificial neural
• potentiation of effective synaptic connections
and decay / depression of ineffective ones
• concept of simultaneous / concurrent /
correlated activation
forms of learning
• winner takes all
• a form of competitive learning in natural and
artificial neural networks
• neurons compete on activation over an input
• winner neuron gets reinforced
• Hebbian-like rule
• will be seen in this module
forms of learning
• evolutionary algorithms
• search algorithms inspired by natural evolution:
population evolves, improving its “fitness”
• concepts of assessment (of an individual), selection,
variation (of population’s individuals over time)
• can be used as optimisation tools, even to "train"
neural networks
• Yamauchi/Beer
• also the way we use them in BEAST
• will be seen in this module
forms of learning
• imitation
• a form of learning in nature and (recently) in
• individuals learn by replication and repetition
of behaviour observed in others
[see demo, CogVis] work by CogVis lab @ SOC
• behaviour is adapted to their particulars
[see demo, tennis]
forms of learning
• mimicry
• a form of evolutionary learning: species / groups learn
by mimicking desirable genetic traits from other
species / groups
• “wasp-like” insects
work by J.Noble / D. Franks, SOC
• social learning
• learning is achieved via the communication of
information within a social structure
• schools, books; birds, mammals
• activity: where are the above used in
nature and in bio-inspired algorithms?
thank you!