Transcript Approaches to Artificial Evolution

Approaches to Artificial
Evolution
Verena Hamburger
Problems in engineering
• Simple designs are often not effective
• Highly sophisticated morphologies are
hard to model
• Each motor action has various effects
• Much effort to make sensors highly
precise measuring devices
• A lot of a priori and designer knowledge
is needed
Benefits of evolutionary
robotics (1)
• Overcoming the stiff human concepts
=> Truly new designs & innovative forms of
sensory-motor-coordination
• Sensory-motor-systems acts as a whole
in close coupling with the environment
=> An essential aspect of real cognition
Benefits of evolutionary
robotics (2)
• Reduction of modelling process
=> Emphasis on behaviour analysis
• More, but simpler sensors
=> Non-linearities are exploited or
amended
=> Sensors are combine to extract useful
information
Open-ended evolution
An evolutionary process that leads to the
ongoing development of new traits
that tend
– to be retained for long evolutionary
periods and
– to constitute important building blocks
for further evolutionary stages.
Requirements to open-ended
evolution (1)
Requirements according to Bianco and Nolfi:
●
●
●
implicit and general selection criteria, but BII
fitness functions may cause a “bootstrap problem”.
favourable organisation of the evolving individuals,
a good genotype-to-phenotype mapping is
expressive, compact, autonomous & complete
dynamically changing environmental conditions
Requirements to open-ended
evolution (2)
Requirements according to Maley:
●
Endogenous implementation of ecological niches
●
Unbounded diversity during growth phase
●
Selection must be embodied
●
System must exhibit continuing (“positive”) new
adaptive activity
Requirements to open-ended
evolution (3)
Requirements according to Channon:
●
●
●
Evolutionary emergence: The constant need to
change one's model to keep up with a system’s
behaviour.
Natural selection: evolution as a result of the
system dynamics is a prerequisite for
evolutionary emergence (also Packard, Ray).
Non-linear systems which do not obey the
superposition principle (also Langton).
Stages of natural evolution
1. The creature's body plan changes
2. New sensors and actuators are explored
3. The environment changes (at least as
the organisms perceives it)
4. The nervous system adapts.
State of the Art
Evolution of complete agents are still quite
restricted to:
– single tasks
– few basic shapes
– limited variety of sensors, actuators
and materials
Some methods require a lot of human
interference.
Modular evolution
●
A simple body plan with two legs
●
Robot drags itself along the ground
●
Development of multi-jointed legs
●
.....
●
Quadruped with stable locomotion
=> Stages are induced by the designer
(Muthuraman, MacLeod and Maxwell)
Evolving Neural Networks
•
•
•
•
•
•
•
•
Support of various learning techniques
Resistant to noise
Biologically inspired
Real values for in- and output
Graceful degradation
Low level
Evtl. flexible neuron model
Recurrent NNs (internal state, rich intrinsic
dynamics)
Evolution of learning
TPE learning in PNN (Hebb, post-/presynaptic, covariance)
• More complex skills (also sequential tasks)
• Fewer generations
• Adapting to environments never seen during
evolution (even different morphology)
• Controllers transferred to reality quickly
(Floreano & Urzelai)
LAE by (Punctuated)
Anytime learning
Algorithm receiving information from the real
world to adapt off-board simulator or GA. Best
solution is periodically sent to controller.
Parker and others: Punctuated anytime learning
with “cyclic GA” for hexapod gaits:
•Incrementally evolving individual leg controllers
•Adapting to a different environments
•Evolving a simulated team of legged robots.
Importance of morphology
• Locomotion for ten legged agents with different
body plans (Bongard and Pfeifer)
=> Shape and mass determine fast/slow/no
success
=> Build up parameter directory
• Closed loop controller for stable biped walking
(Paul and Bongard)
=> The more control over the weight
distribution, the more stable gaits
Importance of environment
Fast locomotion for Sony Aibo:
• TPE on flat carpet did not generalise well to
new surfaces
• TPE on uneven surface generalised well
but: it was tricky to get the level of
unevenness right
(Hornby, Fujita, Takamura, Yamamoto, Hanagata)
Evolution with meta-model
•
Partial order via classification of genotype
• Classifier is evolved online
• Additional regular fitness based evaluation
=> Reduction of evaluation (~50%)
=> Less deterioration of the robot
=> False classification may lead to loss of
best individual
(Jens Ziegler: GP for fast locomotion of Sony Aibo)
Variation of evolutionary
parameters (1)
The performance of GA is sensitive to
initial conditions
but: no quantitative methods are available
=> lower risk through parameter variation
Variation of evolutionary
parameters (2)
• Classification, regression, robot control
• Comparing fitness expectation and variance after
200 generations
• Wilcoxon-Test (confidence niveau 0.95)
=>
deterministic or adaptive variation
with
mutation rate ↑ - crossover rate ↓
(Jens Ziegler)
Architectures of variable
complexity (1)
• Fixed architectures are unsuitable since
complexity cannot be foreseen (fixation =
limitation)
• Open-ended evolution gets interesting when
problem-oriented evolution reaches equilibrium
• At equilibrium all individuals are similar
=> only mutation can bring forth new traits
Architectures of variable
complexity (2)
• Mutation rate tied to whole genotype or single
gene is both inappropriate
=> Mutation-lock of beneficial genes
• Mutation-locked genes evtl. still suboptimal
=> Mutation-lock immunises against major, but
not against minor mutation
My personal idea
We need a methodology for
automatic modular (co-)evolution
of learning in
architectures of variable complexity
– Mutation-lock
– Parameter variation in individual stages
– Evtl. meta-model for real world experiment
in dynamically changing environment