Transcript Lecture 9

CAP6938
Neuroevolution and
Developmental Encoding
NeuroEvolution of Augmenting
Topologies
(NEAT)
Dr. Kenneth Stanley
September 25, 2006
TWEANN Problems Reminder
• Competing conventions problem
– Topology matching problem
• Initial population topology randomization
– Defective starter genomes
– Unnecessarily high-dimensional search space
• Loss of innovative structures
– More complex can’t compete in the short run
– Need to protect innovation
• NEAT directly addresses these challenges
Solutions: NEAT
• Historical markings match up different structures
• Speciation
– Keeps incompatible networks apart
– Protects innovation
• Incremental growth from minimal structure, i.e.
complexification
– Avoids searching in unnecessarily high-d
space
– Makes finding high-d solutions possible
Genetic Encoding in NEAT
Topological Innovation
Link Weight Mutation
• A random number is added or subtracted from
the current weight/parameter
• The number can be chosen from uniform,
Gaussian (normal) or other distributions
• Continuous parameters work best if capped
• The probability of mutating a particular gene
may be low or high, and is separate from the
magnitude added
• Probabilities and mutation magnitudes have a
significant effect
Link Weight Mutation in NEAT C++
randnum=randposneg()*randfloat()*power;
if (mut_type==GAUSSIAN) {
randchoice=randfloat();
if (randchoice>gausspoint)
((*curgene)->lnk)->weight+=randnum;
else if (randchoice>coldgausspoint)
((*curgene)->lnk)->weight=randnum;
}
else if (mut_type==COLDGAUSSIAN)
((*curgene)->lnk)->weight=randnum;
//Cap the weights at 3.0
if (((*curgene)->lnk)->weight > 3.0) ((*curgene)->lnk)->weight = 3.0;
else if (((*curgene)->lnk)->weight < -3.0) ((*curgene)->lnk)->weight = -3.0;
Topology Matching Problem
• Problem arises from adding new genes
• Same gene may be in different positions
• Different genes may be in same positions
Biological Motivation
• New genes appeared over biological evolution as well
• Nature has a solution to still know which is which
– Process of aligning and matching genes is called
synapsis
– Uses homology to align genes:
“. . .Crossing over thus generates homologous
recombination; that is, it occurs between 2 regions of
DNA containing identical or nearly identical
sequences.” (Watson et al. 1987)
Artificial Synapsys: Tracking Genes
through Historical Markings
The numbers tell exactly when in history particular topological features
appeared, so now they can be matched up any time in the future. In
other words, they reveal gene homology.
Matching up Genes
Second Component:
Speciation Protects Innovation
• Originally used for multimodal function
optimization (Mahfood 1995)
• Organisms grouped by similarity (compatibility)
• Fitness sharing (Goldberg 1987, Spears 1995):
Organisms in a species share the reward of their
fitness peak
• To facilitate this, NEAT needs
– A compatibility measure
– Clustering based on compatibility, for fitness sharing
Measuring Compatibility
• Possible in NEAT through historical markings
• 3 factors affect compatibility via historical
markings on connection genes:
– Excess
– Disjoint
– Average Weight Distance W
c1 E c2 D

 c3W
• Compatibility distance  
N
N
Clustering Into Species
Dynamic Compatibility
Thresholding
Fitness Sharing: Assigning
Offspring to Species
Third Component: Complexification
from Minimal Structure
•
•
•
•
•
•
Addresses initialization problem
Search begins in minimal-topology space
Lower-dimensional structures easily optimized
Useful innovations eventually survive
So search transitions into good part of higher-dim. space
The ticket to high-dimensional space
NEAT Performed Well on Double
Pole Balancing Without Velocity
Inputs
DPNV Solutions Are Compact
Harder DPNV (0.3m short pole)
solution
Visualizing Speciation
Next Class:
More NEAT
•
•
•
•
Implementation issues
Where NEAT can be changed
Areas for advancement
Issues in applying NEAT (e.g. sensors and
outputs)
Evolving a Roving Eye for Go by Kenneth O. Stanley and Risto Miikkulainen (2004)
Neuroevolution of an Automobile Crash Warning System by Kenneth O. Stanley and
Risto Miikkulainen (2005)