Lecture 13 - UCF Computer Science

Download Report

Transcript Lecture 13 - UCF Computer Science

Neuroevolution and
Artificial Embryogeny
Real-time NEAT
Dr. Kenneth Stanley
February 22, 2006
Generations May Not Always Be
• When a population is evaluated simultaneously
– Many are observable at the same time
– Therefore, entire population would change at
– A sudden change is incongruous, highly
• When a human interacts with one
individual at a time
– Want things to improve constantly
Steady State GA: One Individual Is
Replaced at a Time
• Start by evaluating entire first generation
• Then continually pick one to remove, replace it
with child of the best
Evaluate All
2) Create
offpsring from
good parents
3) Replace
1) Remove poor individual
Steady State During Simultaneous
Evaluation: Similar but not Identical
• Several new issues when evolution is real-time
Evaluation is asynchronous
When to replace?
How to assign fitness?
How to display changes
Regular NEAT Introduces Additional
Challenges for Real Time
• Speciation equations based on generations
• No “remove worst” operation defined in
• Dynamic compatibility thresholding assumes
Speciation Equations Based on
How to Remove the Worst?
• No such operation in generational NEAT
• Worst often may often be a new species
– Removing it would destroy protection of
– Loss of regular NEAT dynamics
Dynamic Compatibility
Thresholding Assumes A Next
Real-time NEAT Addresses Both the
Steady State and Simultaneity Issues
• Real-time speciation
• Simultaneous and asynchronous
• Steady state replacement
• Fast enough to change while a game is
• Equivalent dynamics to regular NEAT
Main Loop (Non-Generational)
Choosing the Parent Species
Finally: How Many Ticks Between
– The more often replacement occurs, the fewer are eligible
– The larger the population, the more are eligible
– The high the age of maturity, the fewer are eligible
rtNEAT Is Implemented In NERO
• Download at http://nerogame.org
• rtNEAT source soon available (TBA)
• Simulated demos have public appeal
– Over 50,000 downloads
– Appeared on Slashdot
– Best Paper Award in Computational Intelligence and
– Independent Games Festival Best Student Game
– rtNEAT licensed
– Worldwide media coverage
NERO: NeuroEvolving Robotic
• NPCs improve in real time as game is played
• Player can train AI for goal and style of play
• Each AI Unit Has Unique NN
NERO Battle Mode
• After training, evolved behaviors are saved
• Player assembles team of trained agents
• Team is tested in battle against opponent’s
NERO Training: The Factory
• Reduces noise during evaluation
– All evaluations start out similarly
• Robot bodies produced by “factory”
• Each body sent back to factory to respawn
• Bodies retain their NN unless chosen for
• NN’s have different ages
– Fitness is diminishing average of spawn trials:
NERO Inputs and Outputs
Enemy/Friend Radars
Enemy On-Target Sensor
Object Rangefinder Sensors
Enemy Line-of-Fire Sensors
Further Applications?
New kinds of games
New kinds of AI in games
New kinds of real-time simulations
Training applications
Interactive steady-state evolution
Next Topic:
Improving the neural model
• Adaptive neural networks
• Change over a lifetime
• Leaky integrator neurons and CTRNN
Evolutionary Robots with On-line Self-Organization and Behavioral Fitness by Dario
Floreano and Joseba Urzelai (2000)
Evolving Adaptive Neural Networks with and Without Adaptive Synapses by Kenneth O.
Stanley, Bobby D. Bryant, and Risto Miikkulainen (2003)
Homework due 2/27/06: Working genotype to phenotype mapping.
Genetic representation completed. Saving and loading of genome file
I/O functions completed. Turn in summary, code, and examples
demonstrating that it works.
Project Milestones (25% of grade)
2/6: Initial proposal and project description
2/15: Domain and phenotype code and examples
2/27: Genes and Genotype to Phenotype mapping
3/8: Genetic operators all working
3/27: Population level and main loop working
4/10: Final project and presentation due (75% of grade)