Transcript Alex Berry`s Case Study Slides

Evolving Agents in a
Hostile Environment
Alex J. Berry
Training First Responders




VEnOM Labs is developing a suite to train First
Responders.
Is the training effective?
How can we make the training more effective?
Environment lacks atonomous agents that can
interact with trainees in the environment.
Goal

Long Term
 To
develop a system to allow for friendly and hostile
AI agents in the training environment.

Short Term
 To
develop a system to evolve agents in a hostile
environment.
Simulation of Adaptive Agents in a Hostile Environment
[HW95]

Thomas Haynes
Used Genetic Programming
Simple Agents
Mines and Energy

Experiments



 Single Agent,
Static and Random Environment
 Multiple Agent, Static and Random Environment
The Approach


Randomly Generated Map Environment
Three Types of Agents:
 First
Responders
 Terrorists
 Victims

Genetic Programming to Evolve the Agents
Maps




Any Dimension
Percentage walls
Bit Array to Hold the
Data
Used for memory
storage in the Agents
What’s an Agent to do?

Victims





Move Randomly
Remember Things
Forget Things
Survive
Terrorists




Kill Victims
Kill First Responders
Lay Traps
Not Get Caught

First Responders




Help Victims
Find and Disarm Traps
Survive
Catch Terrorists
Evolutionary Algorithm

Two Agents to Evolve







First Responder
Terrorist
Mutation and Crossover are the only operators changed.
The individuals consist of expression and decision trees.
Initialization was based on both random and created
individuals.
Rank Based Selection was used.
Elitist Competition was used.
What An Individual Looks Like

Terminals





Current Grid Location (C)
Surrounding Grid Locations
(S)
Rand (R)
Memory (M)
Non-Terminals

If-Then-Else




Threat
And, Or, Not
Victim, First Responder,
Terrorist, Trap
Valid Move

Actions





Save
Kill
Move
Place Trap
Remove Trap
Sample Individual
MOVE
Genetic Programming Evaluation

First Responder
 Victims
Helped
 Terrorists Caught
 Traps Removed
 Survival Time
 Amount of the Map
explored

Terrorist
 Kills
using Traps
 Kills on Contact
 Survival Time
 Deduction for killing
other Terrorists
Experiments




Static Environment Evolution
Random Environment Evolution
Varying Ratios of First Responders, Victims, and
Terrorists
Evolving one Population at a time