Transcript ML in Games

Exploring Machine Learning in Computer Games
Presented by: Matthew Hayden
Thurs, 25th March 2010
Introduction - Games


Games are important

…for fun

…for learning (coincidentally?)
What are games?

…(arguably) simulations of reality


…don't have all the repercussions (i.e. Simulation)



There are rules and restrictions
Cheap to play (good for us)
Most importantly, fun!
Computer games are big business
Introduction – Games (2)

Computers play games (against humans)

Board games



…'brute force' calculation of the cost of a move
…often 'zero-sum' → minimax algorithm
…estimate (sometimes solve) the next optimal
move
'Noughts and Crosses' or 'Tic Tac Toe' is an example of a popular 'zero-sum' game that has been solved.
Classical Artificial Intelligence


More complex/realistic simulations require
heuristics for intelligence, including...

Rule based systems (easy to exploit)

Path finding algorithms (cannot adapt)

Finite state automata (imperative → fast, but stupid)

Planning (more robust, declarative, but slow)
Could completely remove all need

Scripted sequences

...but we want to make the game more interesting!
Classical Artificial Intelligence (2)
ML in Games – Path finding

Path finding in dynamic landscapes

Changes constantly → adaptive algorithms


Ant Colony Optimisation algorithm
Finding optima quickly

Evolving optimal racing lines (rally games)




Genotype → Decision trees
Phenotype → Ability driving round a track
Evaluation → Make n decisions a second
Selection → Faster more likely to be selected
ML in Games - Adaptation



Games should adapt to be personalised

Different players have their own styles of play

Players get better, need smooth difficulty curve
…we could fake it

More complex AI to anticipate more behaviours

Ramp up the difficulty more gradually
…or we could use Machine Learning

On-line training algorithms

No fixed difficulty curve
ML in Games – Adaptation (2)

Case study: Left 4 Dead – Dynamic AI Director

Player-opponent interaction as well as context

No fixed narrative, generated procedurally



Collects data from user performance/physiology
and construct a player model
Procedural → play lots of of games offline for
validation
Cleverly adapted the current models into a
Machine Learning context

Zombies! → Many data points