Transcript The game: Build & Build

國立雲林科技大學
National Yunlin University of Science and Technology
Evolving Reactive NPCs for the
Real-Time Simulation Game
Advisor ： Dr. Hsu
Reporter ： Wen-Hsiang Hu
Author ： JinHyuk Hong and Sung-Bae Cho
IEEE Symposium on Computational Intelligence and Games
1
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Outline










Motivation
Objective
Introduction
The game: Build & Build
Basic behavior model
Co-evolutionary behavior generation
Experiment and Results
Discussion
Conclusion
Personal Opinion
2
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Motivation

AI in computer games has been highlighted in recent,
but manual works for designing the AI cost a great
deal.
3
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Objective

Designing NPCs’ behaviors without relying on
human expertise.
4
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Basic behavior model

Two different grid scales are used for the input of
the neural network such as 5×5 and 11×11.
random action probability: 0.2
The game: Build & Build
five neural networks
are used to decide
whether the associating
action executes or not.
5
Intelligent Database Systems Lab
Co-evolutionary behavior generation

We use the genetic algorithm to generate behavior systems that are
accommodated to several environments.
6
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
N.Y.U.S.T.
I. M.
Experiment and Results

5×5 obtains lower winning averages for complex environment, while it
performs better when the environment is rather simple.
7
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Introduction





It is challengeable for many researchers to apply AI to control
characters. (AI produce more complex and realistic games.)
Finite state machines and rule-based systems are the most
popular techniques in designing the movement of characters.
While neural networks, Bayesian network, and artificial life
are recently adopted for flexible behaviors.
Evolution generates useful strategies automatically.
This paper proposes a reactive behavior system composed of
neural networks is presented, and the system is optimized by
co-evolution.
8
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Rule based approach

AI of many computer games is designed with rules
based techniques such as finite state machines (FSMs)
or fuzzy logic.

FSMs have a weak point of its stiffness; however, the
movement of a character is apt to be unrealistic.
─
there is a trend towards fuzzy state machine (FuSM).
9
Intelligent Database Systems Lab
Adaptation and learning: NNs, EAs,
and Artificial life


N.Y.U.S.T.
I. M.
The adaptation and learning in games will be one of
the most major issues making games more interesting
and realistic.
Neural network, and evolutionary algorithms (e.g.
genetic algorithm) are promising artificial intelligence
techniques for learning in computer games.
─
─
NN - is badly trained
GE - required too many computations and were too slow to
produce useful results.
10
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Co-evolution

By simultaneously evolving two or more species with
coupled fitness.

Superior strategies for an environment have been
discovered by co-evolutionary approaches.
11
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Reactive behavior

Reactive model performs effectively since it
considers the current situation only.

Neural networks and behavior-based approaches are
recently used for the reactive behavior of NPCs
keeping the reality of behaviors.
12
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
The game: Build & Build

‘Build & Build’ developed in this research is a realtime strategic simulation game, in which two nations
expand their own territory.

Each nation has soldiers who individually build
towns and fight against the enemies, while a town
continually produces soldiers for a given period.
13
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
The game: Build & Build
14
Intelligent Database Systems Lab
Designing the game environment



N.Y.U.S.T.
I. M.
The game starts two competitive units in a restricted
land with an initial fund.
The units are able to take some actions at the normal
land but not at the rock land.
A unit can build a town when the nation has enough
money, while towns produce units using some money.
15
Intelligent Database Systems Lab
Designing
the game
environment
N.Y.U.S.T.
I. M.
(cont.)
16
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Designing NPCs

NPC can move by 4 directions as well as build towns,
attack units or towns, and merge with other NPCs.

The attack actions are automatically executed when
an opponent locates beside the NPC.
17
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Designing NPCs (cont.)
18
Intelligent Database Systems Lab
Designing NPCs (cont.)
N.Y.U.S.T.
I. M.
19
Intelligent Database Systems Lab
Basic behavior model (cont.)

Two different grid scales are used for the input of the
neural network such as 5×5 and 11×11.
20
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
N.Y.U.S.T.
I. M.
Basic behavior model (cont.)
five neural networks are
used to decide whether
the associating action
executes or not.

In order to actively seek a dynamic situation, the
model selects a random action with a probability (in
this paper, a = 0.2) in advance.
21
Intelligent Database Systems Lab
Co-evolutionary behavior generation


N.Y.U.S.T.
I. M.
We use the genetic algorithm to generate behavior systems that
are accommodated to several environments.
Two pair-wise competition patterns are adopted to effectively
calculate the fitness of an individual.
22
Intelligent Database Systems Lab
Co-evolutionary behavior generation (cont.)

The fitness of an individual is measured by the scores
against randomly selected M opponents.
23
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Experiment and Results

Four different battle
maps => demonstrate
the proposed method
in generating strategies
adaptive to each
environment.
24
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Experiment and Results (cont.)


N.Y.U.S.T.
I. M.
The case with 11×11 shows more diverse behaviors than that
with 5×5, since it observes information on a more large area.
5×5 obtains lower winning averages for complex environment,
while it performs better when the environment is rather simple.
25
Intelligent Database Systems Lab
Experiment and Results (cont.)
The 11×11 shows the better
performance than the 5×5, since
it considers more various input
conditions so as to generate
diverse actions.
Fig. 8. Winning rate between
5×5 behavior and 11×11
behavior at each generation
on map type 3.
26
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Experiment and Results (cont.)

For the plain map, 5×5
behavior system
shows a simple
strategy that tries to
build a town as much
as possible. Building a
town leads to generate
many NPCs so as to
slowly encroach on the
battle map as showns
in Fig. 9.
27
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
N.Y.U.S.T.
I. M.
Discussion

The reactive system shows good performance on
simple environments like the plain map, but it does
not work well for complex environments.

Also, the amount of input information is important
for the reactive system when the environment is not
simple.
28
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Conclusion

A reactive behavior system was presented for the
flexible and reactive behavior of the NPC.

Co-evolutionary approaches have shown the
potentialities of the automatic generation of excellent
strategies corresponding to a specific environment.
29
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Personal Opinion

Strength
─

Weakness
─

Designing NPCs’ behaviors without relying on human
expertise.
the limitation of direction
Application
─
real-time strategic simulation game
30
Intelligent Database Systems Lab