Transcript ppt

Games in the real world
Many real-world situations and problems can be
viewed as games of a sort:
•Players have a choice of actions,
•Players have conflicting goals,
•Players may move sequentially or simultaneously,
•Alliances may prosper,
•Treachery may occur,
•Understanding of the goals of others may be useful
in predicting their actions and planning actions of
one’s own.
•Stock Market
•War
•Passing legislation
•Hustling (“Bangneki” in Go)
•Cartels
•Fight for market share
•Biological evolution
•Industrial relations
Parlour games offer environments in which various
kinds of simplification can be made in order to focus
attention on particular AI issues: perception,
representation, reasoning, learning, opponent
modelling, and risk assessment.
http://csiweb.ucd.ie/Staff/acater/comp4031.html
•Democratic elections
•Takeover negotiations
Artificial Intelligence for Games and Puzzles
1
Some Game Theory
Game theory is the formal study of conflict and cooperation, applicable when
several “players” may act independently.
Much of the terminology in use today was introduced in 1944 in the book
Theory of Games and Economic Behaviour, by von Neumann & Morgenstern.
It is an important analytic tool for economics, and is also widely applied in
politics, war, sociology, psychology, evolutionary biology.
It allows one to reason about the strategies that rational players may adopt,
whether acting competitively, cooperatively, or in isolation of others, in deciding
upon courses of action intended to yield some payoff: even when the payoff is
merely the winning of a parlour game.
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
2
Adversarial Games Of Strategy
•Players (decision maker)
•Choices (feasible actions)
•Payoffs (benefits - or penalties -that might accrue)
• Combining whatever factors are needed to make players risk neutral
•Objectives (preferences among payoffs)
When a player must make a choice this should be done
•Not by blind chance
•But by reasoning about which action is best
• Though curiously this may involve chance after all: see mixed strategies
• Using knowledge of what is good for the player
• Also using knowledge of what is good for other players, esp. adversaries
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
3
Static vs. Dynamic games
•In dynamic games the order of moves (performance of choices by players) is
important:eg shopkeeper sets a price, customer then decides whether to buy.
•In static games, choices are unordered (perhaps simultaneous): like Roshambo,
Diplomacy.
Games may be drawn in extensive form: Consider (dynamic version) Roshambo:
Player 1
paper
scissors
rock
Player 2
p
r
s
p
r
s
p
r
s
(0,
(+1,
(-1,
(-1,
(0,
(+1,
(+1,
(-1,
(0,
0)
-1)
+1)
+1)
0)
-1)
-1)
+1)
0)
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
4
Games of perfect information, games of imperfect information
If Player 2 knows Player 1’s move when making his own choice, the game is one
of perfect information (Roshambo is then a PIG to play for player 1!).
When the game is static, moves are simultaneous, player 2 has imperfect
information. In extensive form, ovals denote information sets - nodes in the game
tree which cannot be distinguished by the player making a choice.
Player 1
paper
scissors
rock
Player 2
p
r
s
p
r
s
p
r
s
(0,
(+1,
(-1,
(-1,
(0,
(+1,
(+1,
(-1,
(0,
0)
-1)
+1)
+1)
0)
-1)
-1)
+1)
0)
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
5
Strategies, and Strategic Form
A strategy is a function which determines which choice a player makes at every
choice point. (Even those which do not arise on a particular occasion.)
Games may be described also in an alternative, so-called strategic form:
Player2:
Paper
Player2:
Rock
Player2:
Scissors
Player1: Paper
(0,0)
(+1,-1)
(-1,+1)
Player1: Rock
(-1,+1)
(0,0)
(+1,-1)
Player1: Scissors
(+1,-1)
(-1,+1)
(0,0)
A game is zero-sum iff at each terminal of the extensive form, the sum of the players’
payoffs is zero.
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
6
Two-Person Zero Sum Games & Normal Form
“Normal Form” is a compact notation for the payoffs of two players in a zero
sum game:
Roshambo =
0
+1
-1
Paper
-1
0
+1
Rock
+1
-1
0
Scissors
Paper
Rock
Scissors
The cells of the matrix show the payoffs for player1; the payoff for player2 is
trivially deduced.
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
7
Preferences over payoffs
Suppose now one does not just win or lose, but instead hurts the loser using the
winner’s chosen item (and gets enjoyment proportional to the hurt inflicted)
Ow-ha-ha =
0
+1
-5
Paper
-1
0
+4
Rock
+5
-4
0
Scissors
Paper
Rock
Scissors
How should the players behave?
One thing player1 could do is ask
•for each move I could make, what is for me the worst thing player2 could do?
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
8
Max Min
For each row i, player1 looks for the minimum payoff: mini Gi,j
He then looks for the move(s) i* which make this as good as possible:
Gi*,j = arg maxi minj Gi,j
In this case, i* = {2}
(rock)
Similarly, player2 looks for the minimum payoff in each column j, and picks the
j* that maximises the minimum payoff for player2: equivalently, minimises the
maximum payoff for player1 (zero sum means p1’s gain = p2’s loss)
Gi,j* = arg minj j maxi Gi,j
In this case, j* = {2}
(rock again)
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
9
Stable and Unstable Solutions
In this game, and in any zero-sum game, trying as hard as possible to be sure to
hurt the other is equivalent to trying to profit as much as one can.
•Each player may rationally do this by selecting the move with the least potential
for loss.
•But if one player knows that the other is doing this, they have a better choice:
player 2 for example will do better with move 1 (paper).
•But if player 1 knows that, he will prefer move 3 (scissors). And so on …
This is an unstable solution: a solution is stable if no player wants to move away
unilaterally from the solution. At such a stable solution, a game is in equilibrium.
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
10
Gain Floor, Loss Ceiling, Saddle Point, Value of Game
The Gain Floor of a two-player zero-sum game A, written (A), is the worst
outcome for player 1 if he picks his best strategy i*: (A) =min j A i*,j
The Loss Ceiling (likewise) is the best outcome for player 1 (hence worst for
player 2) if player2 picks his best strategy j*: (A) = max i A i,j*
When Gain Floor equals Loss Ceiling, the game is said to have a saddle point.
In this case, both players will do no better by unilaterally picking a different
strategy than i*, j*. This pair of strategies is a stable solution to the game.
A kind of cooperation emerges, even though the game is adversarial.
The Value of game A is the payoff of its stable solution, (A) = (A) = Value(A)
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
11
Games without saddle points
Zero-Sum games do not always have saddle points: eg Ow-ha-ha:
0
+1
-5
Paper
-1
0
+4
Rock
+5
-4
0
Scissors
Paper
Rock
Scissors
•Here i* = 2, so (Ow-ha-ha) = -1,
•while j* = 2, (Ow-ha-ha) = +1
•The pair of strategies {i=2, j=2} is not stable. A player who can find out what
the other will do can get a better result from a different strategy.
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
12
Mixed strategies
•In a game with a saddle point, each player has a best strategy (or a set of them).
• The strategies remain best even if the other players knows what they are.
•In a game without a saddle point, and hence without a stable solution,
•each pure strategy one player might adopt, if known to an opponent, can be
refuted
•A mixed strategy involves a random choice among pure strategies
•Each pure strategy is picked with its own particular probability
•Mixed strategies may be found which, even if known, cannot be refuted
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
13
Selecting the mixed strategy
A player P will pick among the possible pure strategies S1…Sn using
probabilities X1…Xn, such that  Xi = 1
P should pick these probabilities to give the highest expected payoff.
For the very simple game G =
For player 1:
4
-2
0
1
4
2
If p2 picks
column 1
0
If p2 picks
column 2
-2
If X1= 0 & X2= 1
X1=1/7, X2=6/7
http://csiweb.ucd.ie/Staff/acater/comp4031.html
If X1=1 & X2=0
Artificial Intelligence for Games and Puzzles
14
Selecting the mixed strategy - 2
With pure strategies mixed in the right proportion, player 2 gains no systematic
advantage from choosing either pure strategy even when he knows the mixed
strategy of player 1.
Player 2 can pick a mixed strategy of probabilities Y1, Y2 according to the same
principles:
4
2
If p1 picks
row 1
0
If p1 picks
row 2
-2
If Y1= 0 & Y2= 1
Y1=4/7, Y2=3/7
http://csiweb.ucd.ie/Staff/acater/comp4031.html
If Y1=1 & Y2=0
Artificial Intelligence for Games and Puzzles
15
Stability with mixed strategies
•For each player, there is an optimal mixed strategy.
•One player’s choice of strategy does not depend either
• on knowing the other player’s strategy
• Keeping the strategy secret
•The expected payoff for one player does not depend on the other’s choice of
strategy. The Value of the game is fixed. There is a kind of stability.
•These observations generalise to the case of multiple pure strategies per player.
•Von Neumann’s Minimax Theorem shows that a pair of mixed strategies can
always be found for which the players have the same expected value for the
game (which is not to say they can both expect to gain!)
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
16
Some game-theory-reading
[3] http://www.sfb504.uni-mannheim.de/glossary/game.htm
[39] http://www.cdam.lse.ac.uk/Reports/Files/cdam-2001-09.pdf
http://csiweb.ucd.ie/Staff/acater/comp4031.html
Artificial Intelligence for Games and Puzzles
17