Transcript Conclusions and future work

The Role of
Altruistic Punishment in
Promoting Cooperation
Tongkui Yu, Honggang Li
Department of Systems Science,
Beijing Normal University
The social norm is evolving
Han Fei Tzu: Legalism
(law and punishment）
Kung-fu-tzu: Confucianism
(ethics)
Qin Dynasty (turbulent society)
汉武帝：
罢黜百家，
独尊儒术
(Confuciani
sm)
Han Dynasty (stable society)
Outline




Motivation
Model
Analysis
Conclusions and future work
Cooperation is very important for a
society

of not only
human being
Outline: Motivation Model  Analysis  Conclusions and future work
Cooperation is very important for a
society

but also birds,
animals, ……
Outline: Motivation Model  Analysis  Conclusions and future work
Why people cooperate?



Natural selection (Biology) Competition
Rationality (Economics) Self-interest
Some specific mechanisms for the evolution of
cooperation





Kin selection
Direct reciprocity
Indirect reciprocity
Network reciprocity
Group selection
M. A. Nowak (2006)
Science 314, 1560 -1563
Outline: Motivation Model  Analysis  Conclusions and future work
Altruistic punishment as a mechanism
to promote cooperation

People voluntarily
incur costs to punish
violations of social
norms.
D. J. Quervain, et al (2004)
Science 305, 1254 -1258
Outline: Motivation Model  Analysis  Conclusions and future work
The role of altruistic punishment in
promoting cooperation is ambiguous

Some researchers support



Fehr, E. & Gachter, S. (2002) Nature 415, 137–
140
Henrich, J. etc. (2006) Science 312 ,1767-1770
Some researchers against

Ohtsuki, H. etc. (2009) Nature 457, 79-82.

Altruistic punishment can not lead to an efficient
equilibrium in most situations, and the efficient strategy
is to withhold help for defectors rather than punishing
them.
Outline: Motivation Model  Analysis  Conclusions and future work
Altruistic punishment as a mechanism of
promoting cooperation is ambiguous

In real world, altruistic punishment does exist
K. Sigmund, et al (2010)
Nature 466, 861 - 863
J. Whitfield (2002)
Nature 416, 782-784
Outline: Motivation Model  Analysis  Conclusions and future work
Controversy

Real world:


Theory:


Altruistic punishment exists
Ohtsuki (2009) : Altruistic punishment provides no
efficiency
Question

What does altruistic punishment exist for ?
Outline: Motivation Model  Analysis  Conclusions and future work
Our argument



Ohtsuki’s analysis only focuses on the
equilibrium (i.e. Cooperative Evolutionary
Stable State, CESS)
Although in CESS, punishment is not the
most efficient.
But from an initial state far away from
equilibrium, altruistic punishment may play an
different role in the process of approaching
cooperative evolutionary stable state (CESS).
Outline: Motivation Model  Analysis  Conclusions and future work
Our work

Extends Ohtsuki’s model to a framework with 3-level
of evolution




Reputation is updated instantaneously once an agent takes
an action.
Strategy is updated by agents according to their personal
payoff in a short period of time.
Social norm evolves very slowly according to the global
benefit of all social members in a considerably long period
of time.
Studies the dynamics of the strategy frequency
under social norms with different punishment
attitudes



Non-punishment social norm
Punishment-alternative social norm
Punishment-provoking social norm
Outline: Motivation Model  Analysis  Conclusions and future work
Our results

Altruistic punishment really works in
promoting cooperation in at least two ways:

(1) enlarges the attraction basin of CESS
Non-punishment
social norm
Punishment-alternative
social norm
Punishment-provoking
social norm
Outline: Motivation Model  Analysis  Conclusions and future work
Our results

Altruistic punishment really works in
promoting cooperation in at least two ways:

(2) increases the rate of convergence to CESS
Blue: Punishmentalternative norm
Red: Punishmentprovoking norm
Outline: Motivation Model  Analysis  Conclusions and future work
The Role of Altruistic Punishment in
Promoting Cooperation




Motivation
Model
Analysis
Conclusions and future work
Outline: Motivation Model  Analysis  Conclusions and future work
Model



A society with infinitely large population.
Each individual is endowed with a binary
reputation: good (G) or bad (B).
Everyone in the society agrees on the
reputation of an individual.
Outline: Motivation Model  Analysis  Conclusions and future work
Model



At each small time interval, two players are sampled
randomly. One as donor and the other as recipient.
Donor has 3 choices: cooperation (C), defection (D),
and punishment (P). Recipient does nothing.
If C, donor spends a cost c (c=2) to give a benefit b (b=3)
to recipient; if D, no gain no loss; If P, donor spends a
cost α (α=1) to give a loss β (β=4) to recipient.
Donor
D
R
Recipient
C
D
P
(b,-c)
(0,0)
(-β,-α)
b3 c2
 1
Outline: Motivation Model  Analysis  Conclusions and future work
 4
Strategy (or action rule)
J


Each player has a
strategy
The strategy determines
the donor’s action X (C,
D, or P) according to the
reputation J (G or B) of
recipient.
32  9
X
G
C
C
C
D
D
D
P
P
P
B
C
D
P
C
D
P
C
D
P
Outline: Motivation Model  Analysis  Conclusions and future work
Model

After each interaction, the reputation of the
donor is updated according to the ‘social norm’
D
R
Outline: Motivation Model  Analysis  Conclusions and future work
Social norm


Assign a new reputation
to the donor
According to not only the
action (X) of the donor,
but also the reputation (J)
of the recipient.
J
G
B
C
G
G
X D
B
G
P
B
G
Outline: Motivation Model  Analysis  Conclusions and future work
Cooperative Evolutionary Stable State
(CESS)


Cooperative : Most agents cooperate
Evolutionary Stable : Given all other agents take
some strategy, the best choice for an agent is to
take that strategy.
C
D
P
G
G
B
B
B
G
G
G
CD(CESS): all
agents take
CD strategy
C D
Outline: Motivation Model  Analysis  Conclusions and future work
Cooperative Evolutionary Stable State
(CESS)

Whether is there a CESS, what is the CESS

depends on social norm
C
D
P
G
G
G
G
B
G
G
G
DD (all agents
take DD strategy)
is the only stable
state.
D D
No CESS.
Outline: Motivation Model  Analysis  Conclusions and future work
Ohtsuki’s work(1)

Finds social norms that can foster cooperative
evolutionary stable state (CESS) from 64 candidates
G
C G
D B
P B
B
*
G
*
C D
G
C G
D B
P B
B
*
B
G
C P
*: both G and B are OK.
Outline: Motivation Model  Analysis  Conclusions and future work
Ohtsuki’s work(2)

Find the most efficient social norm for different
parameter settings

Indirect reciprocity provides only a narrow margin of
efficiency for altruistic punishment.
Outline: Motivation Model  Analysis  Conclusions and future work
Extend Ohtsuki’s work

Not just focus on the equilibrium

But try to model the dynamics of strategy
frequency explicitly

Study the dynamics of the strategy frequency under
social norms with different punishment attitudes.
Outline: Motivation Model  Analysis  Conclusions and future work
Social norms
Non-punishment
norm
Punishment-alternative
norm
Punishment-provoking
norm
Outline: Motivation Model  Analysis  Conclusions and future work
Extend Ohtsuki’s work

To explicitly model the dynamics of strategy
frequency

Key question

When and how does an agent update his strategy ?

Can not use the income of one interaction to measure the
fitness of a strategy, because


Agents can be donor or recipient in one interaction.
Agents’ reputation changes instantly.
Outline: Motivation Model  Analysis  Conclusions and future work
Stable reputation frequency

For a fixed strategy frequency, there is a stable
reputation frequency for each strategy .
x1
x2
x3  1  x1  x2
Non-punishment norm
Outline: Motivation Model  Analysis  Conclusions and future work
When and how does an agent update
his strategy

An agent update his strategy



After they can actually perceive the payoff of
different strategies, i.e. after reputation frequecy
converges to the stable state.
According to the expected payoff of his strategy in
the stable reputation frequecy
Anaylitical tractable  Replicator dynamics
Outline: Motivation Model  Analysis  Conclusions and future work
Model

Strategy Frequency Dynamics
p1 
1
1
(c)  bx1  bx2 (1   ) 
2
2
p2 
1
1
g (c)  bx1  bx2 (1   ) 
2
2
p3 
1
1
(0)  bx1  bx2 g3 
2
2
p1  p1  p3 ; p2  p2  p3
Non-punishment
norm
 x1  x1 ( p1'  p )

'
 x2  x2 ( p2  p )
Outline: Motivation Model  Analysis  Conclusions and future work
Model

Strategy Frequency Dynamics
1
1
1
p1  (c)  x3  (  )  bx1  b  x2  x3  (1   ) 
2
2
2
1
1
1
p2  g (c)  x3  (  )  bx1  b  x2  x3  (1   ) 
2
2
2
1
1
1
1
p3  g (c)  (1  g )( )  x3  (  )  bx1  b  x2  x3  (1   ) 
2
2
2
2
1
1
p4  bx1  b  x2  x3  g 4   x3 (1  g 4 )(  )
2
2
p1  p1  p4 ; p2  p2  p4 ; p3  p3  p4
Punishment-alternative
norm
 x1  x1 ( p1'  p )

'
x

x
(
p
 2
2
2  p)

'
x

x
(
p
3
3  p)
 3
Outline: Motivation Model  Analysis  Conclusions and future work
Model

Strategy Frequency Dynamics
1
1
1
p1  (c)  x3  (  )  bx1  b  x2  x3  (1   ) 
2
2
2
1
1
1
p2  g (c)  x3 (1  g 2 )(  )  bx1  bg 2  x2  x3  
2
2
2
1
1
1
1
p3  g (c)  (1  g )( )  x3  (  )  bx1  b  x2  x3  (1   ) 
2
2
2
2
1
1
p4  bx1  b  x2  x3     x3 (1   )(  )
2
2
p1  p1  p4 ; p2  p2  p4 ; p3  p3  p4
Punishment-provoking
norm
 x1  x1 ( p1'  p )

'
x

x
(
p
 2
2
2  p)

'
x

x
(
p
3
3  p)
 3
Outline: Motivation Model  Analysis  Conclusions and future work
The Role of Altruistic Punishment in
Promoting Cooperation




Motivation
Model
Result Analysis
Conclusions and future work
Outline: Motivation Model  Analysis  Conclusions and future work
Phase portrait of three social
norms
Attraction
basin ratio of
CESS in entire
space
  0.02 b  3 c  2
 1   4
15%
60%
81%
Outline: Motivation Model  Analysis  Conclusions and future work
Typical trajectories under ‘GGBGBG’
and ‘GGBBBG’ norm
Outline: Motivation Model  Analysis  Conclusions and future work
The effect of parameters to CESS
attraction ratio
β
α
c
b
  0.02 b  3 c  2
 1
 4
Converge rate from the same initial
point in different social norm
Blue: Punishmentalternative norm
Red: Punishmentprovoking norm
Outline: Motivation Model  Analysis  Conclusions and future work
The Role of Altruistic Punishment in
Promoting Cooperation




Motivation
Model
Analysis
Conclusions and future work
Outline: Motivation Model  Analysis  Conclusions and future work
Conclusion

Altruistic punishment really works in
promoting cooperation in at least two ways:


(1) enlarges the attraction basin of CESS
(2) increases the rate of convergence to CESS
Outline: Motivation Model  Analysis  Conclusions and future work
Han Fei Tzu: Legalism
(law and punishment）
Kung-fu-tzu: Confucianism
(ethics)
Future works




Formal modeling of converge rate
Group selection for social norm evolution by
simulation
The effect of abnormal strategies (such as
DC strategy)
Experiment to test the converge rate and
attraction basin ratio in different social norm
Outline: Motivation Model  Analysis  Conclusions and future work
Acknowledgement

Prof. Shu-heng Chen

AI-ECON Research Center, National
Chengchi University
Thank you!