Transcript Second-order theory of mind

Savvy software agents can encourage
the use of second-order theory of mind
by negotiators
Harmen de Weerd, Eveline Broers, Rineke Verbrugge
Institute of Artificial Intelligence, University of Groningen
인지과학 방법론
발제일: 2015. 9. 16
발제자: 이윤진 (석사 2학기)
Contents
• Big Question
• Brief Introduction
• Theoretical Backgrounds
• Theory of Mind
• Mixed Motive Situation
• Colored Trails
• Experiment
• 선행연구
• Experiment Design
• Results
• Discussion
• Q&A
Big Question
Artificially
Intelligent
Agent
“Savvy software agents can encourage the use of second-order theory of mind by negotiators“
in Mixed-motive Situation (Colored Trails game)
Software
agent with
Theory of
Mind
Human
with
Theory of
Mind
Theory of Mind [마음이론]
• What is ‘Theory of Mind’?
“People use this theory of mind
(1) to understand why other people behave in a certain way, (2)
to predict their future behaviour, and (3) to distinguish
between intentional and accidental behaviour. People also take
this ability one step further, (4) and consider that others have
a theory of mind as well.
This second-order theory of mind allows people to consider
and even expect that others will understand why they behave
the way that they do.”
(Premack & Woodruff, 1978)
Human
with
Theory of
Mind
Theory of Mind [마음이론]
• Second-order theory of mind
(Premack & Woodruff, 1978)
“allows people to reason explicitly about belief attributions made by others.
For example, in the sentence
“Alice knows that Bob knows that Carol is throwing him a surprise party”,
a second-order knowledge attribution is made to Alice, in which she
attributes knowledge to Bob.
The human ability to make use of higher-order (i.e. at least second-order)
theory of mind is well-established, both through tasks that require explicit
reasoning about second-order belief attributions, as well as in strategic
games.”
https://cdn.psychologytoday.com/sites/default/files/blogs/276/2008/05/739-74445.jpg
Theory of Mind [마음이론]
• Theory of Mind as a system
• Agent-based computational modeling
• a research tool
• how behavioral patterns emerge from the interactions between
individuals
• allows precise control and monitors of the mental content of agents,
including application of theory of mind
• zero-order Theory of Mind (ToM0)
first-order Theory of Mind (ToM1)
second-order Theory of Mind (ToM2)
http://www.ieet.org/images/uploads/p20130730b.jpg
Software
agent with
Theory of
Mind
Mixed Motive Situation [동기 갈등 상황]
• as the task of sharing a pie
partially cooperative
&
partially competitive
- pie 늘리기
- pie 분배하기
• Colored Trails
http://peachypalate.com/wp-content/uploads/2014/06/DeepDishBerryCrumblePie15.jpg
Mixed Motive Situation [동기 갈등 상황]
• Colored Trails
• a board game introduced by Grosz, Kraus, and colleagues
(Lin et al., 2008; Gal et al., 2010)
• negotiation setting
• a research test-bed
to study decision making
in humans and computer agents
• Interactions in mixed-motive settings
• trading chips (협력) & win the game (경쟁)
Colored Trails
출발
목표지점 (goal location)
Negotiation
Colored Trails
 Multi-Issue Bargaining Situation
issue
issue
issue
issue
issue
issue
issue
issue
issue
issue
Colored Trails
• https://coloredtrails.atlassian.net/wiki/display/coloredtrailshome/
Experiment
in Mixed-motive Situation (Colored Trails game)
Software
agent with
Theory of
Mind
Human
with
Theory of
Mind
Experiment - Methods
• Participant: Human-software agent
• Human (27 students of the University of Groningen, 10 female, 17 male)
• Software agent (ToM0, ToM1, ToM2)
• 24 games from randomly generated games
• 목표점은 chip 8개로 도달가능 하도록,
• Computational agent가 피험자가 사용하는 마음이론에 따라 다른 결과를 내도록,
• 제안 횟수에 한계가 있도록 (2회~6회),
• Familiarization phase ( Experimental phase)
• Cover story: 회사 법적대리인으로서 다양한 고객과 협상하는 상상하라
• 게임 상대방은 computer player(Alex)라 고지
• 게임 원칙 확인
Experiment - Methods
• 3 blocks (ToM0, ToM1, ToM2 condition, random order) X 8 games/block = 24 games
• 각 block에서 상대방의 전략이 바뀐다는 정보 없이 다른 고객을 상대할 것이라 고지
• 누가 먼저 제안할지 (initiator)는 무작위적으로 배당, 그 후 번갈아 가면서 협상 제안
8 games
[ToM0]
8 games
Human
[ToM1]
8 games
Human
[ToM2]
Human
Experiment - Methods
Theory of Mind in Software Agents
• adapted from De Weerd et al. (2013)
• software agent {ToM0, ToM1, ToM2} X {ToM0, ToM1, ToM2}
• competitive and cooperative aspects 각각 살펴봄
• The use of first-order and second-order theory of mind
allows software agents to balance competitive and cooperative aspects of the game.
• the use of theory of mind prevents negotiations from breaking down
• {software agent} X {human} 연구
 to what extent human participants reason at higher orders of theory of mind
in response to each level of theory of mind
• More than 100 offers  now only six turns
Experiment – Methods
Colored Trails
• 5X5 tiles with four patterns
• Chips and Goal location randomized
• + Uncertainty
상대방의 goal location을 알 수 없음
Reason and update
my belief in partner’s goal location
(and use it in negotiation)
by using theory of mind
• Theory of Mind
software agent- 각 block마다 고정
human participant – 고정 X, ToM3로 추정
Experiment - Methods
Theory of Mind in Software Agents
Zero-order Theory of Mind
(ToM0)
First-order Theory of Mind
(ToM1)
Second-order Theory of Mind
(ToM2)
mental content (goal location) 추론X
O
O
Zero-order belief
First-order belief
Second-order belief
:제안이 수용될 likelihood
Parameter: learning speed (lambda 0~1)
 상대방이 받아들일 만한 제안으로만 추론
예. 적은 수의 chip보단 많은 수의 chip 제안
•
positional bargaining (Fisher & Ury, 1981)
: what its own decision would have been if it
had been in the position of its partner
 the partner has belief similar to my
own이라 추론
 상대방이 제안한 내용에서
상대방의 goal location 추론
 counter-offer
 use Zero-order belief & First-order
belief
: knows the fact that ‘상대방이 나의 제안을 분
석하여 나의 목표점을 추론해낼 수 있다’
•
•
•
•
•
partner could be ToM1
제안에 따라 상대방이 갖고 있는
‘나의 goal location＇에 대한 믿음이
바뀐다는 것을 추론
상대방에게 나의 목표점에 대한 정
보를 제공하여 원하는 chip을 얻어
내고자 함
use Zero, First, Second order belief
사용
interest-based negotiation
(Fisher & Ury, 1981)
Experiment - Methods
Theory of Mind in Software Agents
Zero-order Theory of Mind
[ToM2] (ToM0) Human
First-order Theory of Mind
(ToM1)
Second-order Theory of Mind
(ToM2)
mental content (goal location) 추론X
O
O
Zero-order belief
First-order belief
Second-order belief
:제안이 수용될 likelihood
Parameter: learning speed (lambda 0~1)
 상대방이 받아들일 만한 제안으로만 추론
예. 적은 수의 chip보단 많은 수의 chip 제안
•
positional bargaining (Fisher & Ury, 1981)
: what its own decision would have been if it
had been in the position of its partner
 the partner has belief similar to my
own이라 추론
 상대방이 제안한 내용에서
상대방의 goal location 추론
 counter-offer
 use Zero-order belief & First-order
belief
: knows the fact that ‘상대방이 나의 제안을 분
석하여 나의 목표점을 추론해낼 수 있다’
•
•
•
•
•
partner could be ToM1
상대방도 incomplete information 갖
고 있음을 알고 있음
제안에 따라 상대방이 갖고 있는
‘나의 goal location＇에 대한 믿음이
바뀐다는 것을 추론
상대방에게 나의 목표지점에 대한
정보를 제공하여 원하는 chip을 얻
어내고자 함
use Zero, First, Second order belief
사용
Results
How the score of agents and participants
changed as a result of negotiation for each
participant and each block
• 점선: zero performance line
직선: Pareto efficient outcomes
allocation of resource in which it is possible to
make any one individual better off without making
at least on individual worse off
• Performance Ranking
#1: Second-order Theory of Mind
#2: Zero-order Theory of Mind
#3: First-order Theory of Mind
Results
To what extent participants make use of theory
of mind while playing Colored Trails
(how similar participant’s offers were to ToM0, ToM1,
ToM2 agents)
• ToM3 ‘spectator agent’
: 유사성 측정
: whether the offers of a participant are most
consistent with zero-,first-,or second theory of mind
reasoning
: 각 마음이론의 confidence 측정
- which order of theory of mind would yield the best
outcome
•
Level of
Theory of Mind
Chi-square Results
Zero-order
(X^2 (2) =0:52, ns)
First-order
(X^2 (2) =2:67, ns)
Second-order
(X^2 (2) =24.89, p<0.001)
Results
Effects of opening bid of a negotiation
• 사전연구: 협상 과정에 영향을 미침 (Raiffa et al., 2002)
• Average Outcome:
[initiator] software agent
 software agent & 피험자 모두 평균 +15점
• Software agent ToM2 조건
[initiator] ToM2 피험자 –----- [responder] ToM2 software agent
 피험자 > agent
Discussion 1–
perceived agency problem
• Experimental setting:
informed participants that they were interacting with ‘client’
Would their use of Theory of Mind have changed
if they knew it was not ‘client’, but ‘software’?
Discussion 2–
Colored trails as a method to study human-computer interaction
• Colored Trails: cooperation X competition setting
사회적으로 있을 법한 mixed motive situatio에서
사용하는 마음 이론의 유용성을 따져보는 데는 유용해 보이지만
이를 인간-컴퓨터 상호작용 연구에 적용하는 것이 적합할까?
Discussion 3Augmenting Human Intelligence vs. Building Artificial Intelligence
https://youtu.be/D156TfHpE1Q
https://youtu.be/sNExF5WYMaA
• Both are autonomous agent and would have ‘Theory of Mind’
Should the software program with Theory of Mind be modified
to be more human-like (AI)
or to make an outcome closer to Pareto Optimal Line(IA)?
• Artificial Intelligence (AI)
building a human-like intelligence in the form of an autonomous technological system
• Intelligence amplification (IA)
the effective use of information technology in augmenting human intelligence
Douglas Engelbart
Reference
• de Weerd, H., Verbrugge, R., & Verheij, B. (2013). Higher-order theory of mind in negotiations under
incomplete information. In PRIMA 2013: Principles and Practice of Multi-Agent Systems (pp. 101-116).
Springer Berlin Heidelberg.
• de Weerd, H., Broers, E., & Verbrugge, L. (2015). Savvy software agents can encourage the use of secondorder theory of mind by negotiators. In Proceedings of the 37th Annual Conference of the Cognitive
Science Society. Pasadena: Cognitive Science Society.
• Premack, D., & Woodruff, G. (1978). Does the chimpanzee have a theory of mind?. Behavioral and brain
sciences, 1(04), 515-526.
• https://coloredtrails.atlassian.net/wiki/display/coloredtrailshome/Colored+Trails+Homepage
• https://en.wikipedia.org/wiki/Intelligence_amplification#Douglas_Engelbart:_Augmenting_Human_Intellect
Thank you for your attention
Q&A