Transcript Mechanism Design for Computationally Limited Agents

Mechanism Design for
Computationally Bounded Agents
Kate Larson
Carnegie Mellon University
Thesis Proposal
April 8, 2002
1
Introduction
• Fundamental problem in building open
distributed systems:
– There are users and agents acting in their own
self interest.
– Designer wishes to guarantee optimal system
wide solutions while being unable to enforce
particular behavior on users.
2
Introduction
• Techniques from game theory and
mechanism design have been very
useful:
– Routing in networks
– Resource allocation in operating systems
– Auction design in electronic markets
– Bidding agents for electronic auctions
– ….
3
Introduction
• However, perfect rationality is often
assumed!
• But in real systems…
Cost
Time constraints
Hard problems
4
Package Delivery and Vehicle
Routing
Chicago
Chicago
to Home
Toronto to
Home
Home
Home
to
Chicago
The delivery
Chicago to route can be
computed.
Toronto
2
3
4
Depot
1
5
Toronto
5
Package Delivery and Vehicle
Routing
Chicago
Home
Chicago
to Home
Chicago to
Toronto
4
2
1
Home to
Chicago 3
Toronto to
Home
Depot
5
Toronto
The new package can be easily fit into the
delivery route. The cost can be kept low,
and thus the bid also.
6
Computing of Valuations
• Agents may need to compute their valuations for
(bundles of) goods
– Vehicle routing problem in transportation exchanges
– Manufacturing scheduling problem in procurement
• Value of a set of items
=value of solution with items - value of solution without
• May involve solving multiple NP-complete problems
– Infeasible to solve optimally
7
Incentives
• Game theory handles incentives
– System designer specifies the mechanism
(auction rules)
– Each agent picks its own strategy
• Computational issues have to be handled
also!
– Computing as part of an agent’s strategy
• Interaction between incentives and computing
– Equilibrium for rational agents is not always one
for computationally bounded agents!
8
My Approach
Resource
Bounded
Reasoning
from AI
Game Theory and
Mechanism
Design
Normative model of bounded rationality.
Mechanism design for computationally
bounded agents.
9
Thesis Statement
• It is possible to incorporate agents’
computational actions into a game
theoretic model.
• Agents’ computational restrictions have
a strong impact on the choice of
strategies.
• It is possible to design mechanisms that
take into account agents’ computational
restrictions.
10
Structure
Model of bounded
rationality
Game theoretic formulation
Impact on existing
mechanisms
Mechanisms for
bounded agents
Experiments
11
Outline
• A model of bounded rationality
– A normative model of deliberation control for
computationally bounded agents
• Game theory for computationally bounded
agents
• The impact of agents’ computation
• Mechanism design for computationally
bounded agents
• Experiments
• Schedule
12
Restricted Computational
Resources
• Agents have limited computational
resources
– Deadlines
• All parcels must be delivered by Friday
evening!
• Battery on my laptop only lasts 3 hours
– Cost associated with computation and
deliberation
• Sending jobs to rendering farms
13
Anytime Algorithms
• Anytime algorithms can be used to
approximate valuations
– Anytime algorithms can return a solution at any time
– Solution improves over time
Allows a trade off between computing time and
solution quality
• Examples
– Iterative refinement algorithms: Local search,
simulated annealing
– Search algorithms: Depth first search, branch and
bound
14
Performance Profiles
• Performance profiles describe how computation
changes the solution
– Characterize the quality of an algorithm’s output as a
function of computing time
• Performance profile representation is important
– Earlier methods were not normative: They did not
capture all possible ways an agent can control its
deliberation
15
Performance Profiles
Deterministic
performance profile
Solution
quality
Variance introduced by
different problem instances
Solution
quality
Optimum
Computing time
[Horvitz 87, 89, Dean & Boddy 89]
Computing time
16
Table-based Representation of Uncertainty in
Performance Profiles
[Zilberstein & Russell 91, 96]
.08 .19 .24
.15 .30 .17 .39
.16 .10 .16 .25 .30 .22
Solution
quality
.08
.04 .17
.20 .22 .30 .24 .19 .15
.09
.10
.20 .22
.23 .37 .31 .13 .15
.11
.14
.33 .18
.21 .18 .08
.22
.17
.25 .24
.15 .13
.40
.31
.15 .19
.05
.15
.20
.03
Conditioning on
solution quality so far
[Hansen &
Zilberstein 96]
Ignores conditioning
on the path
.03
Computing time
17
Performance Profile Tree
[Larson & Sandholm 2001a]
P(B|A)
B
4
5
4
A
0
Solution
Quality
P(1)
P(2)
P(C|A)
C
P(3)
2
5
Value Node Stores value of solution so far
10
7
15
20
Random Node Captures uncertainty from random
algorithms, unknown algorithms
18
Properties of
Performance Profile Trees
• Normative - allows conditioning on everything
including
– Path of solution quality
– Path of other solution features
– Problem instance features
• Can model uncertainty from
– Randomized algorithms
– Lack of knowledge about what algorithms other
agents use
– Problem instances
19
Outline
• A model of bounded rationality
• Game theory for computationally bounded
agents
– Roles of computation
– Strategic computing
– Deliberation equilibrium
• Effects of agents’ computation
• Mechanism design for computationally
bounded agents
• Experiments
• Schedule
20
Roles of Computing
• Improving:
– Agents compute to
improve valuations
– Computation
changes the
valuation
• Example: Solving
TSP’s to find cost of
best route
• Refining:
– Agents compute to
refine valuations
– Computation
changes agents’
beliefs about
valuations
• Example:
Researching if an art
work is a forgery
21
Roles of Computation
•
For computationally bounded agents,
computing has several strategic roles:
1. Improves or refines the solution to the agent’s
own problem
2. Reduces uncertainty as to what future
computing steps will bring
3. Improves the agent’s knowledge about others’
valuations
4. Improves the agent’s knowledge about what
problems others may have computed on
22
Strategic Computing
• Good estimates of other agents’ valuations can
allow an agent to tailor its strategy to achieve
higher utility
• Strong strategic computing: Agent uses some
of its deliberation resources to compute on
others’ problems
• Weak strategic computing: Agent uses
information from others’ performance profiles
23
Strategic Computing
• How an agent should allocate its
computation can depend on
– how others allocate their computation
– what actions they all plan on taking
Incorporate computation actions into agents’
strategies.
Analyze games for deliberation equilibria
24
Deliberation Equilibria
•
A (Nash, dominant, etc.) deliberation
equilibrium for computationally bounded
agents is a (Nash, dominant, etc.)
equilibrium where:
1. The agents’ computing strategies form a
(Nash, dominant, etc) equilibrium, and
2. The agents’ noncomputing strategies form a
(Nash, dominant, etc) equilibrium.
25
Outline
• A model of bounded rationality
• Game theory for computationally bounded
agents
• Effects of agents’ computation
– One-to-One Negotiation (Bargaining)
– One-to-Many Negotiation (Auctions)
– Many-to-Many Negotiation (Exchanges)
• Mechanism design for computationally
bounded agents
• Experiments
• Schedule
26
Bargaining
• Bargaining
B
A
B
A
A
A
B
A
B
B
27
Bargaining
[Larson and Sandholm, 2001a]
• Take-it-or-leave-it: One agent makes an offer
to the other who can accept or reject.
– Determined under what settings agents have
dominant strategies.
– Determined under what settings agents will not split
their computation.
– Designed algorithms for determining offline, agents
optimal online strategies
28
Bargaining
[Larson and Sandholm, 2002]
• Alternating Offers Model: Agents take turns
making proposals and counter proposals
• Despite richer strategy space, alternating
offers model is similar to take-it-or-leave it
model
– In (Bayes Nash) deliberation equilibrium, agents behave as
though they were in take-it-or-leave-it setting
29
Future Bargaining Work
• So far have only worked in settings
where computation is free but limited
– What happens if computation is costly?
Conjecture: Costly computation will not
change the results.
30
Auctions and Strategic Computing
Auction
mechanism
Single First price sealed-bid
item for
Dutch (1st price descending)
sale
Vickrey (2nd price sealed bid)
English (1st price ascending)
Multiple Generalized Vickrey
items On which agent, bundle
for sale pair to allocate next
computation step ?
Counterspeculation
by rational
agents ?
Strong strategic computing
Limited
computation
Costly
computation
yes
yes
yes
yes
yes
yes
no
no
yes
no
no
yes
no
yes
yes
[Larson and Sandholm 2001b, Larson and Sandholm 2001c]
31
Exchanges
• Exchanges are a generalization of
auctions where there are potentially
multiple buyers and multiple sellers
– Does strategic computing occur in
exchanges?
– Does the type of restriction on agents’
computational capabilities determine the
occurrence of strategic computation?
Conjecture: Strategic computation can occur with either
type of restriction (costly or limited computation).
32
Outline
• A model of bounded rationality
• Game theory for computationally
bounded agents
• Effects of agents’ computation
• Mechanism design for computationally
bounded agents
• Experiments
• Schedule
33
Mechanism Design
•
•
So far have focused on standard
mechanisms and determining their
strengths and limitations if agents
have computational restriction.
Change focus:
1. Mechanisms for computationally
restricted agents.
2. Measures for mechanisms.
34
Issues for Mechanism Design
• There are several issues that must be
addressed for computationally restricted
agents
– What should agents report to the mechanism?
• Valuations once computed?
• Valuations agents expect to obtain?
• Results of partial computation?
– What is the role of the mechanism?
• To specify allocations only?
• To specify allocations and computation policies?
35
Issues for Mechanism Design
• Does there exist a mechanism that is:
– Strategy proof,
– Efficient,
– Individually rational, and
– Limits wastage of computational resources.
36
Measuring Mechanisms:
Miscomputing Ratio
• How does allowing agents free choice of
computing policies effect the outcome, o?
• Miscomputing Ratio:
R=SW(o*)/SW(o(NE))
SW(o*)=Social welfare if a global controller
enforces computing policies so as to
maximize social welfare
SW(o(NE))=Social welfare of worst Nash Eq.
37
Results and Future Work
• Results (Vickrey auctions):
– If agents have limited or costly computation then the
miscomputing ratio can be unbounded.
– The choice of appropriate cost functions can motivate
bidders to choose strategies that maximize social
welfare.
• Future Work:
–
–
–
–
Constraining deadlines or cost functions
Using performance profile trees to predict R
Constraining performance profile trees
Comparing mechanisms using miscomputing ratio, R
38
Outline
• A model of bounded rationality
• Game theory for computationally
bounded agents
• Effects of agents’ computation
• Mechanism design for computationally
bounded agents
• Experiments
• Schedule
39
Experiments
• Goals:
– To demonstrate that the performance
profile tree is a feasible model.
– To demonstrate that the performance
profile tree model is general.
– To determine if strategic computing is an
artifice of analysis or something that occurs
in the “real world”.
• Tools for controlling computation and
visualization of deliberation and negotiation
40
Status
completed
partially
future work
Model of bounded
rationality
Game theoretic formulation
Impact on existing
mechanisms
Mechanisms for
bounded agents
Experiments
41
Schedule
Bargaining (Costly)
Exchanges
Mech. Design
Miscomputing Ratio
Experiments
Writing
0
Sp02
1
Sm02
2
F02
3
Sp03
4
Sm03
5
F03
6
7
42
Related Publications
• Larson and Sandholm, 2001a, Bargaining with Limited
Computation:Deliberation Equilibrium, Artificial
Intelligence, 132(2), pp.183-217
• Larson and Sandholm 2001b, Computationally Limited
Agents in Auctions, Workshop on Agent based
approaches to B2B, Autonomous Agents 2001
• Larson and Sandholm, 2001c, Costly Valuation
Computation in Auction, TARK VIII
• Larson and Sandholm, 2002, An Alternating Offers
Bargaining Model for Computationally Limited Agents,
AAMAS 2002
• Larson and Sandholm, 2002, Miscomputing Ratio: The
Social Cost of Selfish Computing, draft.
43
Other Publications
• Larson and Sandholm, 2000, Anytime Coalition
Generation: An Average Case Study, Journal of
Experimental and Theoretical Artificial Intelligence,
12(2000),23-42.
• Sandholm, Larson, Andersson, Shehory, and Tohme,
1999, Anytime Coalition Structure Generation with
Worst Case Guarantees, Artificial Intelligence, (111)12, pp209-238.
44
Future Auction Work:
Reverse Auctions
• Are they different from standard auctions
if agents have bounded computational
resources?
– Maybe or maybe not!
– Reverse auctions differ from auctions in:
• Complexity of winner determination
• Communication complexity
– At first look, it does not appear as though
bidding strategies are different.
45
Related Work
• Parkes 2001, Iterated combinatorial auctions so that
both the computational burden of agents and
mechanism are reduced
• Persico 2000, Information acquisition in auctions
• Bergemann and Pesendorfer 2001, Model where the
auctioneer controls what information agents have
available to them
• Bergemann and Välimäki 2001, Costly information
acquisition, but no clear model of how information is
obtained
46
Related Work
• Nisan and Ronen 2000: Algorithmic
mechanism design. What happens if the
mechanism is computationally limited?
Investigates ways of introducing
approximation algorithms to compute
outcomes without loosing incentive
compatibility.
• Kfir-Dahav, Monderer, Tennenholtz 2000
47
Experiments
• To demonstrate that the performance profile
tree is a feasible model:
– Multi-item auction where agents submit bids
where the value is determined by the solution to a
TSP problem
– Prebundling to simplify the winner determination
problem and to restrict the number of optimization
problems agents are faced with
– Focus on having agents use performance profile
trees, determining how to make the performance
profile tree model feasible for actual problems.
48
Experiments
• To demonstrate that the performance
profile tree is general:
– Single item auction where agents
valuations of the item is based on
schedules they compute.
– Use off the shelf schedulers (Micro-Boss,
or a stochastic scheduler from Steve
Smith’s group) to show that the techniques
I propose are general
49
Experiments
• To determine if strategic computation is an
artifice of analysis or something that occurs in
the “real world”:
– Multi-item reverse auction using data from actual
delivery companies
– Agents must solve multiagent vehicle routing
problems
– How does one tailor performance profile trees to
actual data without losing their analytical power?
– Does strategic computing occur?
50
Game Theory Concepts
• Game theory: method of studying systems of
agents in settings where there are strategic
interactions
• Game has
– Set of agents, A
– Set of outcomes, O
– Set of actions for each agent
• Strategy: contingency plan that determines
what action the agent will take at any point in
the game
• Equilibrium is fundamental concept in game
theory
– Stable point in space of agents’ strategies
51
Game Theory Concepts
• Different equilibrium concepts
– Dominant strategy equilibrium
• Every agent has a strategy that they are best off playing,
independent of other agents actions
– Nash equilibrium
• The strategy one agent plays is optimal given what
strategies other agents are playing
• Mechanism Design:
– Study of how to implement good system wide solutions that
involve mulitple self-interested agents
52
Mechanism Design
• Mechanism Design:
– Study of how to implement good system wide
solutions that involve multiple self-interested
agents
• Mechanism: M=(S1,…,Sn,g(.)) is a set of
strategies Si available to each agent i, and an
outcome function g(s).
– Defines strategies available to agents
– Describes methods used to determine outcome
based on agents’ strategies
53
Game Theory
• Game theory studies phenomena that occur
when decision makers interact
– It is assumed that decision makers are strategic.
If he bids $10
then I could bid
$11 and win.
If he bids $11
then I could bid
$12 and win.
54
Auctions with Individual
Bidders
• In auctions where the bidders are individual
consumers, valuations are often idiosyncratic
$5 for
the bear
$100
for the
bear
55
The Two Extremes
Hamlet: “What a piece of work is a man! How
noble in reason! How infinite in
faculties!”
Hamlet, II.2.319
Puck: “Lord, what fools these mortals be!”
Midsummer Night’s Dream, III.3.116
56
One Solution?
“It is evident that the rational thing to do is to
be irrational, where deliberation and estimation
cost more than they are worth.”
Frank Knight, Risk, Uncertainty and Profit,
1921
57
Example of Bounded
Rationality
According to game theory, chess is a trivial game.
-finite, zero sum, two player game with full information
Then why do we find it so challenging to play and
why is it difficult to get computers to play well?
58