Transcript Slides from the AAMAS 2002 talk

Learning and Exploiting Context
in Agents
•
•
•
•
•
About Context
Learning Domains with Content
Trying it out
Integrating Learning and Inference
Learning Context
Bruce Edmonds, AAMAS 2002
(Talk Outline)
Context in Human Cognition
•
•
•
•
•
•
Bruce Edmonds, AAMAS 2002
language
memory
concepts and categorization
affect
social cognition
(probably) reasoning
(About Context)
Context In AI
John McCarthy (1971), Generality in Artificial Intelligence
c : ist (i, p)
“p is true in context i” asserted in context c
p
r
q
s
i
c
Bruce Edmonds, AAMAS 2002
(About Context)
Context In ML
Main purposes:
• to maintain learning when there is a
hidden/unexpected change in context
• to apply learning gained in one context
to different context
• to utilise already known information
about contexts to improve learning
Bruce Edmonds, AAMAS 2002
(About Context)
The Problem
3 choices:
• Use Global Approaches
– But inference and learning can be hard
•
Specify all the contextual information
– Can be onerous
•
Learn Contextual Information with
Content
– Need an algorithm
Bruce Edmonds, AAMAS 2002
(About Context)
Bruce Edmonds, AAMAS 2002
(About Context)
Transfer of knowledge from
learning to application context
Is (only) possible when:
1. some of the possible factors influencing an
outcome are separable in a practical way
2. foreground features and others can be
usefully distinguished
3. background factors are capable of being
recognized later
4. world is regular enough for
– such models to be learnable
– such learnt models to be useful when applied
Bruce Edmonds, AAMAS 2002
(About Context)
Fuzzy Domain & Crisp Content
Bruce Edmonds, AAMAS 2002
(About Context)
Coincident Clusters of
Domain&Content make a Context
M1
M1 M2
Abstract to a context
Bruce Edmonds, AAMAS 2002
(About Context)
An Evolutionary Algorithm
D
3.7
2.1
p
0.9
2.2
Some Space of Characteristics
Bruce Edmonds, AAMAS 2002
(Learning Domain & Content)
Comparison in an Artificial Stock
Market
Environment:
• Traders (n context, n straight GP)
• 1 Market maker: prices and deals: 5 stocks
• Traders buy and sell shares at current
market price, but do not have to do so
• Traders have memories, can observe
actions of others, index, etc.
• Modelling ‘arms-race’
• Actions change environment
Bruce Edmonds, AAMAS 2002
(Trying it out)
Total Assets in a Typical Run
Total Value of Assets
30000
25000
20000
15000
10000
5000
0
0
100
200
300
400
Time
500
Black=context, White= non-context
Bruce Edmonds, AAMAS 2002
(Trying it out)
Total Assets of Context Traders – Total Assets of
Normal Traders, scaled by standard deviation of
assets (7 agents of each type, 9 runs)
2
0
-2
100
200
300
400
500
(Bold=average, Light= scaled difference for one run)
Bruce Edmonds, AAMAS 2002
(Trying it out)
Average for 10 runs with 3
traders of each type
2
0
-2
100
Bruce Edmonds, AAMAS 2002
200
300
400
500
(Trying it out)
Snapshot of model domains in
one trader
Volatility - past 5 periods
950
900
850
800
750
700
750
Bruce Edmonds, AAMAS 2002
850
950
Volume - past 5 periods
(Trying it out)
The model contents in snapshot
model-256 priceLastWeek [stock-4]
model-274 priceLastWeek [stock-5]
model-271 doneByLast [normTrader-5] [stock-4]
model-273 IDidLastTime [stock-2]
model-276 IDidLastTime [stock-5]
minus
[divide
model-399
[priceLastWeek [stock-2]]
[priceLastWeek [stock-5]]]
[times
[priceLastWeek [stock-4]]
[priceNow [stock-5]]]
Bruce Edmonds, AAMAS 2002
(Trying it out)
The Problems of Under- and OverDetermination
1. Under-determination
•
Neither  nor  can be inferred
Choose a more specific context
2. Over determination
•
Both  and  can be inferred
Choose a less specific context
Bruce Edmonds, AAMAS 2002
(Integrating Learning and Inference)
Universal learn and infer loop
repeat
learn and/or up update beliefs
deduce intentions, plans and actions
until finished
Bruce Edmonds, AAMAS 2002
(Integrating Learning and Inference)
Learn and infer loop using
context
repeat
repeat
recognise/learn/choose context, c
induce/update beliefs in c
deduce predictions/conclusions in c
until predictions are possible, consistent
and actions/plans can be determined
plan & act (starting from c)
until finished
Bruce Edmonds, AAMAS 2002
(Integrating Learning and Inference)
Recap - Clusters of
Domain&Content make a Context
M1
M1 M2
Abstract to a context
Bruce Edmonds, AAMAS 2002
(Integrating Learning and Inference)
Heuristics for Learning Context (I)
• Formation: if there is a cluster of similar
domains then create a context
• Abstraction: if contexts share models
with the same domain, abstract them to
super context
• Specialisation: If restricting the domain
allows more models, create a subcontext.
Bruce Edmonds, AAMAS 2002
(Integrating Learning and Inference)
Heuristics for Learning Context (II)
• Content Correction: If only a few models
are in error then remove or correct them
• Content Addition: If a model has the same
domain as an existing context, then add it
Bruce Edmonds, AAMAS 2002
(Integrating Learning and Inference)
Heuristics for Learning Context (III)
• Context Restriction: If most models are in
error, exclude that situation from context
• Context Expansion: If most models work in
new conditions, then expand the context
• Context Removal: If a context has few
models or a tiny domain, forget the context
Bruce Edmonds, AAMAS 2002
(Integrating Learning and Inference)
Bruce Edmonds
bruce.edmonds.name
Centre for Policy Modelling
cfpm.org
Context Home Page
www.context.umcs.maine.edu
Bruce Edmonds, AAMAS 2002
The End!