Transcript Steels96 - University of Washington

Emergent Adaptive Lexicons
Luc Steels 1996
Artificial Intelligence Laboratory
Vrije Universiteit Brussel
Presented by Achim Ruopp
University of Washington
Agenda
• Introduction to Emergence and SelfOrganization
• Steel’s Experiments on the emergence of
the lexicon
• Discussion
Excursion: Emergence and
Self-Organization
• A system exhibits
emergence when
– There are coherent
emergents at the
macro-level
– That dynamically arise
from interactions
between parts a the
micro-level
– Emergents are novel
w.r.t. individual parts of
the system
Excursion: Emergence and
Self-Organization
• Definition
– A dynamical and adaptive process
– Where systems acquire and maintain structure themselves
– Without external control
• Combining emergence and self-organization
Origins of Language
• Still unknown
• Chomsky’s hypothesis
– Species-specific innate language ability
– Refinement by parameter setting process
– Some support by experimental simulation via neural
networks
• Alternative hypothesis
– Language as an emergent phenomenon
• As a mass-phenomenon
• Spontaneously forming/becoming more complex
Steel’s Experiments
• Motivated by symbol grounding problem
• Experiments on robotic and software
agents
– Grounded meaning creation
– Lexicon formation ← Focus of paper
– Syntax
– Emergent phonology
Experimental Model Definitions
Features
• Set of agents A
• ∀a ∈ A ∃ set of features Fa = {f0,…,fn}
• A feature fi consists of attribute-value pairs
– Examples: (weight light) (size tall)
• Distinctive feature set
– Subset of Fa distiguishing agent a from all agents in a
group B
• Filtered subset CK,M = {a|K ⊂ Fa}
– K is a set of features
– M is a set of agents
Experimental Model Definitions
Lexicon
• Word
– sequence of letters drawn from a shared alphabet
• Utterance
– Set of words
– Word order does not play a role
• Lexicon L
– A relation between feature sets and words
– A single word can have several associated feature
sets
– A feature set can have several associated words
Experimental Model Definitions
Lexicon
• Each agent a has lexicon La
– Initially empty
• Feature set of a word: Fw,L
• Cover functions
– cover(F,L): set of utterances U: ∀u ∈ U:
{f|f ∈ Fw,Land w ∈ u}
– uncover(u,L): set of features F:
F = {f|f ∈ Fw,Land w ∈ u}
Coherence through
Self-Organization
• Agents can
– Create new words and associate them with a feature
set
– Form new associations between a word and a feature
set
• Key to self-organized coherence of the lexicon
– Agents participate in communication
– Record the success of particular word-meaning pairs
– Agents (re-)use words that led to high communication
success in the past
Language Game
• Dialog between two agents – Speaker and Hearer
• Dialog topic
– Other Agent
– Chosen by extra-linguistic means (“pointing”)
• Speaker and hearer identify possible distinctive feature
sets of topic
• Speaker
– Selects distinctive feature set
– Translates to words using cover function
• Hearer
– Interprets utterance using uncover function
– Compares interpretation to expectation
– Uses game to
• Learn part of the language
• Check if right meaning is associated with the right words
Language Game
Possible Outcomes
1. No differentiation possible
2. Speaker does not have a word
– May create new word
3. Hearer does not have a word
– Can associate word
– Cannot disambiguate when multiple
distinctive feature sets
Language Game
Possible Outcomes
4. Speaker and hearer know word
•
Meanings are compatible with situation
•
•
Sense-ambiguity possible
Meanings not compatible with situation
•
No communicative success
Experimental Results
One-word Utterances
• Typical experimental setup (5 agents, 10 meanings,
4000 language games)
• Leads to communicative success soon
Average
communicative
success
Number of language games
(scale 1/20)
Experimental Results
One-word Utterances
• Single meanings soon converge on one word
form (10 agents, 5 possible words, 1 meaning)
Average
communicative
success
Time
Experimental Results
Multiple Word Utterances
• In case the distinctive feature set of the
topic contains multiple features
• Can be used by hearer to “guess”
meaning of unknown words
Conclusions
• Self-organization is effective mechanism
for achieving coherence
• Side-effects of lexicon formation
– Synonymy
– Ambiguity
– Multiple-word sentences
Discussion
• Supports the notion that absolute synonymy does not
exist
– “After about 4000 language games the lexicon stabilizes as all
distinctions that need to be made have been lexicalized”
• Are the presented “linguistic” results an artifact of the
experimental setup? I.e. in how far does this experiment
reflect the real world?
– E.g. multi-word sentences
• Can the results just be explained by basic
communication theory?
• Language as an emergent phenomenon
f
– Zipf’s law regarding multiple meanings m 
– Self-organized criticality/highly optimized tolerance
References
Brighton, H., Selina, H.; Introducing Artificial Intelligence; 2003; Icon
Books Ltd.; ISBN 1-84046-463-1
De Wolf, Tom; Holvoet, Tom; Emergence Versus Self-Organisation:
Different Concepts but Promising When Combined; 2005; In
Engineering Self Organising Systems: Methodologies and
Applications, Lecture Notes in Computer Science, volume 3464, pp
1-15
Steels, L.; Emergent Adaptive Lexicons; 1996; In: Maes, P. and
Mataric, M.J. and Meyer, J.-A. and Pollack, J. and Wilson, S.W.
(eds) From Animals To Animats 4: Proceedings of the Forth
International Conference on Simulation of Adaptive Behavior,
SAB'96, Complex Adaptive Systems, pp. 562-567, Cambridge, MA:
The MIT Press
Zipf, G. K.; The Meaning-Frequency Relationship of Words; Journal of
General Psychology 33, 251–256 (1945).