Transcript ppt file - Prof. Paul Mc Kevitt

Visual Semantics and Ontology of
Eventive Verbs
Minhua Eunice Ma and Paul Mc Kevitt
School of Computing and Intelligent Systems
Faculty of Engineering
University of Ulster, Magee
Derry/Londonderry, N. Ireland
Outline
 Background: CONFUCIUS
 Previous verb taxonomies
 Visual semantics & verb classes
 CONFUCIUS’ ontology of verbs
 Current status of implementation
 Relation to other work
 Conclusion & future work
IJCNLP-04
Sanya, China, 22 Mar 2004
Architecture of CONFUCIUS
Natural language sentences
Surface transformer
Media allocator
Prefabricated objects
(knowledge base)
Language knowledge
mapping
3D authoring tools,
existing 3D models &
character models
Visual knowledge
(3D graphic library)
LCS lexicon
Natural
WordNet
Language
Processing
Text To
Speech
Sound
effects
semantic
representations
visual
knowledge
Animation
engine
Synchronising & fusion
3D world with audio in VRML
IJCNLP-04
Sanya, China, 22 Mar 2004
NLP in CONFUCIUS
Pre-processing
Part-of-speech tagger
Connexor
FDG parser
Syntactic parser
Semantic
inference
WordNet
LCS database
Disambiguation
FEATURES
Morphological
parser
Coreference
resolution
Lexical
temporal relations
IJCNLP-04
Sanya, China, 22 Mar 2004
Temporal
reasoning
Post-lexical
temporal relations
Previous verb taxonomies
 Grammatical categorisation & valency
 Thematic roles (Fillmore, 1968; Jackendoff, 1990;
Halliday, 1985; Dowty, 1991)
 Aspectual classes (Vendler, 1967; Stede, 1996)
 Semantic verb classes in WordNet (Fellbaum,
1998)
 Levin’s (1993) verb classes
 Dimension of causation
& Lascarides, 1995)
IJCNLP-04
Sanya, China, 22 Mar 2004
(Asher
Grammatical categorisation & valency
 Subcategorisation description of verb
categories in LDOCE (Longman Dictionary
of Contemporary English)





D – ditransitive
I – intransitive
L – linking verb with complement
T1 – transitive verb with NP object
T3 – transitive verb with infinitival clause as
object
IJCNLP-04
Sanya, China, 22 Mar 2004
Grammatical categorisation & valency
 Subcategorisation description of verb
categories in LDOCE (Longman Dictionary
of Contemporary English)
 Syntactic valency

Obligatory valency fillers (complements)
e.g. subject, object

Optional valency fillers (adjuncts)
e.g. temporal, locational adjuncts
 Semantic valency (Leech, 1981)
IJCNLP-04
Sanya, China, 22 Mar 2004
Thematic roles
 Other names: theta-role, case role, deep
grammatical function, transitivity role,
valency role, case frame
 Extend syntactic analysis into semantic
domain to capture roles of participants
surface case (nominative, accusative)
surface function (subject, object)
Thematic roles (e.g. agent,
patient/theme, instrument,
source, goal, place)
 Classifying verbs based on thematic roles
(Dixon, 1991)
IJCNLP-04
Sanya, China, 22 Mar 2004
Aspectual (temporal) classification
Vendler’s (1967) verb classes







activities: run, swim, sleep, cry
statives: love, hate, know
achievements: arrive, win, find, die
accomplishments: build (a house), write (a book)
Stede’s (1996) MOOSE ontology
Formal ontologies DOLCE, SUMO, and CYC
assume traditional aspectual (temporal)
classification for events
IJCNLP-04
Sanya, China, 22 Mar 2004
Aspectual (temporal) classification


Vendler’s (1967) verb classes
Stede’s (1996) ontology of MOOSE
situation
state
activity
event
protracted moment culmination transition
activity
activity
protracted moment
culmination culmination

Formal ontologies DOLCE, SUMO, and CYC
assume traditional aspectual (temporal)
classification for events
IJCNLP-04
Sanya, China, 22 Mar 2004
Aspectual (temporal) classification


Vendler’s (1967) verb classes
Stede’s (1996) ontology of MOOSE
situation
state
activity
event
protracted moment culmination transition
activity
activity
protracted moment
culmination culmination

Formal ontologies DOLCE, SUMO, and CYC
assume traditional aspectual (temporal)
classification for events
IJCNLP-04
Sanya, China, 22 Mar 2004
Semantic verb classes in WordNet
Lexicographer file
Contents
verb.body
verb.change
verb.cognition
verb.communication
verb.competition
verb.consumption
verb.contact
verb.creation
verb.emotion
verb.motion
verb.perception
verb.possession
verb.social
verb.stative
verb.weather
grooming, dressing, bodily care
size, temperature change
thinking, judging, doubting
telling, asking, ordering, singing
fighting, athletic activities
eating and drinking
touching, hitting, tying, digging
sewing, baking, painting
feeling
walking, flying, swimming
seeing, hearing, feeling
buying, selling, owning
political/social activities
being, having, spatial relations
raining, snowing, thundering
IJCNLP-04
Sanya, China, 22 Mar 2004
 Taxonomic approach
based on pure lexical
semantics
 Reveal semantic
organisation of lexicon
in terms of lexical &
semantic relations
 Top nodes of
WordNet’s verb file
Levin’s (1993) verb classes
 Theoretic ground -- semantic/syntactic
correlations: verbs with similar meaning (identical
LCSs in terms of specific meaning components)
show same syntactic behaviors
Verbs of motion
Verbs of inherently directed motion: arrive, come, enter
Leave verbs: leave, abandon, desert
Manner of motion verbs: roll, run, sneak, waddle
Verbs of motion using a vehicle: bike, drive, fly
Chase verbs: chase, follow, track
Accompany verbs: accompany, escort, guide
Waltz verbs: clog, polka, waltz
IJCNLP-04
Sanya, China, 22 Mar 2004
Dimension of causation
 Asher and Lascarides’ (1995) dimension of
causation-change
 causation and change are specified along four
dimensions: locative, formal, matter, intentional
cause
locative
loc-cause fml-cause mtr-cause
sub of put sub of build sub of paint
formal
intent-cause
sub of amuse
matter
loc-change
obj of put
intentional
fml-change
obj of build
IJCNLP-04
Sanya, China, 22 Mar 2004
change
intent-change mtr-change
obj of amuse obj of paint
Visual semantics & verb classes
Visual factors concerning verb categorisation
 Visual valency
 Somatotopic factors in visualisation
 Level-of-detail of visual information
Verbs belonging to same class in
classification
the
 Visual “synonyms”
 Substitutable in same set of animation keyframes
Visualisation of action verbs is effective
evaluation of the classification
IJCNLP-04
Sanya, China, 22 Mar 2004
Visual valency
 Capacity of verb to take specific number and type of
visual arguments in language visualisation
(3D
animation)
 valency filler -- visual role
 2 types of visual roles requiring different processes in
visualisation


human (biped articulated animate entity)
object (inanimate entity)
 Visual valency overlaps with syntactic & semantic
valency
 Visual modality requires more obligatory roles than
surface grammar or semantics
IJCNLP-04
Sanya, China, 22 Mar 2004
Somatotopic effectors of action verbs
 Theoretical ground: execution/perception/visualisation of
action verbs produced by same somatotopic effector
activate same parts of cortex
 Distinguish facial expression (e.g. lip movement) & body
posture (arm/leg/torso) in our ontological system
facial expression – sing, laugh
Leg – run, kick
body posture
Arm – wave, put
torso – bow
 Further divisions like distinction between upper/lower
arm, hands, & even fingers are possible
IJCNLP-04
Sanya, China, 22 Mar 2004
Level-Of-Detail (LOD)
basic-level verbs & troponyms
EVENT
…
go
cause
event level verbs
…
walk
climb
limp stride trot swagger
Levels
run
jump
manner level verbs
jog romp skip bounce hop
Verbs
troponym level verbs
Basic-level verb for ...
event level
go, move
atomic object
manner level
walk, jump
human/non-atomic object
troponym level
limp, stride
human
IJCNLP-04
Sanya, China, 22 Mar 2004
CONFUCIUS’ verb taxonomy
2.2.1. Action verbs
2.2.1.1. One visual valency (the role is a human, (partial) movement)
2.2.1.1.1. Biped kinematics: arm actions (wave, scratch), leg actions (walk,
jump, kick), torso actions (bow), combined actions (climb)
2.2.1.1.2. Facial expressions & lip movement, e.g. laugh, fear, say, sing, order
2.2.1.2. Two visual valency (at least one role is human)
2.2.1.2.1. One human and one object (vt. or vi.+instrument)
e.g. throw, push, kick, open, eat, drink, bake, trolley
2.2.1.2.2. Two humans, e.g. fight, chase, guide
2.2.1.3. Visual valency ≥ 3 (at least one role is human)
2.2.1.3.1. Two humans and one object (inc. ditransitive verbs), e.g. give, show
2.2.1.3.2. One human and 2+ objects (vt. + object + implicit instr./goal/theme)
e.g. cut, write, butter, pocket, dig, cook
2.2.1.4. Verbs without distinct visualisation when out of context: verbs of trying,
helping, letting, creating/destroying
2.2.1.5. High level behaviours (routine events), political and social activities
e.g. interview, eat out (go to restaurant), go shopping
IJCNLP-04
Sanya, China, 22 Mar 2004
Text-to-Animation of single sentences
 Collision detection example (contact verbs: hit, collide, scratch, touch),
no human role involved
“The car collided with a wall.”

using ParallelGraphics’ VRML extension--object-to-object collision

non-speech sound effects
 H-Anim examples: action verbs
3 visual valency verbs
“John gave Nancy a loaf of bread.”
“John put a cup of coffee on the table.”

H-Anim Site node

locative tags of object (on_table tag for table object)
2 visual valency verbs
“John pushed the door.”
“John ate the bread.”
“Nancy sat on the chair.”
For more demos, please visit
http://www.infm.ulst.ac.uk/~eunice/3D_anim.html
IJCNLP-04
Sanya, China, 22 Mar 2004
Relation to other work
 Categorise verbs from visual semantics perspective
 Language visualisation in CONFUCIUS provides
independent criteria for identifying classes of verbs
sharing certain aspects of meaning,
i.e.
semantic/visual correlations
 Relation to Levin’s verb classes:
“Carol cut the whole wheat bread.”
“Whole wheat bread cuts easily.”
2.2.1.3.2, visual valency=3
2.1.2, visual valency=2
Verbs of cutting
1 to N
“Nancy brought the book to John.” Verbs of sending & carrying
2.2.1.3.1
“Nancy gave the book to John.”
Verbs of change of possession visual valency=3
N to 1
IJCNLP-04
Sanya, China, 22 Mar 2004
Conclusion & future work
 Categorise verbs from visual semantic perspective
 Provides independent criteria for identifying classes
of verbs based on semantic/visual correlations
 Visual semantic analysis of eventive verbs revealed
striking influences in taxonomic verb tree
 Various criteria ranging from visual valency,
somatotopic effectors, to LOD are proposed
 Evaluation issues using automatic animation
generation & psychological experiments
 Discourse level interpretation
IJCNLP-04
Sanya, China, 22 Mar 2004