Transcript presentation source

Agent
animation:
capabilities,
issues, and
trends
Paolo Petta
Austrian Research Institute for
Artificial Intelligence, Vienna
Introduction

Computer animation developments
 Geometry
 Resolution,
 Model-driven
detail
dynamics
 Ambient
physics modeling,
Behavioural modeling
 Control
 Interactivity,
communication
techniques, autonomy, learning
 Population
 Multiple
actors, distributed systems
Typical Applications


Synthetic characters,
virtual Humans,
visualisation/simulation
Design choices
 “Sparse”
top-down models vs.
“complete” bottom-up models

Application requirements
 deep-and-narrow
vs.
 broad-and-shallow
Research topics
Artificial
Intelligence
Robotics
User
Interface
User interface
for
Emotion control
Animation
Actor behaviour
emotion control
Behavioural
animation
Spatial
relationships
shape
transformation
Collision
detection
Geometric Facial
animation
Modelling
Face
design
Image
Synthesis
Skin texture
Visionbased
animation
Path
planning
Walking
models
Object
grasping
Kinematics
Dynamics
Cloth
animation
Muscle Collision
models responses
Finite-element
deformations
Hair
Physics
IMPROV (MRL, NYU)

Artistic and commercial applications
 Animated
staging
 Choreography
 Interactive multi-user environments
 ...


Surface model of mood&emotions
Productivity tool
 API for “laypersons”
(educators, historians, social scientists)
IMPROV

Microlevel:
 Procedural
 Accurate
animation
modeling of single actions
and all permissible transitions
 Statistically controlled parameter
randomization for variability and
consistency
IMPROV

Microlevel:
 Behavioural
 Scripts
layering
are classified in a hierarchy
according to level of behaviour
 User-defined connections between
layers define the effective heterarchy
 Action selection:
deterministic linear scripts or
stochastic selection from alternatives
 Exclusion of pursuit of conflicting
goals at same level
 Parallelism across the hierarchy
IMPROV

Macrolevel:
 Blackboard
architecture
Characters (attributes + scripts)
Stage
Manager
Avatars
Story agent („director“)
IMPROV

Macrolevel:
 Behaviour
layers spanning across
groups of agents for
coordinated action
 Distributed environment modeling:
“Inverse Causality” (=> MOO)
 information
about interactions is
attached to objects
 characters are “contaminated” by
use (new/update of state variables:
competence learning)
Edge of Intention (Oz, CMU)


Interactive drama
Believable autonomous characters
 Goal-directed
 Emotional
(folk theory of emotions, OCC)
 Simple appearance, emphasis on
behaviours
(-> internal processing)

Interaction modes
 Moving/gesturing,
“talking” (typing)
TOK architecture

Microlevel
 Hap
 Goal-oriented
reactive action engine
 Static plan library
• Action behaviours
• Emotion behaviours
• Sensing behaviours
 Sensing
of low-level actions of other
Woggles
 Action blending
TOK architecture

Microlevel
 Em
 Model
of emotional and social
aspects
 Explicit state variables for beliefs and
standards of performance
 Variables are influenced by
comparison of current goal states with
events and perceived actions
(thresholding)
TOK architecture

Microlevel
 Behavioural
 Mapping
features
of emotional state to overt
behaviour
 Manifestation of “personality”
 Tight integration of Hap and Em
 No need for arbitration
TOK architecture
standards
attitudes
emotions
Em
sense
language
queries
behaviour features
and raw emotions
goal successes,
failures & creation
sensory routines and
integrated sense model
The world
goals
behaviours
Hap
sense
language
queries
TOK architecture

Macrolevel:
 Fixed
plan library encodes all
possible communications/interactions
ALIVE (MIT Media Lab)

Entertainment
 Magic
mirror metaphore
 Unincumbered immersive
environment
ALIVE

Microlevel:
 Hamsterdam
 Behaviour
•
•
•
•
•
system for action selection
Based on ethological model
Sensory inputs via release mechanism
Loose hierarchy of behaviour groups
“Avalanche effect” for persistent selection
Inhibited behaviours can issue secondary
and meta commands
 Motor
skills layer for coordination of
motions
 Geometry layer for animation
rendering
ALIVE
External World
World
Sensory
System
Releasing
Mechanism
Goals/Motivations
Internal
Variable
Behaviour
Level
of Interest
Internal
Variable
Inhibition
Motor Commands
ALIVE
ALIVE

Levels of control:
 Motivations
via variables of single
behaviours
 “You
are hungry”
 Directions
 “Go
via motor skills
to that tree”
 Tasks
via sensory, release, and
behaviour systems
 “Wag
your tail”
ALIVE

Increased situatedness
 Synthetic
vision
 For
navigation
 Generic interface
 Plasticity:
 reinforcement
learning (conditioning)
ALIVE

Macrolevel:
 Totally
distributed control
Virtual Humans (Miralab/EPFL)

Goal
 Simulation
of existing people
 Real-time animation of virtual humans
that are realistic and recognizable
 Inclusion of synthetic sensing
capabilities allows simulation of
(seemingly) complex capabilities,
e.g. real-time tennis
Virtual Humans

Issues requiring compromising
 Surface
modeling
 Deformation
 Skeletal animation
 Locomotion
 Grasping
 Facial animation
 Shadows
 Clothes
 Skin
 Hair
Virtual Humans

Methodology
 Modeling:
 Prototype-based
 Head
and hand sculpting
 Layered body definition:
Skeleton, Volume, Skin
 Animation:
 Skeleton
motion
captured, play-back, computed
 Body deformation
for realistic rendering of joints
 Detailled hand and facial animation
Virtual Humans



Synthetic sensing as a main
information channel between virtual
environment and digital actor
(since ca. 1990)
Synthetic audition, vision and tactile
Differs fundamentally from robotic
sensing:
direct access to semantic information
Virtual Humans

Example: synthetic vision
 Environment
is perceived from a fieldof-view that is rendered from the
actor’s point of view
 Access to pixel attributes:
color, distance,
index to semantic information
 Simple
case: color coding of objects
=> perception of color = recognition of
object
 Object attributes are
retrieved directly from the simulation
Virtual Humans

Navigation:
 Path
planning & obstace avoidance
 Global navigation:
 Based
on prelearned model
 Determines the global navigation goal
 Local
navigation
 Purely
indexical, based on sensing
=> No need for model of environment
=> No need for current position
 Three

modules:
synthetic vision, controller, performer
Virtual Humans

Navigation controller:
 Regularly
invokes vision to retrieve
updated state of environment
 Creates temporary local goals if an
obstacle “up front”
 Local goals are determined by
obstacle-specific Displacement local
automata
Virtual Humans

Interaction with the environment:
Smart Objects
 Each
modeled object includes
detailled solutions for each possible
interaction with the object
 Objects are modeled according to
situated decomposition
Virtual Humans

Smart Objects include:
 Description
of moving parts, physical
properties, semantic index
(purpose and design intent)
 Information for each possible
interaction: position of interaction
part, position and gesture information
for the actor (capacity limits!)
 Object behaviours with state
variables (=> actor state info)
 Triggered agent behaviours
Virtual Humans

Example: virtual tennis
 Actor
model based on stack machine
of state automata
 Actor state can change according to
currently active automaton and
sensorial input
Virtual Humans
Architecture
of behaviour
control
Virtual Humans
Tennis
game
automata
sequence
JACK (UPenn)

Ergonomic environment analysis
 Workplace
assessment
 Product evaluation
 Device interfaces
 Logistics
JACK

Microlevel:
 Biomechanically
correct model
 Synthetic sensors for high-level
behaviours
 Three-level architecture realising
“truly situated” low-level behaviour
JACK

Microlevel
PaT-Net
object-specific and generic
symbolic reasoning capabilites
control
systems
stimulus
perceptual
modules
motor
behaviours
(learned sense-control-act loop parameters)
response
JACK

Macrolevel
 Taskable
virtual agent
 Global intentions and expectations of
all characters are statically captured
(explicitly anticipated)
 Parallel Transition networks
JACK

Macrolevel: PaT Net
Topics for Discussion


“Completeness” of modeling
“True” agent characteristics
(Wooldridge&Jennings)
 Autonomy
 Social
abilities
 Reactivity
 Pro-activeness
Topics for Discussion




The “TLA Debate”
Situatedness/synthetic sensing
Variability/adaptiveness/plasticity
Believability
Modelling completeness

“Sparse” models
 Abstract,
“top down”
 Based on explicit, reified design
elements
 Bridging/obviating of full detail by
careful selection of modeled
elements
 Broader coverage at differing
resolution
 Believability/impression over fidelity
 (Bound to) Lose in the long run?
Modelling completeness

“Complete” models
 Situated,
“bottom up”
 Depend on balanced design
(including environment&coupling)
 Limited coverage/complexity
 Allow for flexible action-selection
 Fidelity over believability/impression
 Win in the long run?
Autonomy (McFarland/Boesser)
 Automaton:
state-dependent behaviour
 Autonomous agent:
self-controlling, motivated
 Motivation:
reversable internal processes that are
responsible for changes in behaviour
 Multiple goals/actions are the rule!
=> concurrency, transitioning

Insights on own skills&conditions of
applicability
Social abilities

“Deep” agent modeling
 Of
the self: BDI and variants
 Of others (recursively)
 Of the society


Coordination
Communication
 Generation&understanding
of facial
expressions, postures, gestures, task
execution, text/speech,…
 (social) Emotions
(including display rules)
Social abilities

From Action Selection to Action
expression
 Sign
management: contextdependent behaviour sematics
 What should an agent do at any point
in order to best communicate its
goals and activities?
 Goal: increase comprehensibility of
behaviour
Believability


Quality vs. correctness
Self-motivation
 pursuit
of multiple simultaneous goals
 => entails requirement of broad
capabilities



Personality/Emotion
Plasticity/change over time
Situatedness
 social
skills
 affordances
And then...




Methodologies for assembly of
architectures with
understandable/predicatable
(motivated, goal-directed,…)
behaviour
Agent control systems
Persistency, plasticity
Agent animation as simulation