Transcript Summary of the research conducted during the first six months.

Centre of Research and Technology
Hellas
INFORMATICS & TELEMATICS INSTITUTE
Artificial Intelligence & Information Analysis Group (AIIA)
Profile
 The Informatics and Telematics Institute is a non-profit organization under the
auspices of the General Secretarial of Research and Technology of Greece. Since
March 10th, 2000 it is a founding member of the Centre of Research and
Technology Hellas (CERTH) also supervised by the Greek Secretariat of Research
and Technology.
 The Artificial Intelligence & Information Analysis Group within CERTH/ITI has been
active in 2-D/3-D image processing, human-centered interfaces, and multimedia
authentication for more than two decades. It is also affiliated with the Aristotle
University of Thessaloniki.
Profile
 Two faculty members, several postdoctoral fellows, and more than ten Ph.D.
students
 Participation in 17 European projects (IST, RTN/HCP, ESPRIT, ACTS, LTR/BRA,
RACE, TEMPUS, AIM) and 8 national projects
 Books: 6; One in 3-D Image Processing Algorithms
 Chapters contributed to edited books: 14
 Journal Papers: 111
 Conference Papers: 273
Role
Prior experience in topics related to the project:
3-D Image Processing & Graphics
Human-centered interfaces
 Face analysis; facial feature extraction; face tracking;
facial feature tracking; face expression recognition;
 Graphical communication: audio-visual speech analysis;
Role
 Research activities in synthesis tasks
 Development of talking heads (virtual salesmen) with
emphasis in
 Texture mapping;
 Synthesis of facial expressions; prototypes;
 Synthesis compliant with standards MPEG-4 FDPs, FACS, etc.
 Development activities: Contribution to the integration of
generic tools for graphic design, and speech recognition
/synthesis into the Worlds Studio Platform.
State of the art in
Facial Modeling and Animation
 2D and 3D morphing
 Manually corresponding features
 Combination of 2D morphing with 3D geometric transformations
 Physics based muscle modeling




Spring Mesh Muscle
Vector Muscle
Layered Spring Mesh Muscle
Finite Element Method
State of the art in
Facial Modeling and Animation
 Pseudo or Simulated Muscle
 Free Form Deformations
 Combination of 2D morphing with 3D geometric transformations
 Wires
 Model Fitting
 Adaptation to an example face
 MPEG-4 Compatible Models
 Facial Action Coding System
 Example-driven deformations of the model
 Speech driven heads
Face Detection using Color Information
 HSV thresholding based on facial colors
 Connected components labeling and analysis
 Segmentation of the image
 Moments computation
 Best fit ellipse image
 Spatial constraint for face contour
 Initialization of the snake
 Snake deformation by energy minimization
 Inner face contour image
HSV Thresholding & Connected
Components Labeling
 Thresholding based on facial colors (segmentation)
 Keep only the pixels having color similar to facial
texture
Initial image
Segmented image
HSV thresholding
Connected
components labeling
Moments Computation
 Ellipses in the segmented image
 Best fit ellipse
Segmented
image
Best Fit Ellipse image
Moments Computation
Model Superposition on Face Images
 ICP for superposition of the model on points already
defined on a face image
 Mass-Spring Models to fit the face model on the
face image
I(terative) C(losest) P(oint) algorithm
 ICP is based on the Closest Set of Points
 Closest Set of Points leads to the Quaternion
 Quaternion
 an easily handled vector
 the basis of the ICP transformations
 similar to the rotation and translation matrices
 Convergence
 Monotonically to the nearest local minimum
 rapid during the first few iterations
 globallity depends on the initial parameters
Mass-Spring Models
 FEM restricted models
 Simulates models as
masses connected with
springs
 Physics based simulation
four masses connected
among themselves
with uniform springs
Examples

Randomly initial
positioning of the face
model

Interactive definition of
points on the face image
Examples

Fit of the model by applying
the ICP algorithm

Fit of the model by applying
the Mass-Spring Model
Ellipse fitting
 Ellipse determination based on the model’s position
Model Superposition
Ellipse Image
Model Superposition
based on model’s contour
Combination of two methods
 Ellipse extracted using Moments Computation
 Ellipse extracted using the Model Fitting procedure
 Intermediate ellipse (scale)
Intermediate
Ellipse image
Intermediate
ellipse
Spatial Constraint for Face Contour
 Initialization of the snake
 Circular deformable model
Intermediate Ellipse
Initialized Snake
Spatial Constraint for
Face Contour
Snake Deformation & Face Contour
Detection
 Snake Deformation by Energy Minimization
 Snake Energy:
 Internal Energy:
 External Energy:
Etotal  Eint  Eext
Eint (i )  i  i 1  i  i 1  i 1  2i  i 1
2
Eext (i )   I ( x, y)  k  1 
2
2
d (i 1 ,i )
d (i ,i 1 )
Intermediate Ellipse
Deformated Snake
Snake Deformation
Future Work
 Statistical analysis/synthesis of the images & facial models
 Eigen-decomposition and PCA of the features
 Features correspondences of the images & the model
 Extraction of Face Definition Points (MPEG-4)
 Extraction of Facial Expressions