Transcript NSF I/UCRC Workshop Stony Brook University

NSF I/UCRC Workshop
Dynamic Data Analysis Projects
in the Image Analysis and Motion Capture Labs
Non-rigid Surface Registration Using High-Order Graph Matching
Sparse and Locally Constant Gaussian Graphical Models
Goal: We want to explore the structure of
probabilistic relationships in massive spatiotemporal
datasets. We want to learn sparse Gaussian
graphical models, while enforcing spatial coherence
of the dependence and independence relationships.
Such learned structure permits efficient inference
but also gives insights into the nature of the data
3 Example Applications:
Local constancy:
Examples
Problem definition
Tracking subtle details:
In dynamic 3d data non-rigid registration is essential for 3d surface
tracking, expression analysis and transfer, dense motion capture
data processing etc.
Figure: local constancy in a 2D dataset (spatial neighborhood in
black dashed lines) - local constancy does not discourage long range
interactions
Strictly Concave Penalized Maximum Likelihood:
We perform maximum likelihood estimation with
sparseness and local constancy priors
log-likelihood of the dataset
sparseness
penalty
regularization parameters
Challenges: - original data are not registered in object space and
the points may have different motion vectors and velocities)
- The large size of the datasets (tens of thousands of 3d points per
frame) require accurate and efficient processing
The deformation error can be measured in the 2D domain using
conformal mapping and three correspondences, leading to a highorder graph matching problem
Expression transfer:
+
=
local
constancy
penalty
precision matrix for N variables
Figure: manually labeled walking
sequence. Right: leg/leg,
hand/leg interaction in red,
independent leg motion in blue
sample covariance matrix
Figure: cardiac MRI displacement
and the corresponding spatial
manifold
Current registration results:
discrete derivative operator for M
spatial neighborhood relationships
Results
1.1
* *
Relative log-likelihood
1
*
0.9
Statistical Shadow and Illumination Estimation for Real-World Images
0.8
0.7
Figure: functional brain MRI of a monetary reward task; left: 16
cocaine subjects, more connections in the cerebellum (green); right:
12 control subjects, more connections in the prefrontal cortex(red)
0.6
Synthetic
Indep
MB-and [1]
Cardiac
MRI
Walking
sequence
MB-or
CovSel [2]
Brain MRI
Cocaine
GLasso [3]
Brain MRI
Control
SLCGGM
Figure: cross-validated log-likelihood on the testing set
*not statistically significantly different from our method
Goal: Estimate the Illumination environment from a
Results
single image, with rough knowledge of the 3D geometry
and in the presence of texture
A novel cue for shading/shadow extraction
Illumination from Caltech 101 motorbike images,
using a common 3D model for the whole class:
Simultaneous Analysis of Facial Expression and EEG Data
Goal: Examine facial expressions related to drug craving
and drug addiction
Dataset: Videos of the facial expressions of subjects
and simultaneously captured EEG
(electroencephalogram) data
The subjects watch a series of images belonging in
several categories (happy, unpleasant, drugs, neutral)
Method:
-Facial expression features are tracked using an Active
Appearance Model (AAM)
- FACS (Facial Action Coding System) codes are
retrieved from the feature movement
An MRF model for robust illumination estimation
-Models the creation of
cast shadows in a
statistical framework
- Allows estimation of
the illumination from
real images, modeling
objects with bounding
boxes or general class
geometry
Applications: integration in scene
understanding, search in large
image databases, augmented reality
etc