Transcript Slide - People.csail.mit.edu

Real-Time
Enveloping with
Rotational
Regression
Robert Wang
Kari Pulli
Jovan Popović
Enveloped (skinned) characters are pervasive.
skeleton
mesh
Skeletons are often used to control meshes.
Physically based modeling provides
realistic deformations.
 Realistic deformations
– Finite-element based
[Teran et al. 2005]
– Anatomy based
[Scheepers et al. 1997]
– Elastically deformable
[Capell et al. 2002, 2005]
– Used in movie production
– Off-the-shelf commercial tools
 Slow evaluation
[Teran et al. 2005]
[Absolute Character Tools 1.6]
We learn a fast model from exported examples.
Black Box
Simulation
Exported Examples
(skeleton-mesh pairs) Our method
Fast Model
Artists can still use existing modeling
tools or scanned data.
3-D Scan Data
Exported Examples
(skeleton-mesh pairs)
Fast Model
This is analogous to mesh simplification.
mesh
simplification
learning
Physical simulation
High-resolution
mesh
Rotational
Regressionmesh
Enveloping
Low-resolution
 Higher quality
 Faster to render
 Used in movie production
 Optimized for interactive
applications
How do we map a skeleton to a mesh?
?
What parameters should we learn?
How to model muscle deformations for fast evaluation?
Linear blend skinning linearly maps joint
rotations to vertex positions.
 Most popular enveloping
technique for games
+
 Coarse modeling
y
parameters (but very
simple)
 Not very expressive (but
very fast)
y
Figure from [Wang and Phillips 2002]
Linear blend skinning has many names.
 Also known as,
– Single-Weight Enveloping
– Skeletal Subspace Deformation (SSD)
– Or just, “Skinning”
 We will use “Linear Blend Skinning” or
“SSD.”
The two steps of our work are deformation
gradients prediction and mesh reconstruction.
Deformation
gradients
prediction
(Rotational
Regression)
Mesh
reconstruction
We present a replacement for linear
blend skinning.
+
 Coarse modeling
parameters.
 Can’t handle certain
 Lets you use your
existing modeling tool.
 Good for muscle bulges.
types of deformations.
 Fast
 Fast
Whenever you have an existing model, you
should use our technique instead of linear blend
skinning.
Our model is inspired by the behavior of
a flexing bicep.
Surface
rotation
Bone
rotation
 Rigid components move with the bone rotation
 Other surfaces rotate in the opposite direction
Angle is scaled by u.
Axis is offset by rotation W.
source
rotation
(bone)
target
rotation
(surface)
We map a sequence of bone rotations to
a sequence of surface rotations.
…
…
source
rotation
sequence
(bone)
target
rotation
sequence
(surface)
We fit parameters u and W by regression.
Surface
rotations
u’,W’
Best-fit
parameters
Skeleton
rotations
Rotational regression is good at
capturing muscle bulges.
Mesh reconstruction stitches
deformation gradients together.
Deformation
gradients
prediction
Mesh
reconstruction
Mesh reconstruction solved with
least-squares.
vertex
positions
deformation
gradients
Least –squares
 Least-squares problem
equivalent to linear system.
 Computation is matrix-
C
multiplication.
D(q)
Near-rigid vertices help eliminate
low-frequency errors at extremities.
 Low-frequency errors can accumulate at extremities of
mesh
 We fix a set of near-rigid vertices to their SSD predictions
 Still a least squares problem
We build upon existing mesh
reconstruction work.
 Mesh IK
[Sumner et al. 2005], [Der et al. 2006]
 SCAPE
[Anguelov et al. 2005]
 Similar formulation, faster evaluation.
[Anguelov et al. 2005]
Here’s a review of what we’ve covered.
Rotational
Regression
Deformation
Gradients
Prediction
C
Least-squares
problem
Mesh
Reconstruction
Dk(q)
Model reduction lowers the
dimensionality of problem.
C’

C
Dk(q)
 Large multiplication
on CPU

Dl(q)
SSD
 Smaller multiplications on
GPU
Model reduction uses greedy clustering.
 Vertices clustered into proxy-bones.
 Per-triangle deformation gradients clustered into “key”
deformation gradients.
P = 450
225
110
50
25
Mesh reconstruction reduced to the
following matrix-multiplications.
All on GPU:
 Computation in
C’
fragment
program

SSD weights
Map from “key”
deformation
gradients to
proxy-bones
Dl(q)
“key”
deformation
gradients
Skinning Mesh Animations is a an
alternative approach to model reduction.
 The method from Skinning Mesh Animations uses
mean-shift clustering and is more robust to errors.
[James and Twigg 2005]
 Our method minimizes vertex error and is faster
Deformation gradients prediction
is now on “key” deformation
sequences.
 Fewer deformation gradient sequences to predict rotational
regression.
Mesh reconstruction step reduced
to matrix-multiplications on GPU.
 Smaller matrixC’
Dl(q)
multiplications
 Supported on

graphics hardware
Our Technical Contributions:
Rotational Regression
Accurate and GPU-Ready
Poisson Reconstruction
Model Reduction
Results
Results
Results
Our work approximates the training examples
better than SSD and also generalizes well.
Our model is suitable for interactive
techniques.
 Evaluation speed within a factor of two of SSD
 Off-line training preprocess is usually less than half an
hour
How does our work fit with previous
work?
Our work is complementary to
displacement correcting techniques.
Figure from [Kry et al. 2001]
 Previous work provide corrective displacements.
– Pose space deformation
[Lewis et al. 2000],
– Shape by example
[Sloan et al. 2001],
– Eigenskin
[Kry et al. 2002]
 Our work provides better approximation of rotations.
 Our work complements approaches that build upon SSD.
Displacement correcting approaches fail
when SSD is very wrong.
Our work builds upon previous ideas on
enriching the SSD model.
 Multi-weight enveloping
[Wang and Phillips 2002]
 Additional joints
[Mohr and Gleicher 2003]
 Our technique has more parameters than SSD and generalizes the
additional-bones model.
A more expressive model is useful here.
Our model doesn’t do a perfect job.
 Not perfect reproduction
– Inspired by muscle bulging and twisting. Other
behaviors empirically validated.
– Displacement correcting technique can be used for
exact reproduction of examples.
Conclusion:
Fast and accurate enveloping.
 Fast evaluation of physical simulations through
learning.
– Within a factor of two of SSD on most models
 Accurate reproduction of details
– Better approximation and generalization
– Complementary to previous work
 A replacement for linear blend skinning
Acknowledgements
 Funding
 Data
– Nokia Research Center
– Drago Anguelov
– National Science Foundation
– Joel Anderson
– Pixar Animation Studios
– Michael Comet, Comet Digital, LLC
 Hardware/Software
– NVIDIA Corporation
– Autodesk
– Mark Snoswell, CG Character
– Joseph Teran, Ron Fedkiw
 MIT Graphics Group
– Ilya Baran
– Jiawen Chen
– Sylvain Paris
Questions?
 Thank you for coming to our talk!
Learning tasks trade
expressiveness and simplicity.
Simpler:
More Expressive:
Easier to fit
Captures more
types of
deformation.
Fewer training
examples needed.
Less likely to
overfit.
Rotational
Regression
Linear blend skinning (SSD) is a rough and
ready map from joint rotation matrices to
vertex positions.
 Most popular enveloping technique for games
 Coarse modeling parameters (but very simple)
 Not expressive enough (but very fast)
desired
deformation
SSD
deformation
Model Reduction
 True optimization not as tractable
difficult to solve discrete
simultaneously optimization
 We approximate it with a greedy algorithm inspired
by mesh simplification.
Our work builds upon previous ideas on
 Additional joints
[Mohr and Gleicher 2003]
 Multi-weight enveloping
[Wang and Phillips 2002]
 Our technique generalize the additionalbones model
 We evaluate cross-validation error to
test for overfitting
[Wang and Phillips 2002]
Rotational regression is good at
capturing muscle bulges.