Transcript Automatic Face Feature Localization for Face Recognition

Automatic Face Feature
Localization for Face Recognition
Christopher I. Fallin
honors thesis defense: May 1, 2009
advisor: Dr. Patrick J. Flynn
May 1, 2009
Chris Fallin - thesis defense
1
Outline
• Face recognition
– Methods of evaluation
• Elastic Bunch Graph Matching
– Gabor Jets
– Bunch Graphs and Feature Localization
• My Contributions: Automatic Fiducial Points
– Information Content model
– Fiducial point placement
– results
Face Recognition
• Subfield of biometrics:
– life (bio)
– measure (metric)
– Extract identifying
information from
measures of human
traits
• Face recognition: digital
images of face
• 2D, 3D, infrared,
multimodal, …
http://www-users.cs.york.ac.uk/~nep/research/3Dface/tomh/3DFaces.jpg
May 1, 2009
Chris Fallin - thesis defense
3
Face Recognition Evaluation
Image Set
May 1, 2009
Chris Fallin - thesis defense
4
ROC curves
System Decision
Y
N
Actual
Y
N
http://en.wikipedia.org/wiki/File:Roc-general.png – used under terms of GNU FDL
May 1, 2009
Chris Fallin - thesis defense
5
EER = 11.1%
May 1, 2009
Chris Fallin - thesis defense
6
Rank-one Score
A
Probe
A
B
C
D
E
F
G
B
0.89
0.70
0.10
0.52
0.34
0.48
0.37
C
0.70
0.73
0.45
0.82
0.12
0.43
0.44
Gallery
D
E
0.10
0.52
0.45
0.82
0.92
0.89
0.89
0.56
0.23
0.20
0.82
0.38
0.13
0.14
Rank-one:
May 1, 2009
Chris Fallin - thesis defense
F
0.34
0.12
0.23
0.20
0.82
0.52
0.23
G
0.48
0.43
0.82
0.38
0.52
0.84
0.11
0.37
0.44
0.13
0.14
0.23
0.11
0.99
5/7 = 71.4%
7
Elastic Bunch Graph Matching (EBGM)
• Wiskott et al., USC/Bochum, mid-90s
• Basis of ZN-Face, successful commercial system
• We use Face Identity Evaluation System, from
Colorado State
• Face features represented by Gabor filter
responses
• Features are localized
– Fit an elastic graph onto the features by localization:
local optimization problems
May 1, 2009
Chris Fallin - thesis defense
8
A Face Graph
[Wiskott99]
May 1, 2009
Chris Fallin - thesis defense
9
Gabor Jets
• Vector of filter responses
to 40 Gabor kernels
– 5 wavelengths
– 8 orientations
– Each is complex-valued
• Gabor jets capture
information well: Gokberk
et al. get 91% rank-one
with fixed grid
– On FERET: 78.5% max, with
12 grid points
May 1, 2009
Chris Fallin - thesis defense
10
Bunch Graphs
• Each feature has a bunch
of canonical jets
• Represents typical
features
• Best-match at each
feature point for novel
images
[Wiskott99]
May 1, 2009
Chris Fallin - thesis defense
11
Feature Localization
• Initial alignment: eye locations known a-priori
• Overlay bunch graph with average edge
lengths
• Take Gabor jets; pick best match in each
bunch
• Localize based on displacement estimation
(local optimization problem)
May 1, 2009
Chris Fallin - thesis defense
12
The Idea: Automatic Fiducial Point
Placement
• Bunch graph training requires manual fiducial point
placement
– 70 images, 25 points
• Why not statistically determine optimal features to
match on?
• We can align/normalize all faces and take some
statistical measure at each point in “face space” to
determine goodness
• Replaces training step; back-end algorithm is identical
May 1, 2009
Chris Fallin - thesis defense
13
Related Work
• Gokberk et al.: choosing fiducial points with
genetic algorithms
– But their chosen points are global
– Same goal as our system, excluding prelocalization
• Salient Points
– Wavelet-based approach to image retrieval
– Choras et al., 2006: similar approach with
goodness function, but no EBGM
May 1, 2009
Chris Fallin - thesis defense
14
Information Content: Variance Model
• Compute goodness
function over facespace
• Inter-subject variance
over intra-subject
variance
• Self-normalizing
unitless measure
• Requires multiple
images per subject
May 1, 2009
Chris Fallin - thesis defense
15
Computing the goodness function
• FRGC: 5404 images – 700 MB, 128x128 grayscale
(7 GB before normalization)
• Each pixel: 12 seconds, on fast Athlon 64
• Split into 128 Condor jobs
– Each pixel is independent: easy
• Pre-normalize image set, dump to fast-loading
binary format (single file)
• Run Condor jobs: three hours
• Post-processing to reassemble results
May 1, 2009
Chris Fallin - thesis defense
16
Fiducial Point Placement
• Random placement with
probability density
• Compute gradient of
goodness function
• Probability is product of
gradient and goodness
• Place points sequentially,
decay probability around
points exponentially
• Mirror-point constraint:
mirror placements across
centerline, or snap to
center
May 1, 2009
Chris Fallin - thesis defense
17
Prelocalization: Pseudo-Bunches
• Displacement estimation requires canonical
feature jet from bunch
• We can’t provide this if we have no knowledge
of feature
• Solution: fake a jet bunch
– Make educated guess with K-means clustering on
jets from all images at given point
• Then, run displacement estimation to
prelocalize points on each image
May 1, 2009
Chris Fallin - thesis defense
18
Results
FERET
• Competitive with
original, manual points
– In both cases, automatic
training points yield only
~1% performance drop
– With no human training!
FRGC
• Prelocalization did not
work as intended
• Success without this
suggested by Gokberk’s
results
May 1, 2009
Chris Fallin - thesis defense
19
ROC curves
FERET
FRGC
(EER = 11.4%)
(orig: 11.1%)
May 1, 2009
(EER = 31.4%)
(orig: 34.8%)
Chris Fallin - thesis defense
20
Prelocalization: causes for failure
• Poor pseudo-bunch clustering: K-means often
found optimal clustering at self-imposed cap
of N/10 clusters
– Likely because initial jets are too far off
• Naïve localization: single-step
– Bolme thesis compares several optimization
algorithms
• Average displacement of 2.628 pixels: larger
than 2.021 pixels in manual points
May 1, 2009
Chris Fallin - thesis defense
21
Future Work
• More sophisticated prelocalization
• Look at pseudo-bunch statistics to determine
failure mode in more detail
• Look at per-fiducial point statistics to
determine where performance is weak
• Investigate: are manual pts a theoretical limit,
or can we exceed them?
• Try new image classes – test claim of
genericism
May 1, 2009
Chris Fallin - thesis defense
22
Questions?
• Email [email protected]
• Full thesis and source code will be posted
online: http://c1f.net/research/mark5/
May 1, 2009
Chris Fallin - thesis defense
23
Distance Metrics on Jets
• Phase-insensitive:
magnitude only
– Selects best jet in bunch
• Phase-sensitive
– Can solve for
displacement vector:
basis of localization
• Displacement
estimation
May 1, 2009
Chris Fallin - thesis defense
24