Transcript 3D Imaging Core

Facial Imaging Core
Presenter: Elizabeth Moore
PI: Tatiana Foround
Statistician: Leah Flury
Completed Tasks

Software



Custom plug-in for Rapidform redesigned
to include all sets of measurements and a
final xml file to be uploaded to the Central
Data Repository
Facial Imaging Protocol
Camera Calibration Protocol
Camera Calibration Protocol



Determine the degree (if any) of intercamera variability (or error)
Monitor the performance of each
camera over time
Method had to evaluate 3 measurement
dimensions reliably and be easy to
perform in the field
Calibration: Preliminary Results

Goals:


To demonstrate that the objects being
scanned do not differ by measurement
To establish that reliable measurements
can be taken from the scanned image
Calibration: Preliminary Results


Mandibles were
selected as the
object
Four landmarks
were identified and
marked with a
permanent marker
Inter Dentale
Rt Ramus
Lt. Ramus
Menton
Calibration: Preliminary Results



Measurement error averaged less than
1 mm for both direct and indirect
measurement methods
Difference between estimated true
dimensions of the mandibles was less
than the measurement error
The mandibles are measurably the
same and reliably imaged and
measured
Camera Calibration Plan



Mandibles will be sent to sites currently
using the cameras
In the future a mandible and calibration
protocol will be included with the
camera
Protocol requires the mandible to be reimaged and the image sent to us every
time the camera is set-up
3-D Camera

Finland since August 2004



Buffalo since March 2004


Received 82 images 3 weeks ago
Total: 121 subjects scanned
Total: 93 subjects scanned
San Diego since April 2005

Total: 33 subjects scanned
Total Data


Subjects: 247
Status






Sex




FAS: 83 (33.6%)
Deferred: 29 (11.7%)
Control: 57 (23.1%)
Other: 14 (5.7%)
Unknown: 64 (25.9%)
Male: 87 (35.2%)
Female: 105 (42.5%)
Unknown: 55 (22.3%)
Race



Caucasian: 201 (81.4%)
African American: 30 (12.1%)
Unknown: 16 (6.5%)
Data Analysis


Sample 46
Status





Sex


FAS: 19 (O41.3%)
Control: 22 (47.8%)
Deferred: 3 (6.5%)
Other:2 (4.3%)
Male: 22 (47.8%), Female: 24 (52.2)
Race

Caucasian: 38 (82.6%), African American: 7 (15.2%),
Other: 1 (2.2)

Age mean: 10.7 years (3.7)

16 measurements used in analyses
Data Descriptives

FAS (n=19)

Control (n=22)

Male: 10 (52.6%)

Male: 10 (45.5%)

Caucasian: 12 (63.2%)

Caucasian: 21 (95.5%)

Mean age: 10.7 (4.0)

Mean age: 10.7 (4.0)
nt
Bi
zy
Br
Br
Measurement
lF
a
e
Ht
Ht
Lg
th
ce
Ea
r
To
ta
*
wF
ac
*
th
60
Lg
*
Lo
m
40
Lg
th
h
h
160
Ph
iltr
u
al
Br
Lg
t
Na
s
al
h
*
Dp
t
Dp
t
h
*
Na
s
e
e
wF
ac
id
Fa
c
Dp
t
*
Lo
M
ac
e
80
Br
120
Bi
g
*
Up
F
In
100
Bi
tr
in
Fr
o
140
Br
ne
rC
an
Br
O
ut
er
Ca
n
Br
Pa
lp
Fi
s
M
Mean
Summary Statistics
* = p < 0.05
all p values: corrected alpha = 0.003
controls
FAS
*
*
*
20
0
Linear Discriminant Analysis

Reason:


Models:




Formulate a rule based on the joint distribution of measurements
to classify future observations. Linear rule used due to small n
Using Principal Components based on measurements
Using subset of measurements, selected via stepwise function in
SAS
Age was added as a variable in models above
Evaluation of Model: Based on error rate as measured by the
jacknife, or cross-validation method.



Discrimination function is estimated using n-1 observations
The nth (omitted) observation is then classified using estimated
function
Repeat for all observations and count # misclassified observations
Error Rates Using Jackknife Method
35
% misclassification
30
subset (stepwise)
25
PC1 - PC2
subset, with age
20
PC1 - PC2, with age
15
10
5
0
control
FAS
status
Clinical Measurements

Head Circumference (OFC)

Maxillary Arc

Mandibular Arc

Philtrum Length



Direct Mean: 1.54 (0.21); Indirect: 1.46 (0.22)
Corrleation: r = 0.64; p < 0.001
Palpebral Fissure Length


Direct Mean: 2.54 (0.22); Indirect: 2.59 (0.22)
Correlation: r = 0.76; p = 0.0001
Error Rates Using Jackknife Method
35
subset (stepwise)
PC1 - PC2
30
% misclassification
subset, with age
25
PC1 - PC2, with age
20
add OFC
15
add OFC, age
10
5
add OFC, MAX, MAND
arcs
0
control
FAS
status
add OFC, MAX, MAND
arcs, age
Error Rates Using Jackknife Method
subset (stepwise)
PC1 - PC2
35
subset, with age
% misclassification
30
PC1 - PC2, with age
25
add OFC
20
add OFC, age
add OFC, MAX, MAND
arcs
add OFC, MAX, MAND
arcs, age
OFC only
15
10
OFC, age
5
OFC, MAX, MAND only
0
control
FAS
status
OFC, MAX, MAND, age
Discussion

Similar to previous findings

Measurements used in stepwise analysis



Image measurements: MF, BT, BZ, BG, PF, OC,
UFD, MFD, NBL, NL
Direct measurements: MF, BG, HC, PF, MFD, MX
Good correlation between direct clinical
measurements and indirect measurements
Conclusion



Results from this small sample is
promising
Results similar to findings from study
using direct anthropometry
Combination of indirect and direct
measurements results in highest correct
classification of subjects
Future Plans

Complete measuring of images

Further analysis as sample increases


Incorporate more dysmorphology and
demographic data into analysis
Camera to South Africa in September
Needs

Central repository capability

Dysmorphology data

Race/Ethnicity data

Control subjects

More racial/ethnic diversity
Facial Recognition
Shiaofen Fang, Jeffery Huang
Features Computation: Euclidean
Distances and Geodesic distance
Euclidean distances between landmarks
Geodesic distance (curved distance)
Feature Computation: Curvatures on points
Mean curvature, Gaussian curvature, Principal curvatures and their directions
Feature Computation: Moments

Moment features capture global shape information about the image.
The ( p  q) order moments of a density distribution function
are defined in terms of Riemann integrals as
th

m pq  



 

x p y q  ( x, y)dxdy,
 ( x, y )
p, q  0,1,2,.
The central moments  pq are translate invariant and are defined as
 pq  


p
q
(
x

x
)
(
y

y
)
 ( x, y)dxdy where x  m10 / m00 , y  m01 / m00
 

Seven rotation invariant moments are used:
 20  02 ,
(  20   02 ) 2  411 ,
2
( 30  312 ) 2  (3 21  03 ) 2 ,
( 30  12 ) 2  (  21   03 ) 2 ,






( 30  312 )( 30  12 ) ( 30  12 ) 2  3(  21   03 ) 2  (3 21  03 )(  21  03 ) 3( 30  12 ) 2  (  21  03 ) 2 ,


(  20   02 ) ( 30  12 ) 2  (  21   03 ) 2  411 ( 30  12 )(  21  03 ),


(3 21  03 )( 30  12 ) ( 30  12 ) 2  3(  21  03 ) 2  ( 30  312 )(  21  03 ) 3( 30  12 ) 2  (  21  03 ) 2 .
Feature Computation: Flatness of A Region


Flatness measures how
flat a certain region is. A
region can be defined by
points and shortest paths
between them.
Flatness can be computed
by fitting a planar surface
to the defined region
using a least square
fitting.
Experimenting with Visual Data Mining



Difference maps:




Visual data mining: using visualization to assist the data mining and data
analysis process.
Visual dta mining for FAS problem: generate a visualization of cumulative
“difference” between the FAS group and the controlled group. The
visualization may offer visual clues about potential features that
characterize FAS faces.
1st order difference map: difference in positions by
drawing the cumulative displacement vectors
between FAS group and the controlled group (after
face alignment).
2nd order difference map: difference in orientation
by drawing the cumulative difference of surface
normal vectors.
3rd order difference map: difference in curvatures.
On-going work. No meaningful results yet.
Ellipse Cut of the Frontal Face

To define a common facial region for global
measurements (e.g. moment computation)

Four points are used to define the ellipse region
Region Around Philtrum
Three points are used to define the region around philtrum
Data and Features Used
For Training and Testing

Experimental Data


Total: 65 subjects, FAS: 44, Control: 21
Final Training and Testing Features (49)








Mean curvature of the center of the philtrum (1)
Flatness of left cheek (1)
Length of philtrum, length of nose, and their ratio (3)
Palpebral fissure length (1)
Biocular breadth (1)
Depth moment features of the Ellipsis cut (14)
Depth moment features of the philtrum (14)
Curvature moment features of the philtrum (14)
Correlation-based Feature Selection
Using Best First Search

The Goal of Feature Selection


Heuristic Search Strategy


Decide which of the initial features to include in the final
subset
The Best First Search is applied to search the feature subset
space
Evaluation of “goodness” of feature subsets

The correlation-based evaluation function takes
into account the usefulness of individual features
for predicting the class label along with the level
of inter-correlation among them.
Individual Features Selected


8 features selected
3 global features and 5 moments





Curve of center of philtrum
Flatness of left cheek
Palpebral fissure length
3 moments from elliptical cut (global)
2 moments from the philtrum area
Classifier Methods Use: Support
Vector Machine

Sequential Minimal Optimization (SMO)


Breaks a large quadratic programming (QP)
problem into a series of smallest possible QP
problems
Multilayer Perceptron Networks

Feed-forward neural network trained with
back propagation algorithm
Test Set validation and Leave-OneOut Validation

The Test-Set Validation





Randomly choose 33% of data to be in a test set
Remainder is a training set
Build the model based on the training set
Estimate the future performance with test set
The Leave-One-Out Validation

Each dataset by itself is used as the test set at a
time, and the remaining n-1 datasets become the
training set. This process will repeat n times, and
the average performance can then be measured.
Results using the test set validation

With the SMO support vector machine: the success rate is
95.6522%
Predicted class
Confusion Matrix

Actual class
FAS (+)
FAS ()
FAS (+)
5
1
FAS ()
0
17
With the Multilayer Perceptron Network: the success rate is
86.9565%
Predicted class
Confusion Matrix
Actual class
FAS (+)
FAS ()
FAS (+)
5
1
FAS ()
2
15
Results Using Leave-One-Out Validation

With the SMO support vector machine: the success rate is
89.2308%
Predicted class
Confusion Matrix

Actual class
FAS (+)
FAS ()
FAS (+)
16
5
FAS ()
2
42
With the Multilayer Perceptron Network: the success rate is 89.2308%
Predicted class
Confusion matrix
Actual class
FAS (+)
FAS ()
FAS (+)
17
4
FAS ()
3
41
Data preprocessing




Face alignment: transform all datasets onto a common template face.
Cutting: trim a face dataset to a standard region defined by the
template face.
Distance mapping: map every point of the template face to a unique
point on each face dataset (important for many feature extraction
algorithms).
Feature mapping: using distance mapping to automatically generate
approximated features by mapping pre-defined features on the
template face.
template face
another face data
after alignment and cut
feature mapping