Transcript A_Probabilistic_Framework_for_Segmentation_and_Tracking

A Probabilistic Framework For
Segmentation and Tracking of
Multiple Non Rigid Objects for Video
Surveillance
Aleksandar Ivanovic, Thomas S. Huang
Backmen Institute for Advanced Science and
Technology, UIUC
Outline


Introduction
Pixel probability model
 Background
model
 Foreground probability model




Connected components matching
Object detection
Experimental results
Conclusion
Introduction


In video surveillance, reliable segmentation of moving
objects is essential for successful event recognition
Park and Aggarwal
A Gaussian mixture model is used at the pixel level to train
and classify individual pixel colors
 Markov Random Field (MRF) framework is used at the blob
level to merge the pixels into coherent blobs and to register
inter-blob relations
 A coarse model of the human body (head, upper body,
lower body) is applied at the object level as empirical
domain knowledge to resolve ambiguity due to occlusion
and to recover from intermittent tracking failures

Introduction

Elgammal and Davis




Use maximum likelihood estimation to estimate the best
arrangement for people
Modeling these regions involves modeling their appearance (color
distributions) as well as their spatial distribution with respect to
the body
Assumption: targets are visually isolated before occlusion so that
we can initialize their models
Gomila and Meyer


Each image of a sequence is segmented and represented as a
region adjacency graph
Object tracking becomes a particular graph-matching problem, in
which the nodes representing the same object are to be matched
Background model


Each Lu*v* dimension is modeled with a single
Gaussian
Initialize the background pixel model
 No
foreground objects, computing the statistics over the
training sequence
 Otherwise, use a bootstrapping algorithm [9]

Make use of the Mahalanobis distance M b ( x, y) to
achieve segmentation, which corresponds to the
probability that pixel p( x, y ) belongs to the
background
[9] Gutchess et al., “A background model initialization algorithm for video
surveillance”
Mahalanobis distance

Mahalanobis distance M b ( x, y) between each pixel
and the corresponding background pixel:
M b ( x, y ) 


L( x, y )  Lmean ( x, y )
Lvar ( x, y )
u * ( x, y )  u *mean ( x, y )
u *var ( x, y )
v * ( x, y )  v *mean ( x, y )
v *var ( x, y )
Foreground probability model (1)

For each pixel p ( x, y ) , we have:
Feature vector A( x, y)  [M b ( x, y), D( x, y), Ph ( x, y)]
 Label F ( x, y ) (0 for background, 1 for foreground)
D( x, y ) : absolute distance of the current pixel to the


background pixel in the RGB color space
D( x, y )  R( x, y )  Rmean ( x, y )
 G ( x, y )  Gmean ( x, y )
 B( x, y )  Bmean ( x, y )
where Rmean ( x, ,y) , Gmean ( x, y ) , and Bmean ( x, y) are the
means over the video frame in RGB space
Foreground probability model (2)

A color similarity measure Ph ( x, y)
 Computed
100
from the cumulative histogram of all tracked
objects90
80
 Computed as the number of pixels in the bin that
70
p ( x, y ) divided by the number of the pixels in
contains
60
50
the color
histogram

40
Histogram
30
A
20
histogram
for one component (R, G, B) describes the
10
distribution
of the number of pixels for that component
0
color in the 0-15
16 bins16-31
32-47
…
224-239
240-255
Foreground probability model (3)

Bayesian Network (BN)
 Model


the relationship of pixel F ( x, y )
label with feature vector A( x, y )
Model P( A | F  0) and P( A | F  1) using Gaussian
mixture model (GMM) using K  5 Gaussians
The probability the a pixel belongs to the
foreground Pf ( x, y ) is :
Pf ( x, y ) 
P( A | F  1) P( F  1)
P( A | F  0) P( F  0)  P( A | F  1) P( F  1)
Blob Level



This paper doesn’t mention!
Foreground pixels with the same color distribution
are labeled in the same class
Relabeling the disjoint blobs in connected
component analysis
 Adjacency
 Color Similarity
 Small blob
Components matching (1)

Find the objects in the new frame

Pf ( x, y ) is binarized using an adaptive threshold

A union find algorithm is used to find its 8-connected
components that correspond to foreground objects
4-connected components
 8-connected components


Foreground object matching cases:
One-to-one matching
 Many-to-one matching
 One-to-many matching

Components matching (2)



Use probabilistic matching to solve the problems
mentioned above
(foreground object, connected component): ( f (i), c( j ))
f (i ) , c ( j ) can be described with feature vector :
k (t )  [ xs (t ), ys (t ), S (t ), H (t ), xc (t ), yc (t )]
: horizontal and vertical size of the bounding box
 S (t ) : the size in pixels
 H (t ) : color histogram of object/component t
 ( xc (t ), yc (t )) : centroid of all the pixels of an
object/connected component
 t : index (i.e. t  i or t  j )
 ( xs (t ), ys (t ))
Components matching (3)

Derive information for matching a foreground
object f (i) to a connected component c( j ) :
m(i, j )  [ SC (i, j ), ED(i, j ), HS (i, j ), XC (i, j ), YC (i, j )]
: size change as S ( j ) / S (i)
 ED(i, j ) : Euclidean distance between ( xc (i), yc (i)) , ( xc ( j ), yc ( j ))
 HS (i, j ) : similarity between H (i ) , H ( j )
 XC (i, j ) : horizontal size change as xs ( j ) / xs (i)
 YC (i, j ) : vertical size change as ys ( j ) / ys (i )
 SC (i, j )

If f (i) and c( j ) are matched, label
with M  1 , otherwise M  0
( f (i ), c( j ))
Components matching (4)

Probability of matching
 One-to-one
P(i  j ) 
 Occlusion
matching:
P( M  1 | m(i, j ))
P( M  0 | m(i, j ))  P( M  1 | m(i, j ))
:
P(i1  i2  j ) 
 Object
P( M  1 | m(ic , j ))
P( M  0 | m(ic , j ))  P( M  1 | m(ic , j ))
color similar to background:
P(i  j1  j2 ) 

P( M  1 | m(i, j ))
P( M  1 | m(i, jc ))
P( M  0 | m(i, jc ))  P( M  1 | m(i, jc ))
Probability of foreground object c( j0 ) disappeared
is P( j0  d ) , where d is dummy node
Object Detection

How to decide the connected components should
become new object:
 Define
a set of feature T  [ S , LC , SH , CS ]
S : size of the connected component
 LC : distance to the nearest location of a foreground object
area
SH

 SH : a shape feature, defined as
perimeter
 CS : color similarity of object candidate to the average

2
foreground object
N  0 if candidate is not new object, otherwise
label N  1
 Label
Experimental Results

Test a 55 minute long indoor sequence
(a)(f)
Foreground
object
(b)
Probability based
only on
background model
(c)
Probability of
foreground
(d)(g)
Segmented objects
using only
background model
(e)(h)
Segmented objects
using probability of
foreground
Conclusion

Contributions of this paper:
A
new probabilistic framework for pixel segmentation
and for matching of objects to blobs
 A framework can account for grouping of objects
 A method robust to initialization

The matching formulation is better able to model
multi-object tracking and gives more reliable
segmentation results