Transcript MotionDetection

Activity and Motion Detection
in Videos
Longin Jan Latecki and Roland Miezianko, Temple University
Dragoljub Pokrajac, Delaware State University
Dover, August 2005
1
Definition of Motion Detection
• Action of sensing physical movement in a
give area
• Motion can be detected by measuring
change in speed or vector of an object
2
Motion Detection
Goals of motion detection
• Identify moving objects
• Detection of unusual activity patterns
• Computing trajectories of moving objects
Applications of motion detection
• Indoor/outdoor security
• Real time crime detection
• Traffic monitoring
Many intelligent video analysis systems are based
on motion detection.
3
Two Approaches to Motion Detection
• Optical Flow
– Compute motion within region or the frame as a
whole
• Change detection
– Detect objects within a scene
– Track object across a number of frames
4
Background Subtraction
• Uses a reference background image for
comparison purposes.
• Current image (containing target object) is
compared to reference image pixel by pixel.
• Places where there are differences are
detected and classified as moving objects.
Motivation: simple difference of two images
shows moving objects
5
a. Original scene
b. Same scene later
Subtraction of scene a from scene b
Subtracted image with threshold of 100
6
Static Scene Object Detection and
Tracking
• Model the background and subtract to obtain
object mask
• Filter to remove noise
• Group adjacent pixels to obtain objects
• Track objects between frames to develop
trajectories
7
Background Modelling
by Michael Knowles
8
Background Model
9
After Background Filtering…
10
Approaches to Background Modeling
•
Background Subtraction
•
Statistical Methods
(e.g., Gaussian Mixture Model, Stauffer and Grimson 2000)
Background Subtraction:
1. Construct a background image B as average of few images
2. For each actual frame I, classify individual pixels as
foreground if |B-I| > T (threshold)
3. Clean noisy pixels
11
12
Background Subtraction
Background Image
Current Image
13
Statistical Methods
• Pixel statistics: average and standard
deviation of color and gray level values
(e.g., W4 by Haritaoglu, Harwood, and Davis
2000)
• Gaussian Mixture Model (e.g., Stauffer and
Grimson 2000)
14
Gaussian Mixture Model
• Model the color values of a particular pixel as a
mixture of Gaussians
• Multiple adaptive Gaussians are necessary to cope
with acquisition noise, lighting changes, etc.
• Pixel values that do not fit the background
distributions (Mahalanobis distance) are
considered foreground
15
Gaussian Mixture Model
Block 44x42 Pixel 172x165
Temple1 - RGB Distribution of Pixel 172x165
Temple1 - RG Distribution of Pixel 172x165
250
200
200
150
100
G
B
150
50
100
0
250
50
250
200
200
150
0
0
50
100
150
200
250
G
150
100
100
R
R-G Distribution
R-G-B Distribution
R
16
VIDEO
17
Proposed Approach
Measuring Texture Change
• Classical approaches to motion detection
are based on background subtraction, i.e., a
model of background image is computed,
e.g., Stauffer and Grimson (2000)
• Our approach does not model any
background image.
• We estimate the speed of texture change.
18
In our system we divide video plane in disjoint blocks
(4x4 pixels), and compute motion measure for each block.
mm(x,y,t) for a given block location (x,y) is a function of t
19
8x8 Blocks
20
Block size relative to image size
Block 24x28
1728 blocks
per frame
Image Size:
36x48 blocks
21
Motion Measure Computation
• We use spatial-temporal blocks to represent videos
• Each block consists of NBLOCK x NBLOCK pixels
from 3 consecutive frames
• Those pixel values are reduced to K principal
components using PCA (Kahrunen-Loeve trans.)
• In our applications, NBLOCK=4, K=10
• Thus, we project 48 gray level values to a texture
vector with 10 PCA components
22
Motion Measure Computation
3D Block Projection with PCA (Kahrunen-Loeve trans.)
t+1
t
t-1
4*4*3 spatial-temporal block
Location I=24, J=28,
time t-1, t, t+1
48-component block
vector (4*4*3)
10 principal components
-0.5221 -0.0624 -0.1734 -0.2221 -0.2621 -0.4739 -0.4201 -0.4224 -0.0734 -0.1386
23
Texture of spatiotemporal blocks works better
than color pixel values
• More robust
• Faster
We illustrate this with texture trajectories.
24
499
624
863
1477
25
Trajectory of block (24,8) (Campus 1 video)
Moving blocks corresponds
to regions of high local variance,
i.e., higher spread
3
2
Space of
spatiotemporal
block vectors
PCA 3
1
0
-1
-2
-3
-4
2
-5
0
-6
4
2
0
PCA 2
-2
-4
-6
-2
PCA 1
26
Comparison to the trajectory of a pixel inside
block (24,8)
8
6
3
2
RGB-PCA 3
4
PCA 3
1
0
499
2
695
0
-1
1477
2482
-2
-2
611
1
843
-3
-4
5
-4
2
0
-5
0
-6
4
2
0
-2
PCA 2
-4
Campus 1 video
block I=24, J=28
-6
-2
PCA 1
RGB-PCA 2-5
-10
-2
-1
0
1
2
3
RGB-PCA 1
Standardized PCA components of RGB
pixel values at pixel location (185,217)
that is inside of block (24,28).
27
4
Detection of Moving Objects Based on
Local Variation
For each block location (x,y) in the video plane
• Consider texture vectors in a symmetric
window [t-W, t+W] at time t
• Compute the covariance matrix
• Motion measure is defined as
the largest eigenvalue of the covariance matrix
28
Feature Vectors in Space
3.1
Feature vectors
4.2000
4.1000
3.9000
4.0000
4.1000
4.2000
4.3000
3
2.9
2.8
2.7
3.5000
3.7000
3.9000
4.0000
3.9000
3.8000
3.7000
2.6000
2.8000
2.9000
3.0000
2.8000
2.7000
2.6500
2.6
2.5
Covariance matrix
Current
time
2.4
3.5
0.0089
-0.0120
-0.0096
3.6
3.7
3.8
3.9
4
4.5
4.4
4.3
4.2
4.1
4
3.9
-0.0120
0.0299
0.0201
-0.0096
0.0201
0.0157
3.8
Motion Measure
Eigenvalues
0.0499 0.0035 0.0011
29
Feature Vectors in Space
Feature vectors
4.3000
4.4191
4.1798
4.2980
4.2843
4.1396
4.3257
3.1
3
2.9
2.8
2.7
3.7000
3.5944
3.8415
3.6195
3.7529
3.7219
3.6078
2.6500
2.4329
2.6441
2.5489
2.7114
2.7008
2.8192
2.6
Covariance matrix
2.5
2.4
3.5
0.0087
-0.0063
-0.0051
3.6
Current
time
3.7
3.8
3.9
4
4.5
4.4
4.3
4.2
4.1
4
3.9
-0.0063
0.0081
0.0031
-0.0051
0.0031
0.0154
3.8
Motion Measure
Eigenvalues
0.0209 0.0093 0.0020
30
Graph of motion measure mm(24,8,:) for Campus 1 video
31
Graph of motion measure
mm(40,66) of Sub_IR_2 video
SubIR2 Motion Measure - Block 40x66
SubIR2 Detected Motion - Block 40x66
1000
1000
800
800
600
600
400
400
200
200
0
0
100
200
300
Frames
400
500
Motion Measure
600
0
0
100
200
300
Frames
400
500
600
Detected Motion
32
Dynamic Distribution Learning and
Outlier Detection
(1)
f (t )  mean(t  1)
 C1
std (t  1)
Detect Outlier
(2)
f (t )  mean(t  1)
 C2
std (t  1)
Switch to a nominal state
(3)
mean(t )  u  mean(t 1)  (1  u)  f (t )
Update the estimates of mean and standard
deviation only when the outliers are not
detected
(4)
std (t )   2 (t )
(5)
 2 (t )  u   2 (t  1)  (1  u)  ( f (t )  mean(t  1)) 2
33