Transcript Multimedia Data Mining for Intelligent Surveillance Systems

Jai sri ganeshay namah sri saraswatyiy namah om namah shivay jai bajarang bali dada jai sai baba jai gaytri mata jai tirupati dada jai mahalaxmi mata jai kuldevi ma chamunda mata jai surya chandra mangal budh guru sukra sani rahu ketu devta santi bhavantu jai mariyamma mata jai anjnaiya dada jai mahavir swami bhagvan jai swami swarupanand saraswati ji bhagvan jai dakor na thakorji jai servdevodevi namah stute om shanti bhavantu
Multimedia
Surveillance Data
Mining for Analytics
Outline







Motivation
Introduction
Problem Definition
Proposed Approach for Evacuation Scenario
Statistical data mining Model
Results Obtained on VAST challenge dataset
Future Work
Motivation



Wide use of surveillance system for
monitoring the behavior of people, vehicles
Objective: To detect suspicious behavior
based on available multimodal data
Strong need for automated or semi automated
means for suspicious behavior detection and
prediction
Introduction

Video Surveillance Systems
 Expensive
 Rich

amount of information
RFID Surveillance Systems
 Not
very expensive
 Limited amount of information
Therefore can use appropriate sensory data for the
task at hand and can even use multiple modalities for
redundancy and cost-savings
Introduction

Suspicious movement detection scenarios
 Explosion
event followed by evacuation
 Open firing event followed by chaos
 Even a small accident in office or street leads
to considerable change in normal movement
pattern

Need quick way of analyzing and also the
way of predicting suspicious behavior
Introduction

Video Surveillance
Systems



Observing large volume of
data by a few observers
Suspicious patterns may
not be explicitly visible to
observer
RFID Surveillance
Systems

Suspicious patterns are not
visible to observer
Therefore some automated pattern analysis or data mining
is required
Problem Definition

To build an intelligent surveillance system’s tool
that can,
 Help
investigate suspicious behavior for different
scenarios,
 Automatically or semi automatically incorporating the
intuitions that are similar to the one that security
officer can have.
 Where investigation should give answers to when?,
where?, who?, what? etc.
Evacuation Dataset of
IEEE VAST Challenge 2008



In 2007 an explosive device was set off at a Miami,
Florida DOH building, resulted in casualties and damage
Employees & visitors wore badges (RFID)
Data provided





Time: Ticks, representing intervals between tag readings
Person Id: Tag identification of all employees and visitors
Xcor: the location x-coordinate
Ycor: the location y-coordinate
The file includes data before and throughout the incident.
Input Trajectory Data


Trajectory of 82
people over total
Time Duration of
837seconds on
building map of
91x61 grid space.
Making sense of this
data seems extremely
difficult
Questions for the Evacuation
Scenario
Where was the device set off?
 Identify potential suspects and/or
witnesses to the event.
 Identify any suspects and/or witnesses
who managed to escape the building.
 Identify any casualties.

Proposed Approach




Gather intuitions (hypotheses) for the scenario
Compute the possibly useful parameters like
average speed in certain time interval, average
traversed area in certain time interval
Build a statistical model using the computed
parameters combined with the hunches
Perform Analysis
Intuitions for the Evacuation
Scenario
Evacuation Scenario in office environment
where explosion event is followed by
evacuation.
 Intuition 1 [Normal Behavior]:

 Usual
movement of people will be low before
explosion event and it will increase drastically
afterwards to evacuate the scene.
Intuitions for the Evacuation
Scenario

Intuition 2 [Suspicious Behavior]:
 Suspicious
persons would try to run away
from explosive device location before the
explosion happens.

Intuition 3 [Victims Behavior]:
 Victims
would have normal behavior before
the explosion event but will be injured or have
fainted or be dead on explosion.
Formulation of Statistical Model

Parameters for Statistical Model:
 Time
Window: The analyst needs to input
appropriate time window parameter for the
statistical model to compute the following
 Speed of each Person
 Area Traversed by each Person
 Average Global Speed of People
 Average Global Area Traversed by People
When did the Explosion happen?



Obtain the Global
Average Speed.
Find the Global
Maximum value from
Based on intuition1
we can consider this
GM as approximate
start time of Explosion
Where was the device set off?




Average speed and Average
area traversed by the Victims
will be almost near to zero
after explosion event.
They may not be able to reach
to the Evacuation Area.
They will be found within or
very near to the explosion
area.
Location cluster of such people
represents the area of
explosive device.
Where is Evacuation Location?


Based on intuition1
people are trying to
reach to evacuation
place.
High density region at
end times would be
representing
evacuation place.
Who are the Suspects?



Average speed and Average
area traversed by the persons
will be higher before explosion
event.
Suspicious person should
have visited Explosion location
just prior to the explosion.
They might either reach
Evacuation before others or
will escape without entering
Evacuation area.
INPUT DATA
Time & Location of
each person
Computing required
Parameters ( speed, area
Covered within time window)
Finding the Start Time
Of Event (Explosion)
Analyzing the speed
before Event
(Explosion)
Analyzing the speed
after Event
(Explosion)
High speed people in
this duration is set of
Suspicious people
Low speed people in
this duration is set of
Victims
Traversed through Event
(explosion location) are
strong set of suspects
Clustered at event
(explosion) location
Evacuation Model
Future work

Need to incorporate other data captured
 Video
data
 Audio data
 Fire Alarm, Temperature data etc.

Come up with a Mining/Analytics tool to
facilitate such investigations.
Definition

Data mining:
 “is
the process of automating information discovery”
or
 “is the exploration and analysis by automatic or
semiautomatic means, of large quantities of data in
order to discover meaningful patterns and rules”

“multimedia data mining”
 “knowledge
discovery in a multimedia database”
 “extraction of implicit knowledge, mm data
relationships or other patterns not explicitly stored in
multimedia files”
Motivation
Tremendous benefits of traditional data
mining is proven for structured data.
 Now its time for extending the mining
techniques for unstructured,
heterogeneous data.

MDM Challenges and Problems

Feature Selection Dimensionality Reduction: for reducing the
problem size , enables learning algorithms to operate faster and
effectively.

Feature construction / transformation: by constructing new features
from the basic features set.

How to analyze the heterogeneous data that consist of text, graphs,
images, sounds, videos and other kind of sensor data? Multimedia
data has complex structures that can not be processed as a whole
by available data mining algorithms.

Tokenizing textual document into words and phrases has proven to
work reasonably well for retrieval but images, audio, video etc
cannot be readily decomposed into such semantic units.