Transcript Video Mining

Video Mining
Learning Patterns of Behaviour via
an Intelligent Image Analysis System
Introduction
• All large archives of video are now available in
repositories, there is hidden much potentially useful
knowledge
Large archives of
videos
Data mining
Techniques
Latent, Useful,
Interesting Information
2015-04-13
Jang Hee Kyun
2
Introduction
• Video mining is more challenging, due to lack of
explicit structure in the raw data in video archives
• Ex) Surveillance, Analysis of expert’s activities
Studying animal behaviour, Emulate robot behaviour
• Each applications there are often patterns of activity
• They can be classified in order to gain more general
insights into agent and object movements and
behaviour
2015-04-13
Jang Hee Kyun
3
Video Mining
•
Interesting in that the raw data in videos is expressed
in a way that is not directly amenable to the use of
conventional mining techniques
•
There is a lot of variation in the details of the patterns
and it is important to abstract out the key features of
a behaviour
•
Unwanted or irrelevant details and noise can be
filtered out
2015-04-13
Jang Hee Kyun
4
Video Mining
• Video mining of patterns of behaviour and their interplay, has to deal with very dynamic situations
• Techniques that have been developed for the
extraction of temporal rules from collections of time
series data
• We now want to identify patterns of data that are
unusual, and discover inter-relationships between the
patterns of different agents and objects
2015-04-13
Jang Hee Kyun
5
Discover Rules
• First Stream
• Detect ‘abnormal ‘ or ‘interesting ’ behaviour
• The ability to learn what is ‘abnormal ’ or ‘ interesting ’
• Ultimate goal is that they will use only innate knowledge
• Second Stream
• Summarisation for behavioural pattern detection/ matching in
the second stream using AI (Artificial Intelligence) and DM
(Data Mining) algorithms for time series analysis
2015-04-13
Jang Hee Kyun
6
Discover Rules
• It will be able to operate without the direct
intervention of a user, and be able to control its own
focus of attention to some extent
• This will in turn influence how it operates in related
situations in the future
2015-04-13
Jang Hee Kyun
7
Background and approach
• We use our own system, ModTrack, for vehicle
detection and tracking
• “Independent Moving Object Detection Using a Colour
Background Model” by F. Campbell – West, P. Miller, H. Wang
• DM (Data Mining) aspects
• Tracking system
• Identify abnormal behaviour
• Infer unusual pattern of activity
• AI (Artificial Intelligence) aspects
• Learning how another agent learns and making use of the
results
2015-04-13
Jang Hee Kyun
8
Method of representation and analysis
• Our objective is to reverse engineer what we observe
in the real world by using a vision or imaging system
• We need to emulate the behaviour of the real world actors
• How does a robot adjust its knowledge about the
behaviour of a light using the adaptive learning
paradigm?
2015-04-13
Jang Hee Kyun
9
Method of representation and analysis
• We use the robot’s intention not only as a
consideration for our decision making, but also as a
guide for our accumulation of observations
• We make a qualitative assessment by distinguishing
suggestion and confirmation
2015-04-13
Jang Hee Kyun
10
Example
2015-04-13
Jang Hee Kyun
11
Method of representation and analysis
1. As it detects sequences of such atomic movements
the system records them
2. Behaviour pattern detection
•
•
Classify these behavioural seuqences
Classifier is important requirement
3. Represent activities rule set
•
2015-04-13
Behaviour matching and prediction
Jang Hee Kyun
12
Conclusions and Summary
• ModTrack was used to obtain the behavioural traces
of the robot/agent
• Using representation we can build a new
representation of what is happening with the raw data
• We have shown how detailed behaviours from video
can be coarsened and mined to obtain useful
knowledge
2015-04-13
Jang Hee Kyun
13