Transcript The MIT Artificial Intelligence Lab

Learning Patterns of Activity
Eric Grimson
Paul Viola
Trevor Darrell
MIT Artificial Intelligence Laboratory — Research Directions
Far Field Visual Analysis
• Monitor a distributed
setting
• Track users as they
move throughout a site
• Learn common patterns
of activity
• Recognize participants
and their actions
• Recognize location of
myself relative to world
• Detect unusual events
MIT Artificial Intelligence Laboratory — Research Directions
Examples of Tracking Moving Objects
QuickTime™ and a
decompressor
are needed to see this picture.
MIT Artificial Intelligence Laboratory — Research Directions
Multi-camera Coordination
MIT Artificial Intelligence Laboratory — Research Directions
Mapping Patterns to Groundplane
MIT Artificial Intelligence Laboratory — Research Directions
Detect Regularities & Anomalies in Events?
MIT Artificial Intelligence Laboratory — Research Directions
Example Track Patterns
• Running continuously for over 3 years
– during snow, wind, rain, dark of night, …
– have processed 1 Billion images
• one can observe patterns over space and over
time
• have a machine learning method that detects
patterns automatically
–
–
–
–
Classifies patterns into most probable clusters
Associates statistics of occurrence with each cluster
Learns to identify outliers that don’t fit a cluster
Can use shape, movement, color or other features to
cluster
MIT Artificial Intelligence Laboratory — Research Directions
Automatic Activity Classification
MIT Artificial Intelligence Laboratory — Research Directions
Example Categories of Patterns
• Video of sorted activities
QuickTime™ and a
decompressor
are needed to see this picture.
MIT Artificial Intelligence Laboratory — Research Directions
Analyzing Event Sequences
Histogram of activity over a single day
12am
6am
12pm
6pm
12pm
12am
6am
12pm
6pm
12pm
12am
6am
12pm
6pm
12pm
12am
6am
12pm
6pm
12pm
people
(1993 total with
.1% FP)
Resulting
classifier
groups
of people
(712 total with
2.2% FP)
clutter/lighting
effects
(647 total with
10.5% FP)
cars
(1564 total
with
3.4% FP)
MIT Artificial Intelligence Laboratory — Research Directions
Time
MIT Artificial Intelligence Laboratory — Research Directions
0:00
23:00
22:00
21:00
20:00
19:00
18:00
17:00
16:00
15:00
14:00
13:00
12:00
11:00
10:00
Trip Wire
9:00
8:00
7:00
6:00
5:00
4:00
3:00
2:00
1:00
0:00
Count (per minute)
0:00
23:00
22:00
21:00
20:00
19:00
18:00
17:00
16:00
15:00
14:00
13:00
12:00
11:00
10:00
9:00
8:00
7:00
6:00
5:00
4:00
3:00
2:00
1:00
0:00
Count (per minute)
Example Application
Street Traffic Counts: Weekday
20
15
10
Away from Mall
To Mall
5
0
Time
Street Traffic Counts: Saturday
20
15
10
Away from Mall
To Mall
5
0
0:00
Time
MIT Artificial Intelligence Laboratory — Research Directions
0:00
23:00
22:00
21:00
20:00
19:00
18:00
17:00
16:00
15:00
14:00
13:00
12:00
11:00
10:00
Trip Wire
9:00
8:00
7:00
6:00
5:00
4:00
3:00
2:00
1:00
Speed (mph)
0:00
0:00
23:00
22:00
21:00
20:00
19:00
18:00
17:00
16:00
15:00
14:00
13:00
12:00
11:00
10:00
9:00
8:00
7:00
6:00
5:00
4:00
3:00
2:00
1:00
Speed (mph)
Example Application
Street Traffic Speed: Weekday
100
80
60
Away from Mall
40
To Mall
20
0
Time
Street Traffic Speed: Saturday
100
80
60
Away from Mall
40
To Mall
20
0
Analyzing Individual Motions
• Classify types of
locomotion using
templates of frequency
and phase variations
• Identify individuals
based on gait
Walking
Running
Jumping
Skipping
Crawling
Quad1
MIT Artificial Intelligence Laboratory — Research Directions
…and this works for many problems
• Recognizing activities in and around
buildings
• Detecting unusual events
• Gathering statistics on patterns of events
around a site
• Eldercare monitoring
• Retail marketing and analysis
MIT Artificial Intelligence Laboratory — Research Directions