Transcript Cognitive Vision - Inria Sophia Antipolis

ORION Project-team
Monique THONNAT
INRIA Sophia Antipolis
Creation: July 1995
Multidisciplinary team:
artificial intelligence, software engineering, computer vision
Contents

Team Presentation

Research Directions

Cognitive Vision 2002-2006

Reusable Systems 2002-2006

Objectives for the next Period
Evaluation May 2006
Orion
2
Team presentation (May 2006)
4 Research Scientists:
François Bremond (CR1 Inria)
Sabine Moisan (CR1 Inria, HDR)
Annie Ressouche (CR1 Inria)
(team leader)
Monique Thonnat (DR1 Inria)
1 External Collaborator: Jean-Paul Rigault (Prof. UNSA Inria secondment)
4 Temporary Engineers: Etienne Corvee, Ruihua Ma, Valery Valentin,
Thinh Van Vu
7 PhD Students:
Bui Binh, Bernard Boulay, Naoufel Kayati,
Le Thi Lan, Mohamed Becha Kaaniche,
Vincent Martin, Marcos Zuniga
Evaluation May 2006
Orion
3
Research directions
Objective:
Intelligent Reusable Systems for Cognitive Vision
Cognitive Vision:
 Interpretation of static images
 Video understanding
Reusable Systems:
 Program Supervision
 LAMA Software platform
Evaluation May 2006
Orion
4
Orion team positioning
Cognitive Vision:


Image interpretation (ECVision European network on cognitive vision,
EUCognition) vs. computer vision (INRIA CogB)
Video understanding (USC Los Angeles, Georgia Tech. Atlanta, Univ.
Central Florida, NUCK Taiwan, Univ. Kingston UK, INRIA Prima)
Reusable Systems:

Program supervision: e.g., scheduling (ASPEN and CASPER at JPL),
image processing (Hermès at Univ. Caen, ExTI at IRIT)…

Platform approach: e.g., ontology management (Protegé at Stanford),
frameworks for multi agents (Aglets, Jade, Oasis at LIP6), distributed
object community (Oasis at INRIA Sophia)…
Evaluation May 2006
Orion
5
Cognitive Vision :
Image Interpretation 2002-2006
Objective: semantic interpretation of static 2D images



Recognition of object categories (versus individuals)
Recognition of scenes involving several objects with spatial reasoning
Intelligent management of image processing programs
Towards a cognitive vision platform
Evaluation May 2006
Orion
6
Cognitive Vision :
Image Interpretation 2002-2006
Scientific achievements:

Knowledge acquisition:

A visual concept ontology with 144 spatial, color and texture
concepts [MVA04]


Learning:

Visual concept detectors [IVC06]

Image segmentation parameters [ICVSa06]
Cognitive vision platform

Architecture [ICVS03]

Object class recognition algorithm [CIVR05]
Evaluation May 2006
Orion
7
Cognitive Vision:
Image Interpretation 2002-2006
Self Assessment:

Strong points:

Visual concept ontology as user-friendly intermediate layer between
image processing and application domain


Automatic building of the visual concept detectors
Still open issues:

Learning for image segmentation

Temporal visual concept ontology
Evaluation May 2006
Orion
8
Cognitive Vision:
Video Understanding 2002-2006
Objective:

Real time recognition of interesting behaviors
How?

Data captured by video surveillance cameras

Original video understanding approach mixing:

computer vision: 4D analysis (3D + temporal analysis)

artificial intelligence: a priori knowledge (scenario, environment)

software engineering: reusable VSIP platform
Evaluation May 2006
Orion
9
Cognitive Vision:
Video Understanding 2002-2006
Objective: Interpretation of videos from pixels to alarms
Segmentation
Classification
Tracking
Scenario Recognition
Alarms
access to
forbidden
area
3D scene model
Scenario models
Evaluation May 2006
Orion
A priori Knowledge
10
Cognitive Vision:
Video Understanding 2002-2006
Scientific achievements:



Multi-sensor video understanding:

2 to 4 video cameras overlapping or not [IDSS03,JASP05]

Video cameras + optical cells + contact sensors [AVSS05]…
Learning:

parameter tuning[MVAa06]

frequent temporal scenarios models [ICVSb06]
Temporal scenario:

a new real time recognition algorithm [IJCAI03,ICVS03]

a new representation language [MVAb06,ECAI02,KES02]
Evaluation May 2006
Orion
11
Cognitive Vision:
Video Understanding 2002-2006
Industrial impact:

Strong impact in visual surveillance (metro station, bank agency,
building access control, onboard train, airport)

4 European projects (ADVISOR, AVITRACK, SERKET, CARETAKER)

5 industrial contracts with RATP, ALSTOM, SNCF, Credit Agricole,
STMicroelectronics


2 transfer activities with BULL (Paris), VIGITEC (Brussels)
Creation of a start-up Keeneo July 2005 (8 persons) for
industrialization and exploitation of VSIP library.
Evaluation May 2006
Orion
12
Cognitive Vision:
Video Understanding 2002-2006
Intelligent video surveillance of Bank agencies
Evaluation May 2006
Orion
13
Cognitive Vision:
Video Understanding 2002-2006

“Unloading Global Operation”
Toulouse - 3rd June 2004
Orion
14
Cognitive Vision:
Video Understanding 2002-2006
Airport Apron Monitoring “Unloading Operation”
European AVITRACK project
Toulouse - 3rd June 2004
Orion
15
Cognitive Vision:
Video Understanding 2002-2006
Self Assessment:

Strong points:

Video understanding approach: real time, effective techniques
used by external academic and industrial teams

Launch of an evaluation competition for video surveillance
algorithms (ETISEO) with currently 25 international teams

Still open issues:

Learning

Multi sensor
Evaluation May 2006
Orion
16
Reusable Systems: Program Supervision
Reusable Systems: original approach for the reuse of
programs with program supervision techniques
Program supervision:
Automate the (re)configuration and execution of programs

selection, scheduling, execution, and control of results
Knowledge-based approach: knowledge modeling,
planning techniques, …..
Evaluation May 2006
Orion
17
Reusable Systems: LAMA Platform
Reusable Systems:
Reuse of tools to design knowledgebased systems (KBS)
LAMA
LAMA Software Platform:
provide
generic
components
and tools
raise new
issues, to be
abstracted
into new
components
Problem
Solving
KBS
Virtuous Circle
Evaluation May 2006
Set of toolkits to facilitate design and
evolution of KBS elements:

engines, GUI, knowledge languages,
learning and verification facilities…
Software Engineering approach:
genericity, frameworks, objects and
components
Orion
18
Reusable Systems: LAMA Platform
LAMA
Designer
Expert
Java graphic library for GUIs
Compilers/verifiers generators
for knowledge
description languages
Blocks
Verification library for
knowledge bases
Program
Supervision
Framework
for engine design &
Object
knowledge
Recognition
representation
support and
Model
task specific layers
Calibration
Evaluation May 2006
Orion
Task dedicated GUI
Task dedicated Language
with compiler
& KB verification
Task dedicated
Engine
KBS
Knowledge
Base
User
19
Reusable Systems:
Program Supervision 2002-2006
Scientific achievements:

Improvement of the Pegase engine (Pegase+)


Distributed program supervision


Multithreading, extensions to the YAKL language [ECAI02]
Supervision Web server, multi-agent techniques, interoperability
Pegase/Java/agents [TC06]
Cooperation with image and video understanding


Object recognition task using program supervision [ICTAI03]
Interoperability with VSIP: program supervision for video
understanding [ICVSc06]
Evaluation May 2006
Orion
20
Reusable Systems:
LAMA Platform 2002-2006
Scientific achievements:

Enforcing LAMA safe usage


Verification of LAMA component extensions relying on Model
Checking approach [Informatica01, SEFM04]
Encompassing new tasks


Classification and object recognition in images: new engine and
new knowledge representation language [ICTAI03]
Model calibration in hydraulics: new engine/language (PhD codirected with INPT and CEMAGREF) [KES03, JH05]
Evaluation May 2006
Orion
21
Reusable Systems: Self Assessment
Strong points:

Real time performance (Pegase+ and video)


LAMA genericity at work



Different tasks (supervision, classification, calibration) in
various application domains (hydraulics, biology, astronomy,
video surveillance…)
Shorter development time and safer code
Reuse of concepts as well as code


Using program supervision costs less than 5% of overall
processing time
Several variants of a task sharing common concepts
Extensibility and commitment to Standards
Evaluation May 2006
Orion
22
Objectives for the next period 1/5
Creation of a new INRIA project-team PULSAR
Perception Understanding and Learning Systems for Activity Recognition
Theme:
CogC Multimedia data: interpretation and man-machine interaction
Multidisciplinary team:
artificial intelligence, software engineering, computer vision
Objective:

Research on Cognitive Systems for Activity Recognition

Focus on spatiotemporal activities of physical objects

From sensor output to high level interpretation
Evaluation May 2006
Orion
23
Objectives for the next period 2/5
PULSAR Scientific objectives:
Two research axes:

Scene Understanding for Activity Recognition

Generic Components for Activity Recognition
PULSAR Applications:

Safety/security (e.g. intelligent surveillance)

Healthcare (e.g. assistance to the elderly)
Evaluation May 2006
Orion
24
Objectives for the next period 3/5
PULSAR: Scene Understanding for Activity
Recognition

Perception: multi-sensors, finer descriptors

Understanding: uncertainty, 4D coherency, ontology for
AR

Learning: parameter setting, event detector, activity
models, program supervision KB (risky objective)
Evaluation May 2006
Orion
25
Objectives for the next period 4/5
PULSAR Generic Components for Activity Recognition
From LAMA Platform to AR platform:
 Model extensions:



User-friendliness and safeness of use:



modeling time and scenarios
handling uncertainty
theory and tools for component frameworks
scalability of verification methods
Architecture improvement:



parallelization, distribution, concurrence
real time response
domain specific software and graphical interface plugging
Evaluation May 2006
Orion
26
Objectives for the next period 5/5
Short term objectives:
Scene Understanding for Activity Recognition

Perception: gesture analysis

Understanding:


ontology-based activity recognition

uncertainty management
Learning: primitive event detectors learning
Generic Components for Activity Recognition

Model of time and scenarios

Internal concurrency and distributed architecture
Evaluation May 2006
Orion
27