Transcript SoLSTiCe Similarity of locally structured data in computer vision

SoLSTiCe
Similarity of locally structured data
in computer vision
Université-Jean Monnet (Saint-Etienne)
LIRIS (Lyon)
(1/02/2014 -2018)
Elisa Fromont, Kick-off meeting, 14/02/2014
Présentation du consortium
Main ideas
•
•
•
Aim: design new models and tools for representing and managing images and
videos
Targeted applications: classification, recognition or indexing (in a context of
occlusions and non rigid objects in 2D (+ t), 3D and 3D+t media)
Proposal: explore locally structured data (LSD) = visual features + discrete
structures to model local (spatio-temporal) relationships
• 3 main tasks:
1. [Extracting LSD from images and videos:] extract relevant visual features and
structure them w.r.t. spatial and temporal relationships.
2. [Measuring the similarity of LSD:] design relevant similarity measures for
comparing LSD, and efficient algorithms for computing these measures.
3. [Mining LSD:] characterize LSD by means of frequently (or infrequently) occurring
patterns (itemsets, sequences or graphs) and use them to create discriminative
features for solving computer vision tasks.
The project : 4 tasks interconnected
1.
2.
3.
4.
5.
[Task 0] will be dedicated to the project management;
[Task 1] will design LSD for describing images and videos, and
will design tools for extracting these LSD;
[Task 2] will design kernels, similarity measures and matching
algorithms for comparing LSD;
[Task 3] will design mining algorithms for extracting relevant
patterns in LSD;
[Task 4] will be dedicated to the design and use of demo platforms
to test (and demonstrate) on computer vision benchmarks and new
datasets the models and tools designed in Tasks 1 to 3.
Livrables (1/2)
Tâche 1 From images and Vidéos to LSD (LaHC)
D1.1
Research report describing new descriptors for images
D1.2.2
Survey of state-of-the-art approaches for structuring visual words by
means of strings, trees or graphs
D1.2.2
Research report describing new LSD for images and 3D objects
D1.3
Research report on extensions of LSD of subtask 2.2 for videos, and
evaluation
Tâche 2: Mesuring the similarity of LSD (LIRIS)
D 2.1.1
Research report describing new matching algorithms
D2.1.2
Design of an open-source library of graph matching algorithms
D2.2
Research report on new kernel for combinatorial maps
D2.3
Research report on metric learning or deep learning on locally structured
data
Livrables (2/2)
Tâche 3: Mining LSD
D3.1.1
Research report on mining LSD in images and videos
D3.1.2
Research report on new algorithms to mine LSD in images and/or videos
D3.2
Research report on using frequent substructures to find relevant features for image
classification
D3.3.1
Research report on mining approximate patterns in plane graph
D3.3.2
Research report on finding relevant spatio temporal patterns in videos
Tâche 4: Demonstrations in computer vision
D4.1.1
Creation of the Solstice platform
D4.1.2
Activity recognition module for software platform LIRIS-VISION Tracking
D4.1.3
Demo in the Solstice platform
D4.1.4
Activity recognition module for robotics platform LIRIS-VOIR
D4.2.1
Object recognition module for software platform LIRIS-VISION
D4.2.2
Object recognition demo for the Solstice platform
D4.2.3
Object recognition module for robotics platform LIRIS-VOIR
Planning
Valorisation/Impact
• scientific communications submitted to major conferences and journals
(CVPR, ECCV, ICCV, ICPR, AVSS, KDD, ICML, ECML, PKDD, ICPR, etc.) and
journals (IEEE-T-PAMI, PR, IJCV, CVIU, MLJ, JMLR, etc.) in image
processing, pattern recognition, combinatorial optimization, machine
learning, and data mining.
• open source platforms developed in task 4 (and task 2)
• workshops co-located with major conferences in order to share ongoing
research.
• design educational and recreational demos targeting a non specialist
public to be presented during popular events such as “la fête de la
science”.
Use of resources
• LaHC (140000 euros):
– Staff (100000 euros) Ph.D Student: 36 months on « New matching
strategies for data mining applied to computer vision problems » (tasks 2
and 3 + 1 and 4) co-supervised with liris
– Travels
– Other expenses: master thesis grants + hardware
• LIRIS (134000 euros)
– Staff (100000 euros): Ph.D Student: 36 months on « Analysis of complex
scenes with structured models » (tasks 1 and 2 + 3) co-supervised with
LaHC
– Travels
– Other expenses: master thesis grants + hardware
Points to discuss
• Website (Jean Monnet)
• Include some more members (Taygun,
Romain?)
• How to spend the money for the second thesis
(Remi, Marc, Damien ?)
• Demos?
• Next meetings