Transcript Artefact Structural Learning through Imitation - What is Neuro

Project ArteSImit
Artefact Structural Learning through Imitation
(TU München, U Parma, U Tübingen, U Minho, KU Nijmegen)
Giorgio Panin - TUM
Artesimit Project Partners
Alois Knoll and colleagues (project coordinator, Robotics, Munich)
Giacomo Rizzolatti and colleagues (Neurophysiology, Parma)
Peter Thier and colleagues (Cognitive Neuroscience, Tübingen)
Wolfram Erlhagen and colleagues (Cognitive Modeling, Minho)
Harold Bekkering and colleagues (Cognitive Neuropsychology, Nijmegen)
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
2
The concept of goal-directed Imitation
Central issue of imitation:
do we imitate movements (means) or consequences (goals) of the
observed behavior?
Movements somehow need to be directly mapped (via-point
approach), whereas goals need to be represented.
Movement mapping assumes homologue bodies, full access
of the whole movement and stable environments.
Goal representation assumes an association between an
(observed) action and the (observed) consequence.
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
3
How to work together?
Behavioral measurements (healthy and clinical patients)
Brain research (single-cell, fMRI, MEG, EEG)
Modeling (Dynamic neural fields)
Robotics (Visual analysis, cognition, and artefact control)
THE solution: One common paradigm
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
4
Common paradigm
Starting position
Movement 1
Movement 2
A LED gives information about
either the goal or the means
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
5
Parietal Mirror Neuron findings
Motor responses of parietal neurons
Motor task
Observation task
Visual responses of parietal mirror neurons
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
6
Architecture of the dynamic model
Bridge Paradigm
PFC:
Goal representations
Task input
Object properties (e.g. colour)
STS:
Visual desciption of grip and trajectory
PF:
Sequence of means
F5:
Movement primitives
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
7
Visual system in action: Task recognition
Complete information acquisition for
the decision and planning tasks
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
8
Learning I: Association between object properties and goal
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
9
Learning II: Copying the means of the teacher
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
10
Goal-directed imitation
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
11
Partial information: Cueing of the goal
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
12
Partial information: Cueing of the means
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
13
Summary
The project combined ...
• the development of a new paradigm for goal-directed imitation with imaging studies
across all modalities
•
new findings in experimental neurophysiology and advanced neuromodelling of the
structure of the PFC-(STS-PF-F5)-coupling to develop and implement computeroperational models for goal-directed imitation
•
the development of a powerful model-based hand posture recogniser (using only one
B/W camera)
into ...
• A complete artefact (robot system) capable of mimicking imitation behaviour of
creatures: all project goals have been met!
__________________________________________________________________________________________________________________________________________________________________________________
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-”Neuroinformatics for Living Artefacts” under contract No. IST-2000-29689
14