Transcript 2 - People.csail.mit.edu

Modeling the development of mirror neurons
Giorgio Metta, Paul Fitzpatrick • MIT AI Lab, Humanoid Robotics Group • {pasa,paulfitz}@ai.mit.edu
Question: How do mirror neurons develop?
Answer: Two stage hypothesis: learn about objects and then
imitate the goal of an action if directed towards the same
object.
Our solution
We used poking and prodding as a precursor to full-blown
manipulation
Motivation
Our inspiration comes from neuroscience: in particular,
mirror neurons. This is a class of neurons found in the
monkey's frontal cortex (area F5). A particular mirror neuron
is activated both when the monkey executes an action and
when it observes the same action performed by somebody
else. Mirror neurons can be related to gesture recognition,
language, and learning by imitation.
11. Learn about object affordances (canonical neurons)
22. Interpret observations on the basis of affordances (mirror)
Facts
F5 contains two classes of neurons: canonical
and mirror. Both classes respond when
object-directed actions are executed.
Canonical neurons respond to the
presentation of an object according to the
potential grasp types the object affords.
Mirror neurons respond to an observed action
in accordance to the grasp type.
33. Poke/push objects according to their affordances
44. Mimicry
Object, goal connects robot and human action
2
0.5
estimated probability of occurrence
1
pull in
side tap
0.45
0.35
0.3
A toy car: it rolls in the
direction of its principal axis
A bottle: it rolls orthogonal to the
direction of its principal axis
A toy cube: it doesn’t roll,
it doesn’t have a principal axis
A ball: it rolls,
it doesn’t have a principal axis
0.25
0.2
0.15
Rolls at right
angles to
principal axis
0.1
0.05
0
push away
Bottle, “pointiness”=0.13
0.4
0
10
20
30
40
50
60
70
80
90
difference between angle of motion and principal axis of object [degrees]
back slap
Motor vocabulary
Learn about object affordances
The affordance considered is the direction
of movement with respect to the object’s
principal axis
side tap
0.25
Clustering
Initial position
back slap
0.35
0.3
0.2
Final position
0.25
P(x)
P(x)
0.15
side tap
0.1
0.2
0.15
0.1
0.05
0.05
0
-200
-150
-100
-50
0
50
100
150
0
-200
200
direction of movement [degrees]
-150
back slap
-100
-50
0
50
100
150
200
direction of movement [degrees]
Learn the “geometry” of poking
E.g. poking from the left causes the object to move to the right
3
Initial position
Exploiting affordances during poking
When a known object is presented to the
robot, it chooses the appropriate action to
make the object roll
Interpreting observations
A foreign manipulator (human) pokes an object
The direction of movement is compared with
the object afforadance
Final position
Example 1
Initial position
Final position
search rotation
Example 2
identify
and
localize
object
Previously-poked
prototypes
4
Mimicry
The robot mimics the observed action trying to fulfill the goal
rather than an actual movement
Acknowledgments:
This work is funded by DARPA as part of the “Natural Tasking of Robots Based on Human Interaction Cues” project under contract number DABT 63-00-C-10102.