Transcript No Slide Title

Michael Arbib: CS564 - Brain Theory and Artificial Intelligence
University of Southern California, Fall 2001
Lecture 18.
The FARS model of Control of Reaching and Grasping
Reading Assignments:
Motor Schemas and Cortical Regions:
TMB 2, Sections 2.2, 5.3, 6.3*
FARS Model:
Fagg, A. H., and Arbib, M. A., 1998, Modeling Parietal-Premotor
Interactions in Primate Control of Grasping, Neural Networks,
11:1277-1303.
* Caution: Most of the neuroanatomy in 6.3 is still reliable, but much research has updated our
understanding of cortical correlates of motor control since 1989.
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
1
Perceptual And Motor Schemas
A perceptual schema embodies the process whereby the system
determines whether a given domain of interaction is present in the
environment. {Recall our discussion of VISIONS, TMB2 §5.2}
A schema assemblage combines an estimate of environmental state
with a representation of goals and needs
The internal state is also updated by knowledge of the state of
execution of current plans made up of motor schemas
which are akin to control systems but distinguished by the fact that they
can be combined to form coordinated control programs
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
2
Preshaping While Reaching to Grasp
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
3
Hypothetical coordinated control program for reaching
and grasping
r ecogniti on
crit er ia
Perceptual
Schemas
visual
input
Vis ual
Location
act ivat ion of
visual sear ch
tar get
locat ion
Size
Recognit ion
s ize
act ivat ion
of r eachi ng
Motor
Schemas
visual
input
Hand
P reshape
Slow P has e
Movement
Hand Reachi ng
Ori entati on
Recognit ion
ori entati on visual ,
kines thet ic, and
tactil e input
visual and
kines thet ic input
Fas t P has e
Movement
visual
input
Hand
Rot at ion
Act ual
Grasp
Graspi ng
Dashed lines — activation signals; solid lines — transfer of data.
(Adapted from Arbib 1981)
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
4
"What" versus "How” in Human
DF: Jeannerod et al.
Lesion here: Inability to Preshape
(except for objects with size “in the semantics”
reach programming
Parietal
Cortex
How (dorsal)
grasp programming
Visual
Cortex
Monkey Data:
Mishkin and
Ungerleider on
“What” versus
“Where”
Inferotemporal
Cortex
What (ventral)
AT: Goodale and Milner
Lesion here: Inability to verbalize or
pantomime size or orientation
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
5
Reaching for an object by a patient
with a lesion of the parietal cortex:
Jeannerod, M., Michel, F., Prablanc, C.,
1984, The Control of Hand Movements in a
Case of Hemianaesthesia Following a
Parietal Lesion, Brain107:899-920.
Consider the implications for Project
1.
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
6
Virtual Fingers
Arbib,
Iberall and
Lyons
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
7
Opposition Spaces and Virtual Fingers
The goal of a successful
preshape, reach and grasp
is to match the opposition
axis defined by the virtual
fingers of the hand with
the opposition axis defined
by an affordance of the
object
(Iberall and Arbib 1990)
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
8
Planning for the reach
must take account of
the planned grasp
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
9
Somatosensory areas
SMA
FEF
(saccades)
SMA =
pre-SMA +
SMA-proper
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
10
Somatosensory data: A key to motor control
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
11
Do data support the idea of virtual fingers?
Iberall and Arbib 1988 suggest that
multiple, seemingly non-somatotopic,
representations of the digits could be
due to a virtual finger representation.
Using the tentative identifications:
VF1 involves palm and thumb areas
VF2 involves the index with or without
other fingers
VF3 involves finger combinations
excluding the thumb and the index
yields a possible mapping of virtual
fingers onto the caudal kinesthetic map
that Strick and Preston (1962) found in
the squirrel monkey.
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
12
Iberall and Arbib’s 1988 view of cortical
contributions to the coordinated control
program for reaching and grasping.
Use it as an evaluation point as we develop
the FARS and MNS models. What do we
gain, what have we lost?
See TMB2 §6.3
for the details.
Introducing AIP and F5 (Grasping) in Monkey
A key theme of
visuomotor coordination:
parietal affordances
(AIP)
drive
frontal motor
schemas
(F5)
F5 - grasp
commands in
premotor cortex
Giacomo Rizzolatti
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
AIP - grasp
affordances
in parietal cortex
Hideo Sakata
14
Grasp Specificity in an F5 Neuron
Precision pinch (top)
Power grasp (bottom)
(Data from Rizzolatti et
al.)
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
15
The Sakata Protocol
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
16
Grip Selectivity in a Single AIP Cell
A cell that is
selective for side
opposition
(Sakata)
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
17
Differential Timing of Activity Peaks in
Different AIP Neurons
Note the need for a broad database of many cells within
each region to see that cells are not just “pattern
recognizers” but also have a relationship to the time
course of the ongoing behavior.
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
18
Size Specificity in a Single AIP Cell
This cell is selective toward small objects, somewhat independent of
object type ( Hideo Sakata)
Note: Some cells show size specificity; others do not.
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
19
FARS (Fagg-Arbib-Rizzolatti-Sakata) Model
Overview
AIP extracts the set of
affordances for an attended
object.These affordances
highlight the features of the
object relevant to physical
interaction with it.
AIP
AIP
Dorsal
Stream:
dorsal/ventral
Affordances
streams
Ways to grab
this “thing”
Task Constraints
Task
(F6)Constra ints ( F6)
Working Memory
W orking Me mory (46)
(46?)
Instruction Stimuli
Instruction Stim uli (F2)
(F2)
PFC
Ventral
Stream:
Recognition
F5
“It’s a mug”
IT
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
20
Secondary Somatosensory Cortex (SII)
In the grasp versus point comparison
in a PET study of humans, we found a marked
increase of activity in the
secondary somatosensory cortex (SII).
Ablation of SII in non-human primates results in decrements in tactile
discrimination and impaired tactile learning.
Focal lesions of the parietal operculum in humans characteristically
produce tactile agnosia without loss of simple tactile sensation, or
motor control. This deficit can include
the inability to sort objects based on size or shape,
although sorting on texture is preserved.
The model relates the augmented response to
higher order tactile feedback or tactile expectation.
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
21
Motor Commands, Expectations, and Feeedback
(grasp type) F5
expectation
(sensory
hyperfeatures) SII
A7
(internal model)
MI
(muscle assemblies)
motor commands
SI
(elementary
sensory
features)
sensory info
hand
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
22
Interaction of AIP and F5 During the Sakata Task
Vis ual Input s
Activation Connection
Inhibitory Connection
Priming Connection
AIP
visual
AIP precision-related cell
AIP power-related cell
mot or
F5
P recis ion
P inch
set
Go Si gnal
extens ion
maximum
aperture
reached
fl exion
contact
wit h object
hol d
releas e
2nd Go Si gnal
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
23
The Problem of Serial Order in Behavior
(Karl Lashley)
If we tried to learn a sequence like
A BAC
by reflex chaining, what is to stop A triggering B every time,
to yield the performance A  B  A  B  A  …..
(or we might get A  B+C  A  B+C  A  …..)
A solution: Store the “action codes” (motor schemas) A, B, C, … in one part of
the brain (F5 in FARS) and have another area (pre-SMA in FARS) hold
“abstract sequences” and learn to pair the right action with each element:
(pre-SMA): x1  x2  x3  x4 abstract sequence
(F5):
A
B
C
action codes/motor schemas
Hypothesis: The “Sakata-Protocol Sequencing” is not mediated
within F5 --Sequences are stored in pre-SMA and administered
by the Basal Ganglia (BG)
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
24
Basal Ganglia Anatomy in the Rat
From Prescott et al., HBTNN 2e, to appear
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
25
A 2-Function View of the Basal Ganglia Skeletomotor Pathway
Cortex
Putamen
N ext
S enso ry
S tate
In h ib it
Mo v emen t
GPe
SN c
Indirect
Pathway
Direct
Pathway
STN
GPi
VLo
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
26
Bischoff-Grethe Sequencing Model
Pre-SMA
SMA-Proper
SMA-Proper
Motor Cortex
Basal Ganglia
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
27
The “Visual Front End” of the FARS Model
F4
VIP
(arm goal position)
Parietal
Cortex
(position)
How (dorsal)
(object/grasp transform)
F5
AIP
PIP
(shape, size, orientation)
(grasp type)
Visual
Cortex
IT
What (ventral)
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
28
Positioning F2, F6 and Areas 46 and SII in
Monkey
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
29
Prefrontal Influences on F5
pre-SMA
Frontal Cortex
F6
Inferior
Premotor
Cortex
F4
(arm goal position)
46
(grasp type)
Dorsal
premotor
cortex
F5
F2
(abstract
stimuli)
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
30
The Complete FARS Model
Inferior
Premotor
Cortex
F6
F4
VIP
( ar m goa l position)
Parietal
Cortex
( position)
How (dorsa l)
( obj e ct/gra sp tr a nsf or m )
46
AIP
( gr a sp ty pe) F5
PIP
( sha pe, size, or ie nta tion)
Visual
Cortex
F2
( abstr ac t
stim uli)
IT
W ha t (ve ntr a l)
e xpe c ta tion
( se nsor y
hy per fe a tur e s)
A7
( inte r na l m ode l)
SII
MI
( m usc le assem blie s)
SI
m otor c om m ands
( ele m e nta ry
sensory
f ea tur e s)
sensory info
hand
Michael Arbib CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture
31