Transcript Syntax-07-08-29 - Tulane University

Actions in language
and in the brain
Computational Neuroscience
NSCI 492
Spring 2008
Course organization
• Syllabus at
http://www.tulane.edu/~howard/CompNSCI/
2/27/08
Harry Howard, NSCI 492, Tulane University
2
Intro
Motor units
and
motor unit scheduling
A distinction
• We find it convenient to divide motor
processing into two types
– An ontology of motor units
– A mechanism for scheduling motor units
• We start with the former and draw
inspiration from the linguistic encoding of
motor ontology
2/27/08
Harry Howard, NSCI 492, Tulane University
4
The motor ontology in
language
Aspect and actionality
Actionality
•
Though there had been some previous research, Vendler (1967)
began the work of categorizing (English) verbs according to their
internal dynamic structure:
a)
b)
c)
d)
•
•
2/27/08
STATE: John knows Russian.
ACTIVITY: John walked in the garden.
ACCOMPLISHMENT: John ate an apple.
ACHIEVEMENT: John reached the summit.
There are unfortunately many names for such classes, such as
Aktionsart, aspectual class, and eventuality, to name a few of the
most popular.
We follow Tatevosov (2002) in using the more descriptive term
actional class, so that the phenomena itself is known as actionality.
Harry Howard, NSCI 492, Tulane University
6
Actionality diagnostics
•
•
One of the ways to decide which class a given verb belongs to is to check its
behavior in certain constructions, especially those of an aspectual nature.
In English, for example, the progressive aspect, be V+ing, distinguishes states
from the other three:
a)
b)
c)
d)
•
•
•
*John is knowing Russian.
John is walking in the garden.
John is eating an apple.
John is reaching the summit.
States (a) are considerably less grammatical than the other three classes in this
construction – hence the asterisk marking (a) as deviant.
Achievements (d) can be distinguished from activities (b) and
accomplishments (c) in that there is still some crucial part of the achievement
that has not been attained yet, e.g. actually arriving at the summit.
Activities and accomplishments are more homogenous in this respect, since
they appear to lack any such goal.
2/27/08
Harry Howard, NSCI 492, Tulane University
7
Why actionality?
• We only know of one account that both claims to
be the correct description of aspect and actionality
and explains why language encodes it.
• It is the theory of Narayanan (1997) based on
biological motor control theory:
– Aspectual expressions are linguistic devices referring to
schematized generalizations that recur in process
monitoring and control (such as inception, interruption,
termination, iteration, enabling, completion, force, and
effort).
2/27/08
Harry Howard, NSCI 492, Tulane University
8
X-schema for actionality
• The representation that Narayanan and collaborators
employ for verbs and the eventualities they describe is an
elaboration of the Petri net formalism, see Reisig (1985).
• A Petri net is a weighted graph that consists of circles
representing places and rectangles representing transitions
connected to one another by directed input and output arcs.
• Narayanan uses two basic types, one for individual verbs
and another for generalizations over verb types.
– A network for the verb phrase walk(to store) is
reproduced on the next slide.
2/27/08
Harry Howard, NSCI 492, Tulane University
9
walk(to store)
Fig. 1
2/27/08
Harry Howard, NSCI 492, Tulane University
10
Verbal x-schemas
• A black dot or number over a place converts it into
a resource arc, which specifies a distribution of
tokens measuring energy or force.
• A transition is enabled when its input places are
marked so that it can fire by moving tokens from
input to output places.
• In the schema shown in the figure, the energy
place must have at least w tokens before the “start”
transition fires.
2/27/08
Harry Howard, NSCI 492, Tulane University
11
Executing (or x-) schema
• Narayanan et al. extend this formalism in
several ways and consequently name their
augmented notation ‘executing (or x-)
schema’.
2/27/08
Harry Howard, NSCI 492, Tulane University
12
Arc types
• One addition is the ability to type arcs,
which permits places to represent a variety
of conditions on transitions.
• Typed arcs themselves result in no transfer
of tokens.
• There are two sorts.
2/27/08
Harry Howard, NSCI 492, Tulane University
13
Inhibitory arcs
• An inhibitory arc, designated by a
connection ending in a circle, allows a place
to disable a transition.
• The only instance of this notation in Fig. 1
is the connection emanating from the left of
the “goal=at(store)” place, which prevents
walking from starting if the destination has
already been reached.
2/27/08
Harry Howard, NSCI 492, Tulane University
14
Enable arcs
• An enable arc, designated by a connection lacking a circle
or arrowhead, makes a transition possible.
• There are several in Fig. 1.
– At the beginning of the network, i.e. at the far left, the places
“ok(vision)” and “upright” enable the transition to the “ready”
place. In this particular case, both places have resources, and so the
transition is indeed enabled.
– The “goal=at(store)” place is a precondition to finishing the action.
• With the addition of these typed arcs, a transition only fires
if all enable arcs are marked, and all inhibitory arcs are
unmarked.
2/27/08
Harry Howard, NSCI 492, Tulane University
15
Hierarchical control
• Hierarchical control permits the decomposition of
transitions into sub-schemas.
• In Fig. 1, the motion of walking is ensconced in
the hexagonal transition at the bottom, which
contains a decomposition into a sequence of steps
that could in turn be further decomposed into a
stance and a swing phase.
2/27/08
Harry Howard, NSCI 492, Tulane University
16
Other additions
• There are a few other additions to the Petri
net formalism proposed by Narayanan to
aid in linguistic description, but most are
not necessary for this short introduction.
2/27/08
Harry Howard, NSCI 492, Tulane University
17
Controller x-schemas
• It is not uncommon that a particular precondition
must be satisfied before an event is ready to start,
and even then it may be canceled.
• While the event is going on, it may iterate many
times before it is done, as in Fig. 1.
• Regularities in the evolution of complex events
such as these are captured by a special type of xschema illustrated in Fig. 2 called a controller xschema.
2/27/08
Harry Howard, NSCI 492, Tulane University
18
Controller x-schema
Fig. 2
2/27/08
Harry Howard, NSCI 492, Tulane University
19
Actional composition
• The controller is itself an active x-schema
that interacts with both a verb schema and a
state of the world.
• The next slide, Fig. 3, shows how a subnetwork of the controller can be bound to
different phases of the walk x-schema.
2/27/08
Harry Howard, NSCI 492, Tulane University
20
Diagram of three bindings
Fig. 3
2/27/08
Harry Howard, NSCI 492, Tulane University
21
The bindings as three aspects
• The semantics of actional composition
arises from such dynamic bindings.
• Three phasal aspects that can be read off of
the bindings in Fig. 3, where ‘<=>’ marks
the binding:
a) prospective aspect: ready <=> ok(vision), etc.
b) progressive aspect: ongoing <=> walk schema
c) perfect aspect: done <=> at(store), etc.
2/27/08
Harry Howard, NSCI 492, Tulane University
22
The bindings in prose
• Prospective aspect in (a) binds the ready place
with the preconditions that the agent is upright and
has enough energy to walk, etc.
• Progressive aspect in (b) binds the ongoing place
with the steps to be taken in walking.
• Perfect aspect in (c) binds the done place with the
‘post-conditions’ of being at the store, having used
energy in the walk, etc.
2/27/08
Harry Howard, NSCI 492, Tulane University
23
Problems for x-schema
• While we find it highly likely that x-schema are sufficiently expressive
to describe a host of interesting data on actionality and aspect – though
this is an issue which we wish to pursue as part of the project – where
we envision problems is in their explanatory adequacy.
• The first red flag is that all of Narayanan’s references in support of the
contention that Petri nets simulate ‘biological control’ are from the mid
1970’s.
• A search of “Petri net” on PubMed [1/19/05] does not uncover any
contemporary reference from the biological control of movement.
• Instead, the hits come from research on biochemical pathways or gene
expression, if not medical informatics.
2/27/08
Harry Howard, NSCI 492, Tulane University
24
Motor processing in the brain
Slow vs. fast motion
X-schema and the current
perspective on biological control
• The contemporary literature on the control of natural motion speaks in
terms of a rather different sort and tends to fall on either side of a
natural temporal divide.
• The temporal divide springs from the observation that delays on
sensory feedback lie in the range of 100 to 200 ms. depending on the
modality, see Kawato (2003:190).
– If a motion is slow enough, then such a delay will not destabilize its
execution, and it can be controlled exclusively by sensory feedback.
– However, a wide variety of smooth and coordinated movements are not
slow enough and so must be controlled by some other mechanism.
• Let us call motion that can or cannot be controlled by sensory feedback
‘slow’ and ‘fast’ motion, respectively.
2/27/08
Harry Howard, NSCI 492, Tulane University
26
Slow motion and the cortical
perception-action cycle
• Slow motion is thought to be processed in the
cerebral cortex in a hierarchy of cortical areas
integrated so as to permit the circular flow of
information
–
–
–
–
–
–
2/27/08
from the environment
to sensory structures,
to motor structures,
back again to the environment,
to sensory structures,
and so on.
Harry Howard, NSCI 492, Tulane University
27
The perception-action cycle
• Called the perception-action cycle, Fig. 4
elaborates a visualization of it popularized by
Joaquin Fuster, Fuster (2003a, 2003b, 2004),
though its first sketch goes back at least to
Lichtheim (1885).
• Unlabeled boxes represent sub-areas of labeled
areas or areas interposed between them.
• The arrows depict connections that have been
found in the monkey.
2/27/08
Harry Howard, NSCI 492, Tulane University
28
The perception-action cycle
Fig. 4
2/27/08
Harry Howard, NSCI 492, Tulane University
29
A hierarchy of abstraction
• The figure shows increasing abstractness of
processing within a hierarchy and sensorimotor
integration between hierarchies at the same level.
• As Fuster (2003a:909) explains,
The neural operations of the perception-action cycle
ensure that mutually contingent percepts and acts are
properly integrated in the progression of behavior toward
its goal. The attainment of that goal requires the
attainment of lesser or subordinate goals, each dependent
on particular translation of perception to action and its
consequent change in the environment.
2/27/08
Harry Howard, NSCI 492, Tulane University
30
Routinization
• The height of processing of a behavior depends on
its familiarity.
• If it is new, all levels of the cycle participate.
• New behaviors become routine and automatic by
progressive relegation to lower levels, though
higher levels can still be called upon if the
behavior contains uncertainty or ambiguity.
2/27/08
Harry Howard, NSCI 492, Tulane University
31
Memory
• Fig. 4 appends the alignment of memory to its cortical
substrate discussed more recently in Fuster (2003b, 2004).
• Generalizing across both hierarchies,
– the bottom rung constitutes innate or phyletic ‘memory’, acquired
by evolution and available to instant deployment, as sensation or
movement.
– As an individual acquires new experience, it is added to the
established base in such a way that more complex and abstract
aspects of knowledge and more complex schemas or plans of goaldirected action come to be stored in hierarchically higher cortices
2/27/08
Harry Howard, NSCI 492, Tulane University
32
Fast motion and internal models
• Turning our attention now to fast motion, that is, motion
that cannot be controlled effectively by sensorimotor
feedback, we find a rather different perspective.
• One popular proposal is that the brain predicts the path of
motion – a prediction known as an internal model – and
incorporates it with sensory feedback to control the
motion.
• The internal model can be treated as an error signal, so that
motion can be learned by an error-correction algorithm.
• Internal models and their error correction signals are
thought to be housed in the cerebellum, see among others
Miall (2003) for a short review.
2/27/08
Harry Howard, NSCI 492, Tulane University
33
Comparison to x-schema
• Slow motion is roughly compatible with x-schema.
– The parts of verbal x-schema are 'acts' at the level above motor
cortex in the the perception-action cycle.
– Verbal X-schema are 'programs' at the next level.
– Aspectual morphology should are 'plans' or 'conceptualizations' at
the highest level.
– Somehow programs and plans are bound in actional composition
• Fast motion is very different from x-schema
– It at best corresponds to something like the transition of taking a
single step in Fig. 1, or perhaps 'acts' in the p-a cycle
• Thus it is rather misleading for x-schema to claim support
from biological motor control, at least without explanation.
2/27/08
Harry Howard, NSCI 492, Tulane University
34
Action scheduling
Action selection and the basal ganglia
Action selection
• There is one aspect of x-schema/Petri nets
that is more compatible with motor
processing.
• Narayanan himself wrote a paper proposing
Petri nets as a model for the basal ganglia.
2/27/08
Harry Howard, NSCI 492, Tulane University
36
Basal ganglia
•
The basal ganglia are a group of nuclei in the brain interconnected with the cerebral
cortex, thalamus and brainstem. Mammalian basal ganglia are associated with a variety
of functions: motor control, cognition, emotion, and learning.
2/27/08
Harry Howard, NSCI 492, Tulane University
37
What does it do?
(Gurney et al. 2004)
• … the brain is processing, in parallel, a large number of sensory,
cognitive and motivational streams or channels, each of which may be
‘requesting/promoting’ different actions to be taken.
• For effective use of limited motor resources, it is necessary to suppress
the majority of these requests while allowing the expression of only a
small number (in some cases just one).
• This channel-based scheme is consonant with the view that basal
ganglia comprise a series of afferent and efferent parallel processing
streams or loops
• However, Simon Levy’s idea of a process that serializes the unordered
contents of an associative memory works just as well
2/27/08
Harry Howard, NSCI 492, Tulane University
38
Summary
• The action-perception hierarchy provides an
ontology of motor 'units' (acts < programs <
plans < concepts)
• The basal ganglia select among competing
motor units for execution ('scheduling')
• But note that motor units themselves have
parts that need 'scheduling', so there is
potential recursion.
2/27/08
Harry Howard, NSCI 492, Tulane University
39
Next time
• Two papers that bring a lot of this together in a
Kephera simulation
– Gurney, K.N. et al., 2004, Computational models of the
basal ganglia: from robots to membranes, Trends in
Neurosciences, 27, 453-459.
– Prescott, T.J. et al., 2006, A robot model of the basal
ganglia: behavior and intrinsic processing, Neural
Networks, 19, 31-61.Matlab code for former:
http://senselab.med.yale.edu/ModelDB/ShowModel.asp
?model=83560
• Order Kephera parts from Randy's grant
2/27/08
Harry Howard, NSCI 492, Tulane University
40