Transcript USC Brain Project Specific Aims

Laurent Itti:
CS564 - Brain Theory and Artificial Intelligence
Lecture 7. Didday Model of Winner-Take-All
Reading Assignments:
TMB2:
4.3, pp. 194-197. Prey Selection - or Winner Takes All
4.4. A Mathematical Analysis of Neural Competition
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Didday Prey-Selector
1
The Prey-Selector Model of Didday
Consider how the frog's brain might select
one of several visually presented prey objects.
The task: to design a distributed network (not a serial scan strategy) that
could take a position-tagged "foodness array" and ensure that usually
the strongest region of activity would influence the motor control
system.
The prey-selector: an early, biological example of what connectionists
call a winner-take-all network.
Didday’s model adds a relative foodness layer of cells in topographic
correspondence to the retinotopic foodness layer.
The new layer yields the input to the motor circuitry.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Didday Prey-Selector
2
Winner-take-all Networks
Goal: given an array of inputs, enhance the strongest (or strongest few)
and suppress the others
No clear strong input yields
global suppression
Strongest input is enhanced
and suppresses other inputs
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Didday Prey-Selector
3
Winner-Take-All Networks
strong stimulus
Basic architecture:
- topographically organized
layer of neurons
- each neuron receives excitatory
input (e.g., sensory)
- each neuron also sends inhibition
to neighboring neurons,
possibly via an inhibitory
inter-neuron
weak stimulus
a
b
Example of 2-neuron WTA
with spiking neurons
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Didday Prey-Selector
4
Didday’s Model
= inhibitory inter-neurons
retinotopic
input
= copy of input
= receives excitation
from foodness layer
and inhibition from
S-cells
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Didday Prey-Selector
5
Didday's transformation scheme
from foodness to relative-foodness
It uses a population of S “sameness“
cells in topographic correspondence
with the other layers.
Each S-cell inhibits the activity that
cells in its region of the relativefoodness layer receive from the
corresponding cells in the foodness
layer by an amount that augments
with increasing activity outside its
particular region.
This ensures that high activity in a
region of the foodness layer
penetrates only if the surrounding
areas do not contain sufficiently high
activity to block it.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Didday Prey-Selector
6
Hysteresis
How does the network respond to a
change in the maintained stimulus pattern?
Once in equilibrium, one may increase a
non-maximal stimulus s2 so that it becomes
larger than the previously largest stimulus s1,
yet not switch activity to the corresponding element.
In neural networks with loops - an internal state resists dependence on
input: buildup of excitation and inhibition precludes the system's quick
response to new stimuli.
For example, if one of two very active regions were to suddenly become
more active, then the deadlock should be broken quickly.
In the network so far described, however, the new activity cannot easily
break through the excitation of its competitor and the global inhibition:
there is hysteresis.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Didday Prey-Selector
7
Hysteresis in a Ferromagnet
In a ferromagnetic
material, the atoms act
like magnets which tend
to align themselves with
their neighbors -- this
cooperation is opposed
by thermal noise.
A ferromagnet becomes
magnetized when the
atomic magnets have
high probability of
pointing in the same
direction.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Not a single function
magnetic field  direction
but a history-dependent
relation.
Direction
of External
Magnetic
Field
Didday Prey-Selector
8
Didday's Transformation Scheme
From Foodness to Relative-Foodness
Didday uses a local mechanism,
to combat hysteresis,
introducing a Newness cell "N-cell" for each S-cell to
monitor temporal changes in
the activity of its region.
For a dramatic increase in the
region's activity, it overrides the
inhibition on the S-cell and
permits this new level of
activity to enter the relative
foodness layer.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Didday Prey-Selector
9
Formalizing: An "N-cell" for each S-cell
Introducing an "N-cell" for each S-cell
Formally: neurons Ni have activity ki proportional to dSi/dt.
We then postulate that the
input to the ith element to
our array is no longer si
but si + ki.
If ki is large enough for
long enough, the relative
foodness Ei can gain
ascendancy.
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Si
dSi/dt
Didday Prey-Selector
10
Formalizing Didday's Model
The rest of this lecture provides a mathematical analysis
from TMB2, Section 4.4, and is based on the paper:
Arbib, M.A., and Amari, S.I., (1977) Competition and Cooperation in
Neural Nets, in Systems Neuroscience (J. Metzler, Ed.), New York:
Academic Press, pp. 119-165.
To see the mathematics, switch to the Word file:
7W. Amari-Arbib Mathematics
Laurent Itti: CS564 - Brain Theory and Artificial Intelligence.
Didday Prey-Selector
11