Transcript info day 29/9/97 MEL

Image Segmentation by Complex-Valued Units
Cornelius Weber and Stefan Wermter
Hybrid Intelligent Systems
School of Computing and Technology
University of Sunderland
ICANN Conference, September 2005
Contents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Oscillators for Spike Phases
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Contents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Oscillators for Spike Phases
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Attractor Network:
Competition via Relaxation
rate profile
weight profile
rate update
ri(t+1) = f (Σij wij rj(t))
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Response Characteristics
linear
sparse
competitive
winner
Project
funded
by the Future and Emerging
TechnologiesMaps,
arm of the
IST Programme
Weber
, C.
Self-Organization
of Orientation
Lateral
Connections, and Dynamic
FET-Open
scheme
Receptive Fields in the Primary Visual Cortex. Proc. ICANN (2001)
Learning Object Recognition
apple
background
green
red
attractor network
Binding- and learning problem?
Active units (features) not separated
Learning objects in cluttered background
is difficult
Project funded by
the Future S.M.
and Emerging
Technologies
arm of the IST
Programme
Stringer,
and Rolls,
E.T. Position
invariant
recognition in the visual
FET-Open scheme
system with cluttered environments. Neural Networks 13, 305-15 (2000)
Contents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Oscillators for Spike Phases
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Neckar Cube
Attractor networks that minimize an energy function
do not account for bi-stability
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Neuronal Spike Chaos
A wide range of spiking neuron models displays
three distinct categories of behaviour:
- quiescence
- intense periodic seizure-like activity
- sustained chaos in normal operational conditions
Banerjee, A. On the Phase-Space Dynamics of Systems of Spiking
Neurons. I: Model and Experiments. Neural Computation, 13(1), 161-93
(2001)
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Neuronal Synchrony
“cortical neurons often engage in oscillatory
activity which is not stimulus locked but caused by
W. Synchronization, Bining and Expectancy. In: The
internal interactions” Singer,
Handbook of Brain Theory and Neural Networks, pp. 1136-43
(2003)
“activity synchronization was present in the
expectation period before stimulus presentation and
could not be induced de novo by the stimulus”
Cardoso de Oliviera, S., Thiele, A. and Hoffmann, K.P.
Synchronization of neuronal activity during stimulus expectation
in a direction discrimination task. J. Neurosci., 17, 9248-60 (1997)
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Neuronal Spike Chaos
We need a method to:
- create patterns of synchronization
- avoid long-term stabilization (bi-stability is welcome!)
van Leeuven, C., Steyvers, M. and Nooter, M. Stability and Intermittency in
Large-Scale Coupled Oscillator Models for Perceptual Segmentation. J.
Mathematical Psychology, 41(4), 319-44 (1997)
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Contents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Oscillators for Spike Phases
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Complex Number
i
z
r
r rate
φ phase
φ
1
z = r eiφ
= r cos φ + i r sin φ
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Deterministic Chaos
Logistic map:
Ф(t+1) = A Ф(t) (1- Ф(t))
Phase φ = 2π Ф
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Coupling of the Phases
“Net input” to neuron k:
For rates:
Σkj wkj rj
weighted field
}
}
For phases: Σkj wkj rj eiφj ≡ zkwf
complex number
coupling strength for phases
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Relaxation of the Phases
Compute “net input”: zkwf = Σkj wkj rj eiφj
From zwf, take phase φwf
Compute new phase: Фk(t+1) = A Фkwf(t) (1- Фkwf(t))
(remember: φ = 2π Ф)
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Relaxation of Rates and Phases
Phase of any neuron behaves chaotically
Coupled neurons have similar phases
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Phase Separation Histogram
Large phase differences at boundary of activation hill
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Toward Learning Object Recognition
attractor network
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Toward Learning Object Recognition
attractor network
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Contents
• Attractor Network which Converges
• Non-Convergence and Spike Synchrony
• Coupled Oscillators for Spike Phases
• Outlook
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme
Plans and Questions
- The higher hierarchical level shall benefit!
- Should the rates depend on the phases?
→ This would influence learning!
- Learning with Phase Timing Dependent Plasticity?
Project funded by the Future and Emerging Technologies arm of the IST Programme
FET-Open scheme