Transcript Group10-ClassPresentation

FROM NEURONS TO BRAINS TO
NEURAL NETWORK MODELS
I1
I2
I3
 +  +  +
1


2

+-
x1
3
- x- +- x-- +
2
3
THE UNIFYING THEME OF CELEST
CURRICULUM: METACOGNITION
LEARNING ABOUT LEARNING
THINKING ABOUT THINKING
A focus on neuroscience is a novel and compelling
approach to learning because it explicitly focuses on
human perception and learning
Teaches students various study strategies while
instructing students in a variety of critical math and
science skills
(c) CELEST 2007
2
FROM NEURONS…
Anatomy, morphology, physiology, specialization…
(c) CELEST 2007
3
Neurons and the Synapse
How Neurons transmit an action
potential and how the synapse
works
The Beginning
Any thought, experience, or action that you do can be
known as a stimulus. Those stimuli generate nerve
impulses.
For example, when you see something light is reflected
off a surface and enters your eye. Then it stimulates the
retina’s photoreceptors which begins stimulating the
nerves to create an electric impulse. That goes to the
neurons.
Photoreceptors
on the retina
http://reu.uwosh.edu/vaughan.php
(c) CELEST 2007
5
ANATOMY OF A NEURON
Nerve impulse travels along Nerve cells
otherwise known as neurons. These
neurons have many parts which are
involved in the transmission of the cell.
Here are the parts to a neuron that are
present in the transmission of this
impulse
Dendrites- act to conduct the electrical
stimulation received from other neural
cells to the cell body
Cell Body- also known as the “soma” can vary in size depending
upon the type of neuron. It also contains the nucleus
Nucleus- Is responsible for producing most of the RNA in the
Neurons and most proteins used by neurons are created by
mRNA, which can create structures such as ion channels.
(c) CELEST 2007
6
ANATOMY OF A NEURON 2
Axon- Conducts the electric impulse from
the cell body to the axon terminals
Myelin sheath- Is an insulating material
which prevents the electric impulse from
leaking allowing the impulse to travel rapidly.
Some types of neurons don’t have this.
Schwann Cell- also aids in the insulation allowing the electric
impulse to travel rapidly down the axon
Nodes of Ranvier- is the place where the electric impulse jumps
to in each cell to carry the impulse down the axon acting like an
electric amplifier.
Axon Terminal- Is the end of the axon which is part of the
chemical synapse.
(c) CELEST 2007
7
Types of Neurons
There are many types of Neurons here are four examples
of 4 more common ones.
Bipolar neurons are usually part of sensory path such as
smell, sight, taste, hearing and vestibular functions.
Unipolar neurons are also sensory neurons.
Multipolar neurons are the majority of the brains neurons
Pyramidal neuron are found in the hippocampus and
cerebral cortex.
(c) CELEST 2007
8
ANATOMY OF AN ACTION POTENTIAL
Cell #1
Dendrites
Axon
Axon
Terminals
Cell #2
The electrical impulse if large
enough becomes known as
an action potentials which is
used to communicate with
other neurons
An action potential occurs
when an electrical charge
travels down the axon from
the cell body to the axon
terminals through the Nodes
of Ranvier
(c) CELEST 2007
9
Action Potentials
Sodium ion flow
The impulse causes sodium
channels to open which allows
sodium ions to start flowing into
the neuron changing the charge
gradient causing cell
depolarization ( which means the
potential difference is rising).
The deplolarization eventually
reaches a threshold for starting
the action potential, which means that the neuron will fire and
more sodium channels open. The sodium ions continue to flow in
and the depolariztion continues. As it reaches its maximum
potential the sodium channels begin closing,
(c) CELEST 2007
10
Action Potentials
Potassium ion flow
And the potassium channels
begin opening, which allows
potassium ions to flow into
the cell.
This flow begins repolarization
and starts returning the
potential to the rest
potential.
However, channels stay open too long and the cell becomes
hyperpolarized. At cell cannot fire until the cells restpotential is
restored. This restoration is when the sodium potassium pump
along with the outflow of potassium restores the ion
concentrations to the beginning and the cell is ready to fire again
(c) CELEST 2007
11
Sodium/Potassium Pump
The pump acts to restore the
original Sodium ion and
Potassium ion concentration
because now The
concentration of the Sodium
ions inside the cell is to high
and the Potassium ion
concentration is to low. So,
The pump works by having
the pump works by pumping 3
ATP and 3 sodium ions bind to sodium ions in while 2
potassium ions are pumped
the pump. Then the ATP is
out.
hydrolyzed, which releases
ADP that causes a
comformational change in the
pump and the sodium ions are
exposed outside of the cell.
(c) CELEST 2007
12
Sodium/Potassium Pump
Two potassium ions then
bind with the pump
ATP binds to the pump
again causing it to
reorient and the
potassium ions are
released into the cell.
The process then starts all
over again. So the whole
process continues to
cycle until the rest
potential is restored.
(c) CELEST 2007
13
Action Potential 2
The graph shows the action
potential process in terms
of the electric impuse. It is
important to know that the
cell can depolarize in the
positive or negative
direction.
Getting a visual of this is very
important, so check out
these animations
(c) CELEST 2007
14
HOW NEURONS COMMUNICATE
Neurotransmitter
When the electric impulse
reaches the axon
Axon
terminals the electrical
Terminal
signal is converted to a
chemical signal
These chemical signal are
called neurotransmitters,
which can be either
excitatory or inhibitory
Synapse Neurotransmitters are
released from the axon
terminal through the
synapse to the dendrite
terminals of one or many
other cells
Dendrite Terminal
(c) CELEST 2007
15
Types of Neurotransmitters
The many different types of neurotransmitters are
contained within the vesicles. Each vesicle contains a
specific type of neurotransmitter. On the next slide is a
sample list of some of the more common
neurotransmitters and their functions. Some of them can
be excitatory which means that when they hit the
receptors it causes a depolarization on the post synaptic
neuron. (causing the peak on the graph to go up)
The other neurotransmitters are inhibitory which means
they hyperpolarize postsynaptic neuron. (causing the
peak on the graph to go down)
(c) CELEST 2007
16
Neurotranmitters 2
Acetylcholine - voluntary movement of the muscles
mostly excitatory
Norepinephine- wakefulness or arousal, excitatory
Dopamine - voluntary movement and motivation,
"wanting" , excitatory or inhibitory
Serotonin - memory, emotions, wakefulness, sleep and
temperature regulation, excitatory
GABA (gamma aminobutyric acid) - inhibition of motor
neurons, inhibitor
Glycine- spinal reflexes and motor behavior,
mostly inhibitory
(c) CELEST 2007
17
SYNAPSE
Most synapses are unidirectional: one
neuron sends a neurotransmitter to
the other at that synapse across the
synaptic cleft, but not the other way
around.
The neuron who sends the
Synaptic cleft
neurotransmitter is called the
presynaptic neuron.
The neuron who receives the
chemical messenger is called the
postsynaptic neuron.
More on Synapse
(c) CELEST 2007
18
Neurotransmitters
When the action potential reaches the synapse the
depolarization causes calcium ion channels to allow calcium
ions in which allows the vesicles to fuse with the membrane,
and the vesicles to release the neurotransmitters from the
presynaptic neuron axon terminal. The transmitter then can
bind with the postsynaptic dendrite arm receptors. This
binding then can begin the whole transmission process. The
receptors will then either release the neurotransmitter to be
recycled in a process called uptake or it will be broken down
by enzymes. Lets watch several clips.
(c) CELEST 2007
19
Synapse Clip 1
(c) CELEST 2007
20
Synapse Clip 2
(c) CELEST 2007
21
Synapse Clip 3
(c) CELEST 2007
22
EXCITING A POST-SYNAPTIC NEURON
The level of excitation
the postsynaptic
neuron can receive is a
function of how many
synaptic connections a
neuron’s dendrite has,
as well as how many
receptor sites there are
per synapse
(c) CELEST 2007
23
SIGNAL PROPAGATION
The whole circuit can be broken down
into a number of neurons and
synapses.
Each neuron is in a certain state of
activation.
This state of activation can be
transferred to another cell via
synapses.
(c) CELEST 2007
synapse
24
TO BRAINS…
Anatomy, morphology, physiology, specialization…
(c) CELEST 2007
25
THE BASIC PARTS OF THE BRAIN
MAP OF THE CORTEXES
(c) CELEST 2007
27
INTERNAL STRUCTURES
OF THE BRAIN
(c) CELEST 2007
28
YOUR “3-BRAINS IN ONE”
The Triune Brain
(c) CELEST 2007
29
“BRAIN 3”
(c) CELEST 2007
30
(c) CELEST 2007
31
“BRAIN 2”
• THE “OLD MAMMALIAN” BRAIN”
(c) CELEST 2007
32
(c) CELEST 2007
33
“BRAIN 1”
(c) CELEST 2007
34
REFERENCES
BOOKS, WEBSITES
References
•
•
•
•
•
•
•
•
•
•
•
•
•
Diamond, Marian. MAGIC TREES OF THE MIND.
Jensen, Eric.
TEACHING WITH THE BRAIN IN MIND
TEACHING WITH THE ARTS IN MIND
LeDoux, Joseph. THE SYNAPTIC SELF.
Ratey, John. A USER’S GUIDE TO THE BRAIN.
Wolfe, Patricia MIND MATTERS
Wolfe, Patricia BUILDING THE READING BRAIN
Brain website #1
Brain website #2
Brain website #3
Brain website#4
Brain website #5
Brain website #6
(c) CELEST 2007
36
TO NEURAL NETWORK MODELS…
Goal: represent, explain and predict reality (neuron,
neural mass, electronic properties, chemical
reactions, brain function, animal and human behavior)
Methods: directed graph, mathematical equation
Analysis technique: equilibrium analysis, simulations
with systematic parameter variation
Key concepts: constant and phased input, bottom up
activation, top-down priming, matching…
(c) CELEST 2007
37
MODELING HOW THE BRAIN LEARNS
A mature science of learning requires that
we understand how
BRAIN MECHANISMS
give rise to
BEHAVIORAL FUNCTIONS
(c) CELEST 2007
38
WHY IS IT IMPORTANT TO
LINK BRAIN TO BEHAVIOR?
Mind-Body Problem
Many groups study BRAIN OR BEHAVIOR
BRAIN provides MECHANISMS
BEHAVIOR provides FUNCTIONS
Without a link between them
BRAIN MECHANISMS have no FUNCTION
BEHAVIORAL FUNCTIONS have no MECHANISM
(c) CELEST 2007
CELEST provides this link!
39
HOW DOES THE BRAIN CONTROL
BEHAVIOR?
What level of brain organization
controls behavior?
What is the functional unit of behavior?
BRAIN evolution needs to achieve
BEHAVIORAL success
What level of BRAIN processing governs
BEHAVIORAL success?
40 years of modeling show:
The NETWORK and SYSTEM levels!
(c) CELEST 2007
40
BEHAVIOR IS AN EMERGENT
PROPERTY OF NEURAL NETWORKS
Does this mean that individual neurons are unimportant?
Not at all!
How are individual NEURONS designed and connected
so that the NETWORKS they comprise generate
emergent properties that govern successful BEHAVIORS?
Need to simultaneously describe 3 levels (at least):
BEHAVIOR
NETWORK
NEURON
and a MODELING language to link them
(c) CELEST 2007
CELEST studies all of these levels simultaneously
41
HOW MODELS LINK BRAIN TO BEHAVIOR
A successful MODELING APPROACH has unified these
levels during 40 years of modeling led by CELEST
scientists. In this approach, you analyse:
How an individual
adapts
on its own
in real time
to a complex and changing environment
REAL-TIME AUTONOMOUS LEARNING SYSTEMS!
This theme makes possible a MODELING CYCLE that can
link brain to behavior
(c) CELEST 2007
42
MODELING CYCLE
MIND
Design Principles
Behavioral
Data
Predictions
BRAIN
Neural
Data
Mathematical
and Computer
Analysis
Predictions
Technological Applications
(c) CELEST 2007
43
TWO KEY CONCLUSIONS
1. Advanced brains look like they do to enable
REAL-TIME AUTONOMOUS LEARNING
Lesson: The Architecture is the Algorithm
2. Recent models show how the brain’s ability to
DEVELOP and LEARN greatly constrain the laws of
ADULT INFORMATION PROCESSING
Lesson: You cannot fully understand adult
neural information processing without
studying how the brain LEARNS
(c) CELEST 2007
44
MODELS SERVE AS CELEST UNIFYING
THEMES
BEHAVIORAL AND BRAIN MODELING
of normal and abnormal LEARNING during
Perception
Cognition
Emotion
Action
Discovers MECHANISMS that control learning
(c) CELEST 2007
45
THINKING OUTSIDE THE BOX
Models are not only tied to data
How can we begin to know how the brain
works? Think about it!
Thought experiments are often used to
consider what must be true for particular
situations to exist
(c) CELEST 2007
46
EXAMPLE 1: STABILITY-PLASTICITY DILEMMA
(c) CELEST 2007
47
EXAMPLE 2: MASS ACTION
(c) CELEST 2007
48
EXAMPLE 3: THE GATED DIPOLE
(c) CELEST 2007
49
DESIGN OF LEARNING ENVIRONMENTS:
HOW PEOPLE LEARN
Focus on student’s previous
knowledge and misconceptions
Learner-centered
Knowledge-centered
Structured towards progressive
formalization of knowledge
Promote deep understanding and
subsequent transfer
Assessment-centered
Constant and interactive feedback
Community-centered
Universally relevant and applicable
topic, easily applied to everyday
experience and problem solving
(c) CELEST 2007
50
LEARNER-CENTERED
CELEST curriculum focuses on the learner creating a
deep understanding about how their brain works.
Example: BrightnessLab corrects misconceptions
about how vision works
(c) CELEST 2007
51
KNOWLEDGE-CENTERED
CELEST curriculum is knowledge centered
because it provides for progressive
formalization using a system of models that
range from those similar to everyday
experience to increasingly abstract
conceptual designs and mathematical
formalizations and analysis.
(c) CELEST 2007
52
KNOWLEDGE-CENTERED MODELING
Petrosino, 2003
EXAMPLE: BrightnessLab Models
(c) CELEST 2007
53
ASSESSMENT-CENTERED
CELEST curriculum promotes interaction between
students and their peers, students and their teacher,
and students and the computer
Student activities are designed to provide formative
feedback
Summative feedback activities are designed to test
students’ content-knowledge and provide an arena
to help students develop strategies to expand and
transfer their knowledge to solve a wider variety of
problems
(c) CELEST 2007
54
COMMUNITY-CENTERED
CELEST curriculum is universally relevant and
applicable to all people and readily transferred to
everyday experience
Everybody has a brain!
(c) CELEST 2007
55
WHAT WEB-BASED CURRICULUM EXISTS?
BrightnessLab:
Seeing is Believing / Brightness Contrast
Sequence Learning:
Make Your Memory Stronger!
Associative Learning:
Learning in the blink of an eye
Obstacle Avoidance Navigation:
Watch Where You’re Going!
Recognition:
How do we know what we know?
http://cns.bu.edu/CELEST/
(c) CELEST 2007
http://cns.bu.edu/CELEST/private
56
WHY WE BEGIN WITH THESE MODULES
Perception: the basis for knowledge about the world.
Half of the brain is dedicated to visual processing.
BrightnessLab begins the systematic study of visual
processing
Action: reflex and planned movements. Given a goal,
how do we decide to move as a reaction to sensory
(visual) input? Obstacle Avoidance Navigation
explores the question of reactive movement as
opposed to memory guided movement
Cognition: how we know that we know. We begin the
exploration of memory and learning by studying
Sequence Learning of numerical lists and continue
through an examination of Recognition and
metacognition
Emotion: spontaneous physical and mental states.
Since perception, action & cognition are all mediated
by emotions, we begin to address them by studying
adaptive timing, a basic
of Associative Learning
(c) CELESTform
2007
57
COMMON PRINCIPLES
Laminar or layered organization
Parallel and interaction processing streams
Activation (excitatory and inhibitory) has a limit
Activation will naturally (passively) decay
(c) CELEST 2007
58
BREAKTHROUGHS IN BRAIN COMPUTING
Models that link detailed brain CIRCUITS to the
ADAPTIVE BEHAVIORS that they control
Mind/Body Problem
Describe NEW PARADIGMS for brain computing
INDEPENDENT MODULES
Computer Metaphor
COMPLEMENTARY COMPUTING
Brain as part of the physical world
LAMINAR COMPUTING
Why are all neocortical circuits laminar?
How do laminar circuits give rise to biological intelligence?
(c) CELEST 2007
59
Principles of
UNCERTAINTY and COMPLEMENTARITY
Multiple Parallel Processing Streams Exist
HIERARCHICAL INTRASTREAM INTERACTIONS
UNCERTAINTY PRINCIPLES operate at individual levels
Hierarchical interactions resolve uncertainty
PARALLEL INTERSTREAM INTERACTIONS
Each stream computes COMPLEMENTARY properties
Parallel interactions overcome complementary weaknesses
(c) CELEST 2007
ADAPTIVE BEHAVIOR = EMERGENT PROPERTIES
60
VISUAL BOUNDARY AND SURFACE
COMPUTATIONS ARE COMPLEMENTARY
Neon color spreading
BOUNDARY
COMPLETION
SURFACE
FILLING-IN
oriented
inward
(c) CELEST 2007
insensitive to
direction-of-contrast
unoriented
outward
sensitive to
direction-of-contrast
61
CELEST PROJECTS TO DEVELOP UNIFIED MODEL
OF HOW VISUAL CORTEX SEES
WHAT STREAM
WHERE STREAM
PFC
BOTTOM-UP
TOP-DOWN
HORIZONTAL
interactions
everywhere to
overcome
COMPLEMENTARY
WEAKNESSES
PFC
Object plans
and working
memory
Spatial plans
and working
memory
IT
PPC
Spatially invariant
object recognition and
attention
Spatial
attention and
tracking
V4
MST
3-D filling-in of
binocular
surfaces and
figure-ground
perception
Predictive
target
tracking and
background
suppression
Optic flow
navigation
and image
stabilization
V2
Not independent
modules
V2
Depthselective
capture and
filling-in of
monocular
surfaces
Boundarysurface
consistency
3-D boundary
completion
and
separation of
occluding
and occluded
boundaries
MT
Formotion
binding
Enhancement of
motion direction
and feature
tracking signals
V1
Monocular
doubleopponent
processing
Stereopsis
Motion
detection
(c) CELEST 2007
Retina
and LGN
Photodetection and discount illuminant
62
BIOLOGICAL TAKE HOME LESSONS
1. Need to model
PAIRS OF COMPLEMENTARY CORTICAL STREAMS
to compute
COMPLETE INFORMATION
about a changing world
2. Need
INTERACTING TEAMS OF SCIENTISTS
A CENTER!
to characterize the large
FUNCTIONAL BRAIN SYSTEMS
that control adaptive behavior
(c) CELEST 2007
63
COMPLEMENTARY STREAMS COOPERATE
TO COMPUTE COMPLETE INFORMATION
Perception-cognition-emotion-action systems use
several types of
MULTI-DIMENSIONAL
LEARNED INFORMATION FUSION
Multiple sources of partial information are combined
during learning
Complementary types of learning work together to solve
environmental problems
e.g., What-Where learned information fusion
CELEST thrusts are designed to model how this works
(c) CELEST 2007
64
CELEST MODELS COMPLETE BRAIN
SYSTEMS
Perception-cognition-emotion-action systems use
several types of
MULTI-DIMENSIONAL
LEARNED INFORMATION FUSION
Multiple sources of partial information are combined
during learning
Complementary types of learning work together to solve
environmental problems
e.g., What-Where learned information fusion
Not a future wish; a present coordinated research program
(c) CELEST 2007
65
WHY THESE PARTICULAR THRUSTS?
ORDINARY BEHAVIORS USE LARGE FUNCTIONAL
BRAIN SYSTEMS
Child’s task:
Visually find and pick up a stationary cup of milk to drink
THRUST
Spatially orient to the cup
See cup
Recognize cup
Want to pick cup up
Plan to pick cup up
Pick cup up
Where stream
What stream
What stream
What stream
What-Where stream
What-Where stream
3
1
1
3
3,5
1,3,5
This perception-cognition-emotion-action cycle
uses What-Where learned information fusion
(c) CELEST 2007
Need visual, temporal, parietal, prefrontal cortices...
66
WHY THESE PARTICULAR THRUSTS?
ORDINARY BEHAVIORS USE LARGE FUNCTIONAL
BRAIN SYSTEMS
Child’s task:
Orient to mother’s voice and say: “Mommy, give me milk”
THRUST
Hear mother’s voice
Recognize voice
Spatially orient to voice
Want to talk to mother
Plan to talk to mother
Talk to mother
What stream
What stream
Where stream
What stream
What-Where stream
What-Where stream
2
2
3
3
3,5
2,3,5
This perception-cognition-emotion-action cycle
also uses What-Where learned information fusion
(c) CELEST 2007
Need auditory, temporal, parietal, prefrontal cortices...
67
CELEST THRUSTS ENABLE MODELING OF
COMPLETE BRAIN SYSTEMS
Perception-cognition-emotion-action systems
enable the brain to learn adaptive behaviors
in real time within a changing world
Just as important for
developing new engineering systems that intelligently
process huge amounts of data in unpredictably
changing environments
providing insights into how to improve
learning in the classroom
(c) CELEST 2007
68
WHY IS THIS POSSIBLE NOW?
Recent models and modeling PARADIGMS
developed by CELEST scientists:
COMPLEMENTARY COMPUTING
and
LAMINAR COMPUTING
have begun to clarify how these large
functional brain systems
compute the sort of
complete information
that controls successful
adaptive behaviors
CELEST brings together personnel
and resources needed to take the next steps
(c) CELEST 2007
69
A NEURON-INSPIRED MODEL
Key:
xi
vi
zij
eij
xj
vj
(c) CELEST 2007
xi
Short-term memory traces
vi
Cell populations
eij
Axons
zij
Long-term memory traces
xj
Short-term memory traces
for the next neuron
vj
Cell populations
Source: S. Grossberg (1988). Nonlinear neural networks:
Principles, mechanisms, and architectures. Neural
Networks, 1, 17-61.
70
Source: http://webspace.ship.edu/cgboer/neuron.gif
© Copyright 2003 C. George Boeree
GRAPHING CONVENTIONS
Modulators
Excitation
Inhibition
Learned weights
+
-
(c) CELEST 2007
71
TYPES OF CONNECTIONS
Convergent
Divergent
“In-star”
“Out-star”
(c) CELEST 2007
72
TYPES OF CONNECTIONS
Feedforward
Feedback
(c) CELEST 2007
73
VARIETIES OF LEARNING MUST BE MODELED
Recognition
Reinforcement
Timing
Spatial
Motor Control
Identify
Evaluate
Synchronize
Locate
Act
What
Why
When
Where
How
CogEM MODEL
A model that clarifies how animals learn to attend
to external events that predict satisfaction of
internal drives in real time
Autonomous Adaptive Mobile Robots
e.g., MAVIN Robot
Waxman et al., MIT Lincoln Lab
(c) CELEST 2007
75
CogEM MODEL:
3 Types of Representations and Learning
Grossberg, 1971+
SENSORY
SCS1
+
Competition
for STM
+
CS1
CS2
SCS2
Conditioned
Reinforcer
Learning
D
+
Incentive
Motivational
Learning
DRIVE
Internal Drive Input
Motor
Learning
MOTOR
(c) CELEST 2007
76
DRIVE REPRESENTATIONS
Sites where reinforcement and homeostatic
inputs interact to generate emotional and
motivational output signals
Emotion nodes
Bower et al., 1981
Adaptive Critic Elements
Barto, Sutton, and Anderson, 1983
Facilitator Neuron (Aplysia)
Walters and Byrne, 1983
Hawkins, Abrams, Carew, and Kandel, 1983
(c) CELEST 2007
77
NEURAL DRIVE REPRESENTATIONS
Facilitator Neuron (Aplysia)
Walters and Byrne, 1983
Hawkins, Abrams, Carew, and Kandel, 1983
Amygdala
Aggleston et al., 1995
LeDoux et al., 1988
(c) CELEST 2007
78
INTERPRETATION OF CogEM ANATOMY
SENSORY
CORTEX
PREFRONTAL
CORTEX
AMYGDALA
DRIVE
(c) CELEST 2007
79
AMYGDALA AND NEARBY AREAS
(c) CELEST 2007
80
Visual
Cortex
Somatosensory
Cortex
Auditory
Cortex
Gustatory
Cortex
Olfactory
Cortex
A
M
Y
G
D
A
L
A
Lateral
Prefrontal
Cortex
Orbital
Prefrontal
Cortex
Adapted from Barbas (1995)
(c) CELEST 2007
81
APLYSIA
Buonomano, Baxter, & Byrne, Neural Networks, 1990
Grossberg, Behavioral and Brain Sciences, 1983
+
-
+
-
+
+
FACILITATOR NEURON ~ DRIVE REPRESENTATION
(c) CELEST 2007
82
Why similar circuit in MAMMALS and INVERTEBRATES?
Both solve similar environmental/behavioral
problems!
SYNCHRONIZATION PROBLEM
Variable CS-US Delays
PERSISTENCE PROBLEMS
Multiple emotional meanings
CS1
CS2
Food
Sex
Grossberg (1975)
CR1
CR2
OPPONENT EMOTIONS
IN DRIVE REPRESENTATIONS
FEAR vs RELIEF
Why is ONSET of a shock
NEGATIVELY REWARDING?
FEAR
Why is OFFSET of a shock
POSITIVELY REWARDING?
RELIEF
(c) CELEST 2007
84
OPPONENT REBOUND IS UBIQUITOUS
REINFORCEMENT
Shock on  Fear
Shock off  Relief
(Estes & Skinner, 1941)
(Denny, 1971)
VISUAL PERCEPTION
ON
Picture on Percept
Picture off  Negative Aftereffect
OFF
MacKay Illusion
(c) CELEST 2007
85
OPPONENT PROCESSING
Cognitive-Drive Associations
Primary inhibitory associations
Primary excitatory associations
CS
CS
US
US
Fear
Fear
Relief
Rebound
CS
CS
CS
Fear
ON
OFF
Fear
Relief
Relief
(c) CELEST 2007
86
BEHAVIORAL CONTRAST: REBOUNDS!
1. A sudden DECREASE in frequency or amount of FOOD can
act as a NEGATIVE reinforcer: Frustration
2. A sudden DECREASE in frequency or amount of SHOCK
can act as POSITIVE reinforcer: Relief
Shock
Level
(c) CELEST 2007
Trials
87
BEHAVIORAL CONTRAST
Responses
per minute
(VI schedule)
Reynolds (1968)
Daily sessions
TRIAL
1-5
6-10
11-15
16-20
21-25
SHOCK LEVEL
(c) CELEST 2007
0
Moderate
0
Intense
0
88
MULTIPLE FUNCTIONAL ROLES OF SHOCK
1. Reinforcement sign reversal
An ISOLATED shock is a negative reinforcer
In certain CONTEXTS, a shock can be a positive reinforcer
2. STM-LTM interaction
Prior shock levels need to be remembered (LTM) and used to
calibrate the effect of the present shock (STM)
3. DISCRIMINATIVE AND SITUATIONAL CUES
The present shock level is UNEXPECTED (NOVEL) with respect
to the shock levels that have previously been contingent upon
experimental cues
1. Shock as a reinforcer
2. Shock as a sensory cue
3. Shock as an expectancy
(c) CELEST 2007
89
OPPONENT PROCESSING
How are ON and OFF reactions generated at the drive
representations?
Through a
GATED DIPOLE
OPPONENT PROCESS
Grossberg (1972)
(c) CELEST 2007
90
UNBIASED TRANSDUCER
Grossberg (1968)
S = input
T = output
T = SB
B is the gain
Suppose T is due to release of chemical transmitter y
at a synapse:
S
RELEASE RATE:
y
T=Sy
T
(mass action)
ACCUMULATION: y ~= B
(c) CELEST 2007
91
TRANSMITTER ACCUMULATION AND RELEASE
Transmitter y cannot be restored at an infinite rate:
T=Sy
yB
Differential Equation:
d
dt
y = A (B – y) – S y
Accumulate
Release
Transmitter y tries to recover to ensure
unbiased transduction
What if it falls behind?
Evolution has exploited the good properties that happen then
(c) CELEST 2007
92
HABITUATIVE TRANSMITTER GATE
T=Sy
d
dt
y = A (B – y) – S y
Recent experiments support this prediction:
Visual Cortex: Abbott et al. (1997): depressing synapses
Somatosensory Cortex: Markram et al. (1998)
(c) CELEST 2007
93
MINOR MATHEMATICAL MIRACLE
At equilibrium:
dy
0   A(B  y)  Sy
dt
Transmitter y decreases when input S increases:
AB
y
A S
However, output Sy increases with S!
ABS
Sy 
A S
(gate, mass action)
(c) CELEST 2007
94
HABITUATIVE TRANSMITTER GATE
ABS1
A  S0
ABS0
A  S1
Weber Law
(c) CELEST 2007
95
NONRECURRENT GATED DIPOLE
(c) CELEST 2007
96
ON-RESPONSE TO PHASIC ON-INPUT
ON
S1=f(I+J)
AB
y1 
A  S1
T1  S1y1 
ABS1
A  S1
+ -
S2=f(I)
AB
y2 
A  S2
ABS2
T2  S2 y2 
A  S2
A 2B(f(I  J) - f(I))
ON  T1 - T2 
(A  f(I))(A  f(I  J))
OFF
J
- +
T1
y1
T2
y2
s1
s2
I
Note Weber Law
When f has a threshold, small I requires larger J to fire due to
numerator, but makes suprathreshold ON bigger due to denominator
(c) CELEST 2007
When I is large, quadratic in denominator
and upper bound of f make
ON small
97
OFF-REBOUND DUE TO PHASIC INPUT OFFSET
Shut off J (Not I!). Then: S1 = f(l) and S2 = f(l)
y1 
AB
A  f(I  J)
<
y2
AB

A  f(I)
y1 and y2 are SLOW
T1 = S1y1
T2 = S2y2
T1 < T2
OFF = T 2  T 1 
ABf(I)(f(I  J)  f(I))
(A  f(I))(A  f(I  J))
Arousal sets sensitivity of rebound:
Note Weber Law
due to
remembered
previous input
OFF
ON
=
f(I)
A
(c) CELEST 2007
Why is the rebound transient? Note equal f(l) inputs
98
NOVELTY RESET: REBOUND TO AROUSAL ONSET
Equilibrate to I and J:
S1=f(I+J)
AB
y1 
A  S1
S2=f(I)
y2 
AB
A  S2
Keep phasic input J fixed; increase arousal I to I* = I + ∆ I:
OFF reaction if T1 < T2
OFF = T2 - T1 = f(I*+J) y2 - f(I*) y1
AB(f(I*) - f(I *  J)) - B(f(I*)f(I  J) - f(I)f(I *  J)

(A  f(I))(A  f(I  J))
How to interpret this complicated equation?
(c) CELEST 2007
99
NOVELTY RESET: REBOUND TO AROUSAL ONSET
f(w)
f(w)= Cw: Linear signal
OFF 
ABJ( D I - A)
(A  I)(A  I  J)
∆I = I*- I
OFF > 0 only if there is enough novelty: ∆I > A
OFF response increases with J:
If a given cell has a greater effect on a mismatched
expectation, then it is reset more vigorously
Selective reset of dipole field by unexpected event
(c) CELEST 2007
100
GOLDEN MEAN
INVERTED U AS A FUNCTION OF AROUSAL
A 2B(f(I  J) - f(I))
ON 
(A  f(I))(A  f(I  J))
Behavior
Arousal
Underaroused Depression
Elevated threshold
Hyperexcitable
above threshold
Overaroused Depression
Low threshold
Hypoexcitable
above threshold
“UP” brings excitability “DOWN”
(c) CELEST 2007
101
Consider the simplest type of
COGNITIVE-EMOTIONAL LEARNING
(c) CELEST 2007
102
CLASSICAL CONDITIONING
(Nonstationary prediction)
Bell (CS)
Bell (CS)
(CR)
Fear
(UR)
ASSOCIATIVE LEARNING
AB
CS US
AB
CS US
AB
CS US
(c) CELEST 2007
A
CS
B
US
CR
103
INTERSTIMULUS INTERVAL (ISI) EFFECT
CS
ISI
US
CR
0
0
ISI
Large ISI obvious: No CS-US correlation
(c) CELEST 2007
Why poor learning at 0 ISI, with good correlation?
104
INTERSTIMULUS INTERVAL (ISI) EFFECT
(c) CELEST 2007
105
SECONDARY CONDITIONING
(Advertising!)
CS1
US
FEAR
CS1 becomes a CONDITIONED REINFORCER
CS2
CS1
FEAR
CS2 becomes a CONDITIONED REINFORCER
(c) CELEST 2007
106
How are
CLASSICAL CONDITIONING
and
ATTENTION
related?
(c) CELEST 2007
107
PARALLEL PROCESSING OF
EQUALLY SALIENT CUES
CS1
Bell
t
CS2
t
Light
US
t
FEAR
t
CS1
FEAR
CS2
FEAR
vs. OVERSHADOWING (Pavlov)
(c) CELEST 2007
108
BLOCKING
MINIMAL ADAPTIVE PREDICTION
Phase I
CS1
US
FEAR
CS1
Phase II
CS1
CS2
US
CS2
FEAR
(c) CELEST 2007
CS2 IS IRRELEVANT
109
BLOCKING = ISI + SECONDARY CONDITIONING
Blocking
Zero ISI
1)
CS1
US
Fear
2)
CS2
CS
CS1
US
Fear
Fear
No CS2 conditioning
No CS conditioning
(c) CELEST 2007
110
CONDITIONED REINFORCER
CS1 becomes a conditioned reinforcer by learning to
activate a strong reinforcer-motivational (emotional)
feedback pathway
Sensory
Representation
CS1
Conditioned
Reinforcer
+
Incentive Motivation
Drive Representation
US
(c) CELEST 2007
111
CogEM EXPLANATION OF ATTENTIONAL BLOCKING
SENSORY
SCS1
Competition
for STM
+
+
CS1
CS2
Conditioned
Reinforcer
Learning
D
Internal
Drive
Input
SCS2
+
Incentive
Motivational
Learning
DRIVE
MOTOR
Motor
Learning
1. Sensory representations compete for LIMITED CAPACITY STM
2. Previously reinforced cues amplify their STM via
POSITIVE FEEDBACK
(c) CELEST 2007
3. Other cues lose STM via COMPETITION
112
Sensory
Input
CS
time
STM
activity
without
motivational
feedback
STM
activity
with
motivational
feedback
+
(c) CELEST 2007
113
BLOCKING
X2
CS2
X1
CS1
+
STM suppressed
By competition
X2
t
STM amplified
By (+) feedback
X1
(c) CELEST 2007
t
114
POSITIVE ISI
CS
input
SCS
activity
Sampling interval
US
input
SUS
activity
SCS
SUS
(c) CELEST 2007
115
ISI EFFECT
Grossberg and Levine, 1987
(c) CELEST 2007
116
EMOTIONAL CONDITIONING
CS1’S STM trace
D’s STM Trace
CS1
+
D
Anticipatory CR
c
a
US’s STM Trace
CS1 D LTM
Trace
(c) CELEST 2007
117
b
d
BLOCKING
CS1’S STM trace
CS1
D LTM Trace
CS1 - US CS1 + CS2 - CS2
US
a
CS2’s STM Trace
CS2
c
CS2
D LTM Trace
CS1
+
(c) CELEST 2007
b
d
118
UNIFIED EXPLANATION OF
BLOCKING
ISI EFFECT
SECONDARY CONDITIONING
ANTICIPATORY CR
COOPERATION
between
COGNITIVE and EMOTIONAL
representations
COMPETITION
between
COGNITIVE
representations
(c) CELEST 2007
119
MINIMAL ADAPTIVE PREDICTION
BLOCKING
1) CS1
US
Fear
CS1
2) CS1 + CS2
CS2
UNBLOCKING
x
US
1) CS1
CS1
2) CS1 + CS2
Fear
CS2 is irrelevant
CS2
US1
Fear
US2
Fear
if US2 > US1
Relief
if US2 < US1
CS2 predicts US change
Learn if CS2 predicts a different (novel) outcome than CS1
CS2 not redundant (“wallpaper”)
(c) CELEST 2007
120
MINIMAL ADAPTIVE PREDICTOR
1. Pay attention to (code, learn) RELEVANT cues
CS1 predicts US1
2. Unexpected CONSEQUENCES redefine the set of
relevant cues
Changing US1 to US2 makes CS2 relevant
3. Unexpected consequence (NOVELTY) feeds back in
time via a NONSPECIFIC event to redefine relevant
cues
t
CS2
US2
4. Distinguish NOVELTY from EMOTIONAL SIGN
(US1 >< US2)
(c) CELEST 2007
HOW ART WAS DISCOVERED IN 1973!
121