Transcript Ch03b

Part 1. Review of the CNS
The Central Nervous System is made
up of 100 Billion NEURONS
Neurons are Cells
Anatomy of a Neuron
Myelin is supplied by support
cells known as Glia (Glue)
White Matter and Gray Matter
• White Matter
– Axons (highways)
• Gray Matter
– Cell bodies (cities)
Nervous System Cells
• Neurons and communication
– Neurons are “charged” like batteries and
have the ability to send electrical messages
over long distances to other cells.
• Sensory neurons carry information from
sense organs to the central nervous system.
• Motor neurons carry information from the
CNS to muscles
Charging the Neuron
Action Potential
A fast burst of electrical energy
sent down the axon to the synapse,
a gap between two neurons.
Integrate and Fire!!
• Neurons add up information from excitatory and
inhibitory sources
• If the net activity raises the level of excitation
above a certain threshold, the neuron will fire an
action potential down its axon
• A neuron is either sending a message or it isn’t.
– It’s a binary system (more on this later!)
Nervous System Cells
• Neurons and communication
– Sending a message results in the depletion
of the battery’s power, so the neurons need
to be constantly recharged.
Nervous System Cells
• Nerve Impulses
– Each neuron can either increase or decrease
the likelihood that the next cell will continue
to transmit.
• Excitatory messages increase the probability that
the next cell will “fire” - continue the
transmission.
• Inhibitory messages decrease the likelihood that
transmission will continue to travel.
Synaptic
Transmission
• Electrical message
is converted to a
chemical message
– Activates another
neuron
– Moves a muscle
– Triggers hormonal
release from a gland
Neural Networks
• Each neuron receives
information from many others
and sends information to many
others.
– The significance (or weight) of a
synaptic connection depends on
the number and size of synapses
shared by two neurons
Synaptic Weight
• Although both neurons A and B have excitatory
synapses with C, Activity in cell A will have a
bigger effect on C than the same amount of
activity in cell B.
A
C
• Neurotransmitters can be either excitatory or inhibitory
• Lock and Key System
Neurotransmitters & Behavior
• Our understanding of neurotransmitters is the key
to treating many brain-related disorders
• Parkinson’s Disease
– Parkinson’s Disease is a condition in which the
individual has trouble executing voluntary
movements, and has tremors, rigidity and a
depressed mood.
– This condition has been linked to a gradual decay
in a system of axons that release the
neurotransmitter dopamine.
Summary
• The Brain is an Electrical System
– Neurons fire in an all-or-none fashion
– Neurons may either increase or decrease
another neuron’s chances of firing
• Some Terms
– Axon, threshold, dendrite, integrate and
fire, neurotransmitter, excitation,
inhibition, synapse, weight
CRUM Theory
•
Computational-Representational
Understanding of the Mind
– Information is stored and processed
1. The flow of neural activity = processing
2. The state of synapses = storage
History of Connectionism
• Logical operations are not new (ancient Greece)
• Connectionists seek to develop models based on what
we know about the human brain.
• A computer is not a very good connectionist model
– A computer is a serial processor
– Brain is a parallel processor
• Connectionists believe that the phenomenon known as
“mind” can be explained by interconnected neural units
that follow the rules of neurons.
Logical Operations
• All statements are TRUE or FALSE
War and Science
The Beginning: Artificial Intelligence
• Is the intelligence of the computer
the same as that of the human?
• Initially, idea eagerly accepted
• 1950 Turing test: can a subject
interacting with a computer be
persuaded that he/she is
communicating instead with a
human?
Alan Turing (1912-1954)
• Chinese Room Problem
• Deep Blue vs. Kasparov
The McCullough and Pitts Neuron
• They proposed that simple electrical circuits that behaved
like neurons could perform some basic logical functions.
– Neurons were either ON or OFF (binary)
– Neurons had thresholds
• A certain amount of activation was required in order to
get them to fire
– Connections (synapses) had weights that could vary in
intensity and polarity (+ or -)
– Computation took place in a succession of time slices
(i.e. it took time for a message to travel from one
neuron to the next)
McCullough and Pitts (1943)
• Using Electrical Circuits to Study
Neurons
Θ = Threshold
W = Weight or connection Strength
W=2
Θ=2
McCullough and Pitts (1943)
• Logical “OR” operation
W=2
Θ=2
McCullough and Pitts (1943)
• Logical “AND” operation
Θ=2
McCullough and Pitts (1943)
• Logical “NOT” operation
Θ=2
Donald Hebb (1949)
Donald Hebb: Canadian Neuropsychologist who mediated
between the Gestalt and atomist-localizating points of view.
The Organization of Behavior (1949)
• Opposed to the behaviorist “Black Box”
• It is impossible to describe behavior as interaction
directly between sensory and motor processes
• Impossible to describe behavior as an interaction
between sensory and motor processors, thinking
intervenes
• Unlike M&P, Hebb proposed a Dynamic Network
Properties of a Hebb Synapse
According to Hebb,
Changes in the brain
(learning) were the
result of changes at
the synapses.
This has been
supported by
hundreds of
subsequent studies
Rosenblatt Perceptron (1958)
• Learned to categorize
– The perceptron had to be trained
• If it was wrong, connections were adjusted
Perceptron Convergence Rule
Netinputout = Σinwain
aout = 1 if netinputout > Θ
= 0 otherwise
Input vs. Output
ain
w
w
Θ
ain
If the activity level is too low then threshold should be
decreased and connection strength should be increased.
If too high, vise versa.
ΔΘ = -εδ
Δw = εδ a in
where δ = (tout-aout)
and ε = learning rate
constant (between 0 and 1)
Minsky and Pappert (1969)
• A Critique of Perceptrons
– The best perceptron could not solve the
exclusive OR problem (XOR)
– The sophisticated “alphabet skill” is not
so impressive when one realizes that the
simple XOR problem cannot be solved
– Sunk connectionism for 15 years
– See Pinker p. 110
Concepts and Connectionism
• What might work and what won’t
• Remember Pinker and “mentalese” as
we discuss these examples
Categories and Terms
• Exemplar – Individual members of a
category
• Rule – A precise definition of the
criteria for a category
• Prototype – Specifies the properties
that are most likely to be true of a
category
PROTOTYPICALITY:
For each category, assign the number “1” to the
best exemplar, “2” to the second best, “3” to the
third best, “4” to the fourth best, and “5” to the
worst member of the category.
VEHICLES
car
elevator
sled
tractor
train
CLOTHING
jacket
mittens
necklace
pajamas
pants
FRUIT
olive
grapefruit
orange
pear
honeydew
Typicality Gradients
• Sentence verification experiments
Prototypes vs. Exemplars
• Prototype theory – We have one “ideal”
member of a category and we make
judgments by comparing a stimulus to the
ideal (which may not be an exemplar!).
• Exemplar theory – We have lots of
exemplars stored and we make judgments
by comparing a stimulus to all exemplars
and adding up the result.
How does Classification Emerge?
Rosch provided evidence for the idea that the basic level is
“psychologically privileged.” We start at the middle.
- Categories at the middle level are most consistent
across cultures, easiest to process, and members are
more clearly grouped (What else do you notice?)
Weights
Weights
A parallel-distributed-processing (PDP) network.
All seven units carry the message for both
animals. The pattern of activity determines
what you are thinking about.
Learning in a PDP network.
(a) Initially presenting canary
causes a pattern of activation
in the output units that is
different than the pattern that
stands for canary. (b) An error
signal is transmitted back
through the network to
indicate how weights need to
be changed to achieve the
correct output response.
continued on next slide
Figure 8.30c (p. 300)
(c) After the processes in (a) and (b) are repeated many times, the
network has learned to respond correctly to canary.
Pros and Cons of PDP
• Uses the rules of the nervous system
• Uses rules of learning
– Try, get feedback, adjust, try again
• Exhibits graceful degradation
• Can’t explain fast learning
• New learning will compromise old knowledge
• May explain some kind of learning, but not all.
The Excel Worksheet
• The worksheet is the area where you enter all
your data.
• It is divided into vertical columns labeled A, B,
C,…, and horizontal rows labeled 1, 2,3,…
• A cell is the intersection of a row and a column.
For example, cell D4 is found where column D
intersects row 4.
Entering Data (cont)
 In
Excel there are two types of data
– The first type, a Value, is
either numeric data or a
formula that generates
numeric data.
– The second type of data is
called a Label. A Label is
any string of characters
(letters or numbers) that is
used for descriptive
purposes rather than as a
numeric value or formula.
Formulas
• Formulas are entered into cells to do
calculations.
• A Formula must start with an = sign. The = sign
lets Excel know that the formula is not a label
(see previous slide).
• A Formula can contain many mathematical
symbols including addition (+), subtraction (-),
multiplication (*), division (/), and
exponentiation (**).
Formulas (cont)
• The order of calculation in a formula is
based on standard mathematical
operational precedence: from left to
right, first what is contained in
parentheses, then exponentiation, then
multiplication and division, then
finally addition and subtraction. (2+1)2
*3 – 5=22
Formulas (cont)
• Cell references can be used in formulas as well. This feature
shows the power of spreadsheets.
• For example, suppose the cell B7 contains the value 8 and the
cell C7 has the value 100. We can enter a value in another
cell, say D8, to multiply cell B7 by C7.
• The cell in D8 would then hold the formula: =B7*C7 which
gives a value of 800.
• If we change the value in cell B7 to 5, then the value in cell D8
will change to 500 automatically.
Functions
• Excel provides many functions to carry out mathematical
calculations, lookups, etc. There is extensive on-line help
that lists all of the functions available and how to use them in
Excel.
• To access the on-line help, first select the ‘Search for Help
on…’ or the Microsoft Excel Help option under the Help
menu
• You can also click the function wizard (fx) to search
Treating Cells as Neurons
• The “neuron” of neural modeling is
referred to as a unit
• We can define cells in an Excel
Spreadsheet as units and use other cells to
define weights (synaptic strength) that
allow values in one cell to influence those
in another
Modeling Binary Units in Excel
0
0
• One unit is represented by cell B5 and
another is represented by cell F5
• The arrow represents an axon
– Not necessary, but helpful
0
0
• Cell D6 represents the weight (synaptic
strength) between the presynaptic and
postsynaptic units.
?
• “Activate” the presynaptic unit by entering a
number into cell B5
• The activation that will reach the synapse is the
product of the activity of the presynaptic unit and
the weight of the connection between the two units
(1x2)=2
?
• Define the threshold of the postsynaptic cell
to determine if it will be active or not (G7)
• If the threshold is exceeded, the activity level
of the Postsynaptic cell will be set to 1
?
• The following will set the postsynaptic cell to
1 if the product of B5 and D6 is greater than
the threshold (G7).
• Put this equation in the postsynaptic unit (F5)
=if(B5*D6 >= G7,1,0)
Exercise 1
• Set up a simple network of binary units
– 3 Presynaptic units converging on one
postsynaptic unit
– Set up cells that represent the 3 presynaptic
units, the 3 weight values, the postsynaptic
cell and the postsynaptic cell’s threshold so
you can quickly change these values and test
how these values affect the postsynaptic
cell’s output
Exercise 2
• Set up a network of five units using two input units,
two hidden units, and one output unit as illustrated:
– Make all activation and weight values easily
changeable
– Go slowly! Build and test one pair of units at a time
– Try adding inhibition by giving negative values to
some of your weights
Exercise 3
• Set up a network that computes
– AND
– OR
– NOT
• Try XOR – Do you understand Minsky
and Pappert’s complaint?