Transcript Ch04a

Summary
• The Brain is an Electrical System
– Neurons fire in an all-or-none fashion
– Neurons may either increase or decrease
another neuron’s chances of firing
• Some Terms
– Axon, threshold, dendrite, integrate and
fire, neurotransmitter, excitation,
inhibition, synapse, weight
History of Connectionism
• Logical operations are not new (ancient Greece)
• Connectionists seek to develop models based on what
we know about the human brain.
• A computer is not a very good connectionist model
– A computer is a serial processor
– Brain is a parallel processor
• Connectionists believe that the phenomenon known as
“mind” can be explained by interconnected neural units
that follow the rules of neurons.
Logical Operations
• All statements are TRUE or FALSE
War and Science
The Beginning: Artificial Intelligence
• Is the intelligence of the computer
the same as that of the human?
• Initially, idea eagerly accepted
• 1950 Turing test: can a subject
interacting with a computer be
persuaded that he/she is
communicating instead with a
human?
Alan Turing (1912-1954)
• Chinese Room Problem
• Deep Blue vs. Kasparov
The McCullough and Pitts Neuron
• They proposed that simple electrical circuits that behaved
like neurons could perform some basic logical functions.
– Neurons were either ON or OFF (binary)
– Neurons had thresholds
• A certain amount of activation was required in order to
get them to fire
– Connections (synapses) had weights that could vary in
intensity and polarity (+ or -)
– Computation took place in a succession of time slices
(i.e. it took time for a message to travel from one
neuron to the next)
McCullough and Pitts (1943)
• Using Electrical Circuits to Study
Neurons
Θ = Threshold
W = Weight or connection Strength
W=2
Θ=2
McCullough and Pitts (1943)
• Logical “OR” operation
W=2
Θ=2
McCullough and Pitts (1943)
• Logical “AND” operation
Θ=2
McCullough and Pitts (1943)
• Logical “NOT” operation
Θ=2
Illusion of Hot (Later)
Donald Hebb (1949)
Donald Hebb: Canadian Neuropsychologist who mediated
between the Gestalt and atomist-localizating points of view.
The Organization of Behavior (1949)
• Opposed to the behaviorist “Black Box”
• It is impossible to describe behavior as interaction
directly between sensory and motor processes
• Impossible to describe behavior as an interaction
between sensory and motor processors, thinking
intervenes
• Unlike M&P, Hebb proposed a Dynamic Network
Properties of a Hebb Synapse
According to Hebb,
Changes in the brain
(learning) were the
result of changes at
the synapses.
This has been
supported by
hundreds of
subsequent studies
Rosenblatt Perceptron (1958)
• Learned to categorize
– The perceptron had to be trained
• If it was wrong, connections were adjusted
Minsky and Pappert (1969)
• A Critique of Perceptrons
– The best perceptron could not solve the
exclusive OR problem (XOR)
– The sophisticated “alphabet skill” is not
so impressive when one realizes that the
simple XOR problem cannot be solved
– Sunk connectionism for 15 years
The Excel Worksheet
• The worksheet is the area where you enter all
your data.
• It is divided into vertical columns labeled A, B,
C,…, and horizontal rows labeled 1, 2,3,…
• A cell is the intersection of a row and a column.
For example, cell D4 is found where column D
intersects row 4.
Entering Data (cont)
 In
Excel there are two types of data
– The first type, a Value, is
either numeric data or a
formula that generates
numeric data.
– The second type of data is
called a Label. A Label is
any string of characters
(letters or numbers) that is
used for descriptive
purposes rather than as a
numeric value or formula.
Formulas
• Formulas are entered into cells to do
calculations.
• A Formula must start with an = sign. The = sign
lets Excel know that the formula is not a label
(see previous slide).
• A Formula can contain many mathematical
symbols including addition (+), subtraction (-),
multiplication (*), division (/), and
exponentiation (**).
Formulas (cont)
• The order of calculation in a formula is
based on standard mathematical
operational precedence: from left to
right, first what is contained in
parentheses, then exponentiation, then
multiplication and division, then
finally addition and subtraction. (2+1)2
*3 – 5=22
Formulas (cont)
• Cell references can be used in formulas as well. This feature
shows the power of spreadsheets.
• For example, suppose the cell B7 contains the value 8 and the
cell C7 has the value 100. We can enter a value in another
cell, say D8, to multiply cell B7 by C7.
• The cell in D8 would then hold the formula: =B7*C7 which
gives a value of 800.
• If we change the value in cell B7 to 5, then the value in cell D8
will change to 500 automatically.
Functions
• Excel provides many functions to carry out mathematical
calculations, lookups, etc. There is extensive on-line help
that lists all of the functions available and how to use them in
Excel.
• To access the on-line help, first select the ‘Search for Help
on…’ or the Microsoft Excel Help option under the Help
menu
• You can also click the function wizard (fx) to search
Treating Cells as Neurons
• The “neuron” of neural modeling is
referred to as a unit
• We can define cells in an Excel
Spreadsheet as units and use other cells to
define weights (synaptic strength) that
allow values in one cell to influence those
in another
Modeling Binary Units in Excel
0
0
• One unit is represented by cell B5 and
another is represented by cell F5
• The arrow represents an axon
– Not necessary, but helpful
0
0
• Cell D6 represents the weight (synaptic
strength) between the presynaptic and
postsynaptic units.
?
• “Activate” the presynaptic unit by entering a
number into cell B5
• The activation that will reach the synapse is the
product of the activity of the presynaptic unit and
the weight of the connection between the two units
(1x2)=2
?
• Define the threshold of the postsynaptic cell
to determine if it will be active or not (G7)
• If the threshold is exceeded, the activity level
of the Postsynaptic cell will be set to 1
?
• The following will set the postsynaptic cell to
1 if the product of B5 and D6 is greater than
the threshold (G7).
• Put this equation in the postsynaptic unit (F5)
=if(B5*D6 >= G7,1,0)
Exercise 1
• Set up a simple network of binary units
– 3 Presynaptic units converging on one
postsynaptic unit
– Set up cells that represent the 3 presynaptic
units, the 3 weight values, the postsynaptic
cell and the postsynaptic cell’s threshold so
you can quickly change these values and test
how these values affect the postsynaptic
cell’s output
Exercise 2
• Set up a network of five units using two input units,
two hidden units, and one output unit as illustrated:
– Make all activation and weight values easily
changeable
– Go slowly! Build and test one pair of units at a time
– Try adding inhibition by giving negative values to
some of your weights
Exercise 3
• Set up a network that computes
– AND
– OR
– NOT
• Try XOR – Do you understand Minsky
and Pappert’s complaint?