Transcript PowerPoint Slides

Where are we? What’s left?
•
•
•
•
HW 7 due on Wednesday
Finish learning this week.
Exam #4 next Monday
Final Exam is a take-home handed out next
Friday in class
• Scheduled Final Exam meeting – turn in
your exam, your team’s final paper, and
your final timecard
What are these?
AND gate and OR gate
Truth table for an AND gate
Truth table for an OR gate
A
B
Q
A
B
Q
0
0
0
0
0
0
0
1
0
0
1
1
1
0
0
1
0
1
1
1
1
1
1
1
Artificial Neural Networks
•Biological Inspiration
•Brain vs. Computers
•The Perceptron
•Multilayer networks
•Some Applications
ICS611
4
Biological inspiration
•Animals are able to react adaptively to changes in
their external and internal environment, and they use
their nervous system to perform these behaviours.
•An appropriate model/simulation of the nervous
system should be able to produce similar responses
and behaviours in artificial systems.
•The nervous system is build by relatively simple
units, the neurons, so copying their behavior and
functionality should be the solution.
Brain and Machine
• The Brain
–
–
–
–
Pattern Recognition
Association
Complexity
Noise Tolerance
• The Machine
– Calculation
– Precision
– Logic
The contrast in architecture
• The Von Neumann architecture uses
a single processing unit;
– Tens of millions of operations per
second
– Absolute arithmetic precision
• The brain uses many slow
unreliable processors acting in
parallel
The Structure of Neurons
•1011 neurons of at least 20 types.
•1014 synapses
•1-10 ms cycle time
•Signals are noisy “spike trains” of electrical potential
•Neurons die off frequently (never replaced)
•Compensates for problems by massive parallelism
synapse
nucleus
cell body
dendrites
axon
The Structure of Neurons
• A neuron only fires if its input signal exceeds
a certain amount (the threshold) in a short
time period.
• Synapses vary in strength
– Good connections allowing a large signal
– Slight connections allow only a weak signal.
– Synapses can be either excitatory or inhibitory.
The Structure of Neurons
•The spikes travelling along the axon of the presynaptic neuron trigger the release of neurotransmitter
substances at the synapse.
•The neurotransmitters cause excitation or inhibition in
the dendrite of the post-synaptic neuron.
•The integration of the excitatory and inhibitory signals
may produce spikes in the post-synaptic neuron.
•The contribution of the signals depends on the
strength of the synaptic connection.
Translating this to ANN
The McCullogh-Pitts model:
• spikes are interpreted as spike rates;
• synaptic strength are translated as synaptic weights;
• excitation means positive product between the
incoming spike rate and the corresponding synaptic
weight;
• inhibition means negative product between the
incoming spike rate and the corresponding synaptic
weight;
The McCulloch-Pitts “Unit”
• Each neuron has a threshold value
• Each neuron has weighted inputs from other
neurons
• The input signals form a weighted sum
• If the activation level exceeds the threshold, the
neuron “fires”
The Activation Function
(a) Is a step function or threshold function
(b) Is sigmoid function [ 1/(1+e-x) ]
Changing the bias weight Wo,I moves the
threshold location.
Any Boolean function can be implemented
using a McCulloch and Pitts perceptron
-0.5
-1
What function does
perceptron #1 represent?
0.6
0.7
0.8
0.8
What function does
perceptron #2 represent?
0.8
0.2
0.8
0.3
What function does
perceptron #3 represent?
0.6
- 0.4
0.3
0.8
- 0.8