Transcript Slayt 1 - Department of Information Technologies

Question
Although we have only a rudimentary understanding of biological neural
networks,
is it possible to construct a small set of simple artificial “neurons” and perhaps
train them to serve a useful function?
The answer is “yes.”
Architecture ?
Learning Rule ?
Architecture
Architecture
Architecture
Architecture
Learning Rule, Biological Inspiration
Learning is viewed as
the establishment of new connections between neurons or
the modification of existing connections.
Neural structures continue to change throughout life. These later changes
tend to consist mainly of strengthening or weakening of synaptic
junctions.
For instance, it is believed that new memories are formed by modification
of these synaptic strengths. Thus, the process of learning a new friend’s
face consists of altering various synapses.
Learning Rule, ANN
Learning is viewed as
the establishment of new connections between neurons or
the modification of existing connections.
learning rule is defined as
a procedure for modifying the weights and biases
of a network.
This procedure may also be referred to as a training algorithm
Learning Rules Categories
In supervised learning,
the learning rule is provided with a set of examples (the training set) of proper
network behavior
Where
pq is an input to the network, and
tq is the corresponding correct (target) output.
In unsupervised learning,
the weights and biases are modified in response to network inputs only. There
are no target outputs available. Most of these algorithms perform clustering
operations. They categorize the input patterns into a finite number of classes.
Perceptrons
The perceptron was created by Rosenblatt.
Single-layer network whose weights and biases could be trained to produce a
correct target vector when presented with the corresponding input vector.
Perceptron Architecture ?
Perceptron Learning Rule ?
Perceptron Neuron
The perceptron neuron produces a 1 if the net input into the transfer function
is equal to or greater than 0; otherwise it produces a 0.
Perceptron Architecture
Perceptron Architecture
Perceptron Learning Rule (learnp)
CASE 1. If an input vector is presented and the output of the neuron is
correct (a = t, and e = t – a = 0), then the weight vector w is not altered.
CASE 2. If the neuron output is 0 and should have been 1
(a = 0 and t = 1, and e = t – a = 1), the input vector p is added to the
weight vector w.
CASE 3. If the neuron output is 1 and should have been 0
(a = 1and t = 0, and e= t – a = –1), the input vector p is subtracted from
the weight vector w.
Perceptron Learning Rule (learnp)
CASE 1. If an input vector is presented and the output of the neuron is correct
(a = t, and e = t – a = 0), then the weight vector w is not altered.
CASE 1. If e = 0, then make a change Δw equal to 0.
CASE 2. If the neuron output is 0 and should have been 1
(a = 0 and t = 1, and e = t – a = 1), the input vector p is added to the weight
vector w.
CASE 2. If e = 1, then make a change Δw equal to pT.
CASE 3. If the neuron output is 1 and should have been 0
(a = 1and t = 0, and e= t – a = –1), the input vector p is subtracted from the
weight vector w.
CASE 3. If e = –1, then make a change Δw equal to –pT.
Perceptron Learning Rule (learnp)
CASE 1. If e = 0, then make a change Δw equal to 0.
CASE 2. If e = 1, then make a change Δw equal to pT.
CASE 3. If e = –1, then make a change Δw equal to –pT.
All three cases can then be written with a single expression
Δw = (t – a)pT = epT
Perceptron Learning Rule (learnp)
The Perceptron Learning Rule can be summarized as follows
Wnew = Wold + epT
bnew = bold +e
where e
=t–a
Perceptron Learning Rule (learnp)
>> net = newp([-1 1;-2 +2],1);
>> p =[1 -1 0; 2 2 -2];
>> t =[0 1 1];
>> net = train(net,p,t);
>> a = sim(net,p)
An Illustrative Example
A produce dealer has a warehouse that stores a variety of fruits and vegetables. When
fruit is brought to the warehouse, various types of fruit may be mixed together. The
dealer wants a machine that will sort the fruit according to type. There is a conveyer belt
on which the fruit is loaded. This conveyer passes through a set of sensors, which
measure three properties of the fruit: shape, texture and weight .
The shape sensor will output a 1 if the fruit is approximately round and a 0 if it is more
elliptical. The texture sensor will output a 1 if the surface of the fruit is smooth and a 0 if
it is rough. The weight sensor will output a 1 if the fruit is more than one pound and a 0
if it is less than one pound.
Pineapple
Banana
Perceptron Learning Rule (learnp)