Transcript Document

Neural Networks
• Examples of Single Layer Perceptron
– Convergence in Linear Case
– No Convergence in Non-linear Case
• (Hyper)Line of separation
– Separation of space by (hyper)line/plane
– Orthogonality to weight vector
Lecture 7, CS567
1
Neural Networks
• Consider 2 input single neuron for Boolean
prediction of clinical drug effectiveness, based on
results from 2 experiments X and Y
– Neuron output = 1 => Clinically effective drug
– Neuron output = 0 => Drug not effective clinically
• Inputs to network
– p1 = Prediction of Experiment X [Yes/No] = [0/1]
– p2 = Prediction of Experiment Y [Yes/No] = [0/1]
• Consider wonder drug candidates dA, dB, dC, dD
for revolutionary product “eMemory” that is
proclaimed to increase IQ (as it turns on genes
responsible for mental alertness and suppression
of neuronal loss) used to train network
Lecture 7, CS567
2
Neural Networks
Exp X Exp Y
Observed Clinical effect
dA
0
0
0
dB
1
0
0
dC
0
1
0
dD
1
1
1
0 = No effect; 1 = Effect observed
Lecture 7, CS567
3
Neural Networks
• Consider 2 input SLP with random initial
weights and bias
p1
w1,1=1
w1,2=1
Neuron
b=0.5
p2
Sum = x = w1,1p1 + w1,2p2 + b
= p1 + p2 + 0.5
Output = fhardlimit(x) = 0 if (x < 0), else 1
Lecture 7, CS567
4
Neural Networks
• Consider NN prediction for dA (Target prediction
is 0)
p1
p2
0
w1,1=1
w1,2=1
Neuron
b=0.5
Sum = 0.5; Output f(0.5)= 1
0
Error e = Target - f(x) = 0 – 1 = -1
Wcurrent = Wprevious + eP => w1,1=1; w1,2=1
bcurrent = bprevious + e => b = 0.5 – 1 = -0.5
• Note that output now for dA = -0.5 => 0 (as
Lecture 7, CS567
expected)
5
Neural Networks
• Consider NN prediction for dB (Target prediction
is 0)
p1
p2
1
w1,1=1
w1,2=1
Neuron
b=-0.5
Sum = 0.5; Output f(0.5)= 1
0
Error e = Target - f(x) = 0 – 1 = -1
Wcurrent = Wprevious + eP => w1,1=0; w1,2=1
bcurrent = bprevious + e => b = -0.5 – 1 = -1.5
• Note that output now for dB = -0.5 => 0 (as
expected)
Lecture 7, CS567
6
Neural Networks
• Consider NN prediction for dC (Target
prediction is 0)
p1
p2
0
w1,1=0
w1,2=1
Neuron
b=-1.5
Sum = -0.5; Output f(-0.5)= 0
1
Error e = Target - f(x) = 0 – 0 = 0
No change in weights or bias
Lecture 7, CS567
7
Neural Networks
• Consider NN prediction for dD (Target prediction
is 1)
p1
p2
1
w1,1=0
w1,2=1
Neuron
b=-1.5
Sum = -0.5; Output f(-0.5)= 0
1
Error e = Target - f(x) = 1 - 0 = 1
Wcurrent = Wprevious + eP => w1,1=1; w1,2=2
bcurrent = bprevious + e => b = -1.5 + 1 = -0.5
• Note that output now for dD = 1.5 => 1 (as
expected)
Lecture 7, CS567
8
Neural Networks
• Check NN prediction is correct for all
inputs dA:dD
• OK for all except dC (expect 0)
p1 0
p2 1
w1,1=1
w1,2=2
Neuron
b=-0.5
Sum = 1.5; Output f(1.5)= 1
Error e = Target - f(x) = 0 - 1 = -1
Wcurrent = Wprevious + eP => w1,1=1; w1,2=1
bcurrent = bprevious + e => b = -0.5 - 1 = -1.5
Lecture 7, CS567
9
Neural Networks
• Check NN prediction is correct for all inputs
dA:dD
p1
p2
w1,1=1
w1,2=1
Neuron
b=-1.5
• Converged: Gives right prediction for all possible
inputs. (Try starting with different initial states)
Lecture 7, CS567
10
Neural Networks- (Hyper)Line of separation
1
2
p2
• 2 input Boolean problem is linear if line can be drawn
between the two classes of result
• 3 input Boolean problem is linear if plane can be drawn
between the two classes of result
• Note that Weight vector points to the positive side and is
orthogonal to line of separation
Effective drug
W
1
Ineffective drug
2
p1
WP + b = 0
Lecture 7, CS567
11
Neural Networks
• Will this converge? (XOR does not converge)
Matches
Player X Player Y
Match Result
A
0
0
0
B
1
0
1
C
0
1
1
D
1
1
0
Lecture 7, CS567
12