Transcript Slide 1

Non-linear classification problem using NN
The Use of NN in Classification
Architecture
Three layers Feedforward Neural
Network (FFNN) is sufficient for
realizing a broad class of input/output
non-linear maps (Kolmogorov’s
theorem)
Disadvantages:
• number of neurons in the hidden layer
cannot be determined
• number of neurons can be large
implying expensive calculation
Fainan May 2006
Training
Backpropagation Algorithm
Disadvantages:
• number of training epochs can not
be determined
• local minima
Pattern Classification and Machine Learning Course
Non-linear classification problem using NN
Alternative: NN Design Using Voronoi Diagrams
Given two classes S1 and S2 and two features x1 and x2:
S1 = {(4,0),(0,4)} S2 ={(0,0),(4,4)}
2 features  two neurons at the first layer
2 classes  two neurons at the output layer
Solution:
Fainan May 2006
Step 1: Draw convex hulls
related to each class
Pattern Classification and Machine Learning Course
Non-linear classification problem using NN
x1-2=0
Step 2: Specify Hyperplanes
4-Veronoi cells  4
neurons at the hidden layer
x2-2 = 0
Step 3:
Form a cluster corresponding to each class:
C1 the cluster corresponding to class S1:
C2 the cluster corresponding to class S2:
Fainan May 2006
H

1
H

1
 

 

 H 2  H1  H 2
 H 2  H1  H 2
Pattern Classification and Machine Learning Course
Non-linear classification problem using NN
Step 4: Now we are ready for the net synthesis
Specification
Number of
neurons
Activation
function
Bias vector
Weight vector
Input layer
”The
hyperplanes”
2
Bipolar discrete
(outputs either
-1 or +1)
[-2
-2]
Ones
Hidden layer
”AND function”
4
Bipolar discrete
(outputs either
-1 or +1)
[-1.5
-1.5
-1.5
-1.5]
 1 1 
 1  1


 1  1


1 1
Output layer
”OR function”
2
Bipolar discrete
(outputs either
-1 or +1)
[0.5
0.5]
Ones
Layer
Fainan May 2006
Pattern Classification and Machine Learning Course
Non-linear classification problem using NN
FFNN to solve non-linear classification problem
[Ref.] N. K. Bose, and A. K. Garga, ”Neural Network Design Using Voronoi Diagrams,” IEEE
trans. On Neural Networks, vol. 4, no. 5, Sept. 1993.
Fainan May 2006
Pattern Classification and Machine Learning Course