Transcript Document

Davide Ballabio
Milano Chemometrics and QSAR Research Group
Università Milano - Bicocca
Kohonen Maps and
Counterpropagation
Artificial Neural Networks
Toolbox for MATLAB
Kohonen and CPANN Toolbox for MATLAB
• Kohonen Maps (or Self Organizing Maps) and
Counterpropagation Artificial Neural Networks (CPANNs) are two
of the most popular Neural Networks (algorithms that simulate the
human learning).
• can handle both supervised and unsupervised problems
• Kohonen and CP-ANN toolbox is a collection of MATLAB
modules freely available via internet:
http://michem.disat.unimib.it/chm
Kohonen maps
• Kohonen Maps (or Self Organizing Maps) can handle
unsupervised problems
SOM are made of neurons
and need time (iterations)
to learn to describe the
data.
The time (number of
iterations) and the size
(number of neurons) must
be defined by the user
Kohonen maps
• Kohonen Maps (or Self Organizing Maps) can handle
unsupervised problems
TOP MAP
x1
x2
x3
x4
p Kohonen layers
(p = number of variables)
A
Kohonen layers
Kohonen maps
• Kohonen Maps (or Self Organizing Maps) can handle
unsupervised problems
Toroidal geometry
X
CPANN
• Counterpropagation Artificial neural Networks can handle
supervised problems (in this case classification problems)
• CPANNs are an evolution of Kohonen Maps
CPANN
• Class unfolding: transform the class vector in a matrix of zeros
and ones
Class (n x 1)
1
1
2
2
3
3
..
G
G
unfolding
Multi-y (n x G)
1
1
0
0
0
0
0
0
1
1
0
0
0… … …0
0… … …0
0… … …0
0… … …0
1… … …0
1… … …0
0 0 0 … … …1
0 0 0 … … …1
CPANN
• Counterpropagation Artificial neural Networks can handle
supervised problems (in this case classification problems)
A
x1
x2
x3
TOP MAP
Kohonen layers
x4
y1
y2
y3
Output layers
CPANN
• Counterpropagation Artificial neural Networks can handle
supervised problems (in this case classification problems):
For each sample,
for each epoch:
1) Find the winning neuron
2
m
out c  min   xsi  w ji  
 i 1

CPANN
• Counterpropagation Artificial neural Networks can handle
supervised problems (in this case classification problems):
For each sample,
for each epoch:
2) Update the Kohonen
weights

dr 

  x i  w old
Δw r  η  1 
r
d

1
max



learning
rate
topological
distance

CPANN
• Counterpropagation Artificial neural Networks can handle
supervised problems (in this case classification problems):
For each sample,
for each epoch:
3) Update the output
weights

dr 

  c i  y old
Δy r  η  1 
r
d

1
max




Kohonen Maps and CPANN
• In order to better understand how variables can characterize the
data, we can do Principal Component Analysis on the
Kohonen weights:
variables
PCA
neurons
W
Weights matrix
1. Eigenvalues:
decide how many
component we can retain
Nxp
3. Scores
analyse the neurons
2. Eigenvectors (loadings)
analyse the variables
relate neurons (and samples placed in neurons)
and variables in a global way
Kohonen Maps and CPANN toolbox
• Features of the toolbox:
• user defined data scaling + automatic range scaling
• modules for fitting and validating models
• main available settings:
• size and epochs (required)
• boundary condition
• learning rates can be modified
• missing values are handled
• visualize top map with Graphical User Interface
• visualize PCA on weights with Graphical User Interface
The toolbox is based on the algorithm explained in:
Zupan J, Novic M, Ruisánchez I. Chemometrics and Intelligent Laboratory
Systems (1997) 38 1-23.
Kohonen Maps and CPANN toolbox
• Example of application on the ITAOILS dataset:
• 572 olive oil samples,
• each sample is described by the percentage composition of
8 fatty acids (variables)
• samples belong to 9 different Italian areas of productions
(classes)
• the final aim of the classification model is the geographical
origin determination of the samples.
Data reference:
M. Forina, C. Armanino, S. Lanteri, E. Tiscornia, in: Food Research and Data
Analysis, Classification of olive oils from their fatty acid composition, Applied
Science Publishers, London, 1983.
Kohonen Maps and CPANN toolbox
• Principal Component Analysis for data structure evaluation
Davide Ballabio
Milano Chemometrics and QSAR Research Group
Department of Environmental Sciences
Università Milano – Bicocca
You can download the Kohonen and CP-ANN toolbox 1.0 here:
http://michem.disat.unimib.it/chm
Version 2.0 will be available soon…
Paper in preparation, to be submitted to Chemometrics
and Intelligent Laboratory Systems
Thanks for your attention !