Transcript Neural Networks - Hochschule Wismar

Business School
Institute of
Business Informatics
Unsupervised Learning
Uwe Lämmel
www.wi.hs-wismar.de/~laemmel
[email protected]
1
Unsupervised Learning
Neural Networks
– Idea
– Artificial Neuron & Network
– Supervised Learning
– Unsupervised Learning
– Data Mining – other
Techniques
2
Unsupervised Learning
Unsupervised Learning
–
–
–
–
–
3
Unsupervised Learning
Self-Organizing Map (SOM)
Learning
Clustering – Example
Visualisation
Application: TSP
Self Organizing Maps (SOM)
 A natural brain can organize itself
 Now we look at the position of a neuron and its
neighbourhood
Kohonen Feature Map
 two layer pattern associator
- Input layer is fully connected with map-layer
- Neurons of the map layer are fully connected to each
other (virtually)
4
Unsupervised Learning
Clustering
- objective:
All inputs of a class are mapped onto one and the same
neuron
f
ai
output B
Input set A
- Problem: classification in the input space is unknown
- Network performs a clustering
5
Unsupervised Learning
Winner Neuron
Input-Layer
Winner
Neuron
Kohonen- Layer
6
Unsupervised Learning
Learning in an SOM
1. Choose an input k randomly
2. Detect the neuron z which has the
maximal activity -> winner neuron
3. Adapt the weights in the neighbourhood of z:
neuron i within a radius r of z.
4. Stop if a certain number of learning steps is
finished
otherwise decrease learning rate and radius,
go on with step 1.
7
Unsupervised Learning
Centre of Activation
- Idea:
highly activated neurons push down the activation of
neurons in the neighbourhood
- Problem: Finding the centre of activation:
- Neuron j with a maximal net-input
w
iz
i
 oi max
j
w
ij
 oi
i
- Neuron j, having a weight vector wj which is similar
to the input vector (Euklidian Distance):
z:
8
Unsupervised Learning
 x - wz  = minj  x - wj 
SOM Training
• find the winner neuron z
for an input pattern p
(minimal Euclidian distance)
min m p W j  m p W z
input
pattern mp
j
• adapt weights of connections
• input neurons  winner neuron
• input neurons  neighbours
wij    h jz  (mi  wij ) , if dist( j , z ) r
w 
, otherwise
wij
/
ij
9
Unsupervised Learning
Wj Kohonen layer
h jz  e

dist ( j , z ) 2
2r 2
Example Credit Scoring
A1: Credit History
A2: Debts
A3: Collateral
A4: Income
• We do not look at the Classification
• SOM performs a Clustering
10
Unsupervised Learning
Credit Scoring
– good
= {5,6,9,10,12}
– average = {3, 8, 13}
– bad
= {1,2,4,7,11,14}
11
Unsupervised Learning
Credit Scoring
– Pascal tool box (1991)
– 10x10 neurons
– 32,000 training steps
12
Unsupervised Learning
Visualisation of a SOM
• Colour reflects Euclidian
distance to input
• Weights used as
coordinates of a neuron
• Colour reflects cluster
NetDemo
ColorDemo
13
Unsupervised Learning
TSPDemo
Example TSP
– Travelling Salesman Problem
– A salesman has to visit certain cities and will return
to his home. Find an optimal route!
– problem has exponential complexity: (n-1)! routes
Experiment: Pascal Program, 1998
14
Unsupervised Learning
31/32 states in Mexico?
Nearest Neighbour: Example
– Some cities in
Northern Germany:
– Initial city is Hamburg
Kiel
Hamburg
Exercise:
• Put in the coordinates of
20 important places
• Find a solution for the TSP
using a SOM!
Frankfurt
15
Unsupervised Learning
Schwerin
Hannover
Essen
Rostock
Berlin
SOM solves TSP
Kohonen layer
Draw a neuron at
position:
input
w1i= six
X
Y
16
Unsupervised Learning
w2i= siy
(x,y)=(w1i,w2i)
SOM solves TSP
–
Initialisation of weights:
– weights to input (x,y) are calculated so
that all neurons form a circle
– The initial circle will be expanded to a
round trip
– Solutions for problems of several hundreds of
towns are possible
– Solution may be not optimal!
17
Unsupervised Learning
Applications
– Data Mining - Clustering
– Customer Data
– Weblog
– ...
– You have a lot of data, but no teaching data
available – unsupervised learning
– you have at least an idea about the result
– Can be applied as a first approach to get
some training data for supervised learning
18
Unsupervised Learning