Transcript Image Classification - Heriot

Image Classification
MSc Image Processing
Assignment
March 2003
1
Summary


Introduction
Classification using neural networks



Perceptron
Multilayer perceptron
Applications
2
Introduction

Definition



Assignment of a physical object to one of
several pre-specified categories
Unsupervised
Supervised
For more details
See Image Processing course
3
Classification
Classification
Supervised
Pattern
recognition
Parametric
Bayes
Unsupervised
k-means
Fuzzy k-mean
Algebraic
Non-parametric
Neural nets
SVM
Minimum distance
K-nearest neighbour
Decision trees
4
Neural nets


Inspired by the human brain
Useful for



Classification
Regression
Optimization …
5
Model
x1
.
.
.
.
.
w1
wn

f
y=f(wi xi + w0)
x. n
x=(x1…xn) input vector
w=(w0…wn) weight vector
f activation function
6
Perceptron

f=sign
1
-1

2 inputs
w1x1+w2x2+w0=0
7
Perceptron (2)

Example: AND function
1
w0=1
x1
w1=1

w2=1
x2
x1 -1
x2
1
-1
-1
-1
1
-1
1
sign
x2
-1+x1+x2=0
-
+
x1
8
Perceptron (3)

Algorithm




Minimise set of misclassified examples
Gradient ascent
Converges if data linearly separable
Demo
9
Perceptron (4)

XOR problem
Problem when
Data non-linearly separable

Solution: change activation function
For more details
Matlab classification toolbox
http://tiger.technion.ac.il/~eladyt/Classification_toolbox.html
10
Multilayer Perceptron (MLP)



Able to model
complex non-linear
functions
Hidden layers with
neurons
Backpropagation
algorithm
outputs
inputs
11
MLP (2)

f=sigmoid
y
w0
1
y   (x) 
a
1 e
w1
x1
w2
x2
n
a  w0   wi xi
i 1
12
MLP demo


Matlab Classification Toolbox
Handwritten digits classification

Discriminate between 10 digits
13
MLP demo (2)





Pre-processing
Feature extraction
Choice of neural network
Training
Test
Input
layer
For more details
See our program
1st hidden
layer
2nd hidden
layer
Output
layer
F
E
A
T
U
R
E
S
8 features
10
neurons
10
neurons
O
U
T
P
U
T
10
neurons
10
neurons
14
MLP performance



Able to model complex, nonlinear
mapping and classification
Can be trained by examples, no
mathematical description needed
In practice, shows good results
15
MLP limitations



Extensive training data must be
available
Computation time
Curse of dimensionality


Generalisation
Overfitting
To go further
See Neural Network Toolbox, demo on generalisation
16
A few applications




Medicine
Defence
Radar & Sonar
Finance …
17
Thank you.
18