Transcript Data Mining and Neural Networks

Data Mining and Neural
Networks
Danny Leung
CS157B, Spring 2006
Professor Sin-Min Lee
Artificial Intelligence for
Data Mining




Neural networks are useful for data mining and
decision-support applications.
People are good at generalizing from experience.
Computers excel at following explicit instructions
over and over.
Neural networks bridge this gap by modeling, on a
computer, the neural behavior of human brains.
2
Neural Network
Characteristics


Neural networks are useful for pattern
recognition or data classification, through a
learning process.
Neural networks simulate biological
systems, where learning involves
adjustments to the synaptic connections
between neurons
3
Anatomy of a Neural
Network



Neural Networks map a set
of input-nodes to a set of
output-nodes
Number of inputs/outputs is
variable
The Network itself is
composed of an arbitrary
number of nodes with an
arbitrary topology
Input 0
Input 1
...
Input n
Neural Network
Output 0
Output 1
...
Output m
4
Biological Background

A neuron: many-inputs / one-output unit

Output can be excited or not excited

Incoming signals from other neurons
determine if the neuron shall excite ("fire")

Output subject to attenuation in the
synapses, which are junction parts of the
neuron
5
Basics of a Node
 A node is an
element which
performs a function
Wb
y = fH(∑(wixi) +
Wb)
Input 0
Input 1
...
Input n
W0
W1
...
Wn
+
+
fH(x)
Connection
Output
Node
6
A Simple Preceptron

Binary logic application

fH(x) [linear threshold]

Wi = random(-1,1)

Y = u(W0X0 + W1X1
+ Wb)
Wb
Input 0
Input 1
W0
W1
+
fH(x)
Output
7
Preceptron Training

It’s a single-unit network

Adjust weights based on a how well the current weights
match an objective
Perceptron Learning Rule
Δ Wi = η * (D-Y).Ii
– η = Learning Rate
– D = Desired Output
8
Neural Network Learning



From experience: examples / training data
Strength of connection between the neurons
is stored as a weight-value for the specific
connection
Learning the solution to a problem =
changing the connection weights
9
Neural Network Learning

Continuous Learning Process

Evaluate output

Adapt weights

Take new inputs

Learning causes stable state of the weights
10
Learning Performance

Supervised
– Need to be trained ahead of time with lots of data

Unsupervised networks adapt to the input
–
–
–
–
Applications in Clustering and reducing dimensionality
Learning may be very slow
No help from the outside
No training data, no information available on the desired
output
– Learning by doing
– Used to pick out structure in the input:
– Clustering
– Compression
11
Topologies – BackPropogated Networks


Inputs are put
through a ‘Hidden
Layer’ before the
output layer
All nodes connected
between layers
...
Input 0
Input 1
H0
H1
...
Hm
O0
O1
...
Oo
Output 0
Output 1
...
Input n
Hidden Layer
Output o
12
BP Network – Supervised
Training

Desired output of the training examples

Error = difference between actual & desired output

Change weight relative to error size

Calculate output layer error , then propagate back to previous
layer

Hidden weights updated

Improved performance
13
Neural Network Topology
Characteristics

Set of inputs

Set of hidden nodes

Set of outputs

Increasing nodes makes network more
difficult to train
14
Applications of Neural
Networks

Prediction – weather, stocks, disease

Classification – financial risk assessment, image
processing

Data association – Text Recognition (OCR)

Data conceptualization – Customer purchasing
habits

Filtering – Normalizing telephone signals (static)
15
Overview

Advantages
– Adapt to unknown situations
– Robustness: fault tolerance due to network
redundancy
– Autonomous learning and generalization

Disadvantages
– Not exact
– Large complexity of the network structure
16
Referenced Work


Intro to Neural Networks - Computer Vision Applications and Training
Techniques. Doug Gray. www.soe.ucsc.edu/~taoswap/
GroupMeeting/NN_Doug_2004_12_1.ppt
Introduction to Artificial Neural Networks. Nicolas Galoppo von Borries.
www.cs.unc.edu/~nico/courses/ comp290-58/nn-presentation/ann-intro.ppt
17