Transcript Neural Computing

Neural Networks II
By
Jinhwa Kim
Neural Computing: The Basics

2
Neural Computing is a problem solving
methodology that attempts to mimic
how human brain function

Artificial Neural Networks (ANN)

Machine Learning
Neural Computing

Computing technology that mimic certain
processing capabilities of the human brain
Knowledge representations based on
 Massive parallel processing
 Fast retrieval of large amounts of information
 The ability to recognize patterns based on
historical cases
Neural Computing = Artificial Neural Networks (ANNs)

Purpose of ANN is to simulate the thought process
of human brain

Inspired by the studies of human brain and the
nervous system

3
The Biology Analogy

Neurons: brain cells




4
Nucleus (at the center)
Dendrites provide inputs
Axons send outputs
Synapses increase or
decrease connection
strength and cause
excitation or inhibition of
subsequent neurons
Artificial Neural Networks (ANN)




5
A model that emulates a biological neural
network
Software simulations of the massively
parallel processes that involve processing
elements interconnected in a network
architecture
Originally proposed as a model of the human
brain’s activities
The human brain is much more complex
Artificial Neural Networks (ANN)
Three Interconnected Artificial Neurons
Biological
Soma
Dendrites
Axon
Synapse
Slow speed
Many neurons
(Billions)
6
<->
<->
<->
<->
<->
<->
Artificial
Node
Input
Output
Weight
Fast speed
Few neurons
(Dozens)
ANN Fundamentals

Components and Structure


“A network is composed of a number of processing elements
organized in different ways to form the network structure”
Processing Elements (PEs) – Neurons
Network


Structure of the Network

Figure 15.3
7
Collection of neurons (PEs) grouped in layers
Topologies / architectures – different ways to interconnect PEs
ANN Fundamentals
8
ANN Fundamentals

Processing Information by the Network

Inputs
Outputs
Weights
Summation Function

Figure 15.5



9
ANN Fundamentals

Transformation (Transfer) Function



10
Computes the activation level of the neuron
Based on this, the neuron may or may not produce an output
Most common: Sigmoid (logical activation) function
Learning in ANN
1.
2.
3.
11
Compute outputs
Compare outputs with
desired targets
Adjust the weights and
repeat the process
Data Collection and Preparations




12
Collect data and separate it into
 Training set (50%), Testing set
 Training set (60%), Testing set
 Training set (70%), Testing set
 Training set (80%), Testing set
 Training set (90%), Testing set
(50%)
(40%)
(30%)
(20%)
(10%)
Make sure that all three sets represent the
population: true random sampling
Use training and cross validation cases to adjust the
weights
Use test cases to validate the trained network
Neural Network Architecture

13
There are several ANN architectures
:feedforward, recurrent, Hopfield et al.
Neural Network Architecture

Feed forward Neural Network

14
Multi Layer Perceptron, - Two, Three, sometimes
Four or Five Layers
How a Network Learns


Step function evaluates the summation of
input values
Calculating outputs
Measure the error (delta) between outputs and
desired values
 Update weights, reinforcing correct results
At any step in the process for a neuron, j, we get
Delta(Error) = Zj - Yj
where Z and Y are the desired and actual outputs,
respectively

15
Backpropagation






Backpropagation (back-error propagation)
Most widely used learning
Relatively easy to implement
Requires training data for conditioning the
network before using it for processing other
data
Network includes one or more hidden layers
Network is considered a feedforward
approach
Continue
16
Backpropagation
Initialize the weights
Read the input
vector
Generate the output
Compute the error
Error = Output –
Desired output
Change the weights
1.
2.
3.
4.
5.

Drawbacks:


17
A large network can take a very long time to train
May not converge
Testing




18
Test the network after training
Examine network performance: measure the
network’s classification ability
Black box testing
Do the inputs produce the appropriate outputs?
ANN Development Tools











19

E-Miner
Clementine
NeuroSolutions
Statistica Neural Network Toolkit
Braincel (Excel Add-in)
NeuralWorks
Brainmaker
PathFinder
Trajan Neural Network Simulator
NeuroShell Easy
SPSS Neural Connector
NeuroWare
Benefits of ANN
Diverse Applications:

Pattern recognition, learning, classification,
generalization and abstraction, and interpretation of
incomplete and noisy inputs

Character, speech and visual recognition
20
Advantages:

Can provide some human problem-solving
characteristics

Can tackle new unseen kinds of problems

Robust

Fast decision making
Limitations of ANN




21
Lack explanation capabilities
Limitations and expense of hardware
technology restrict most applications to
software simulations
Training time can be excessive and
tedious
Black box that is hardly understood by
human
Business ANN Applications

Accounting



Finance










22
Identify tax fraud
Enhance auditing by finding irregularities
Signatures and bank note verifications
Foreign exchange rate forecasting
Bankruptcy prediction
Customer credit scoring
Credit card approval and fraud detection*
Stock and commodity selection and trading
Forecasting economic turning points
Pricing initial public offerings*
Loan approvals
…
Business ANN Applications

Human Resources



Management





Consumer spending pattern classification
Sales forecasts
Targeted marketing, …
Operations


23
Corporate merger prediction
Country risk rating
Marketing


Predicting employees’ performance and behavior
Determining personnel resource requirements
Vehicle routing
Production/job scheduling, …