Transcript ppt

Neural Network and
its Applications
Dr. Priti Srinivas Sajja
Associate Professor
G H Patel P G Department of Computer Science and Technology
Sardar Patel University, Vallabh Vidyanagar 388 120, India
September 15, 2010
Priti Srinivas Sajja
1
Introduction and Contact Information

Name: Dr.

Communication:



Priti Srinivas Sajja
Email : [email protected]
Mobile : +91 9824926020
URL :http://pritisajja.info

Academic qualifications : Ph. D in Computer Science

Thesis title: Knowledge-Based Systems for SocioEconomic Development

Subject area of specialization : Artificial Intelligence

Publications : 84 in International/ National Conferences,
Journals & Books
September 15, 2010
Priti Srinivas Sajja
2
Lecture Plan
Introduction and Background




Artificial Intelligence and Data Pyramid
Connectionist and Symbolic Systems
Biological Neuron and Artificial Neuron
Characteristics of Artificial Neural Network
Architecture of ANN




Hopfield Model and Parallel Relaxation
Single Perceptron and Linearly Separable Models
Multi Layer Perceptron and Back Propogation
Supervised Learning and Training Data
Example ANNs
Hybrid Systems
Conclusion and References
September 15, 2010
Priti Srinivas Sajja
3
Artificial Intelligence
“Artificial Intelligence(AI) is the study
of how to make computers do
things at which, at the moment,
people are better”

September 15, 2010
Elaine Rich, Artificial Intelligence,
McGraw Hill Publications, 1986
Priti Srinivas Sajja
4
Knowledge-Based Systems
(Symbolic Representation of Knowledge)
Heuristics
and
models
Wisdom
Novelty
Knowledge
Rules
Experience
Information
K
Concepts
Data
Raw Data
through fact
finding
Researching
Understanding
Absorbing
Doing
Interacting
Reflecting
Convergence from data to intelligence
Knowledge-Based Systems (KBS) are Productive
Artificial Intelligence Tools working in a narrow
September 15, 2010
domain.
Priti Srinivas Sajja
5
Structure of KBS
Explanation/
Reasoning
Knowledge Base
Inference Engine
Self
Learning
User Interface
September 15, 2010
Priti Srinivas Sajja
6
Limitations of Symbolic Systems

Nature of knowledge




Knowledge acquisition




Limited knowledge structures support
KBS development models


Fact finding methods support only
Tacit and higher level knowledge
Multiple experts
Knowledge representation


Hard to characterize
Voluminous
Dynamic
Only SAD/SE guidelines and a few quality metrics
Large size of knowledge base
September 15, 2010
Priti Srinivas Sajja
7
Biological and Artificial Neurons
September 15, 2010
Priti Srinivas Sajja
8
Artificial Neural Network
(towards Connectionist Representation of Knowledge)

Objective: Not to mimic brain
functionality but to receive inspiration from
the fact about
working.

how brain is
Characterized by:


A large number of very simple neuron like processing
elements.
A large number of weighted connection between the
elements. This weights encode the knowledge of a network.

Highly parallel and distributed control.

Emphasis on learning internal representation automatically.
September 15, 2010
Priti Srinivas Sajja
9
Architectures of ANN





Hopfield network
Perceptron
Multi-layer Perceptron
Self Organizing Network
etc.
September 15, 2010
Priti Srinivas Sajja
10

In a Hopfield network, all processing units/elements are in two
states either

active or inactive.
Units are connected to each other with weighted Connections.
-1
Active
-1
+1
+3
-1
+1
+2


+3
-2
+1
Inactive
-1
A positively weighted connection indicates that the units tend
to active each other.
A negative connection allows an active unit to deactivate a
neighbouring unit.
September 15, 2010
Priti Srinivas Sajja
11
-1
Active
-1
+1
+3
-1
+1
+2
+3
-2
+1
Inactive
-1
• A random unit is chosen.
• If any of its neighbours are active, the unit computes the sum of
weights on the connections to those active neighbours.
• If the sum is positive, the unit becomes active else new random unit
is chosen.
• This process will continue till the network become stable. That is no
unit can change its status. This process is known as parallel
relaxation.
September 15, 2010
Priti Srinivas Sajja
12
Perceptron
X1
W1
T
∑Wi Xi
W2
X2
Importance
to Mom
Mom
0.6
0.6
∑Wi Xi
(0.3)
0.4
Dad
(0.5)
September 15, 2010
Importance
to Dad
= 0.3*0.6 + 0.5*0.4
= 0.18 +0.20
= 0.38 which is < 0.6
Priti Srinivas Sajja
13
Logical Gate AND and OR
X1
0.5
0.6
∑Wi Xi
0.5
X2
X1
X2
X1AND X2
0
0
0*0.5 + 0*0.5 = 0 <0.6
 0
0
1
0*0.5 + 1*0.5 = 0 <0.6
 0
1
0
1*0.5 + 0*0.5 = 0.5 <0.6
 0
1
1
1*0.5 + 1*0.5 = 1
 1
(1,1)
(1,1)
September 15, 2010
>0.6
Priti Srinivas Sajja
14
Logical Gate AND and OR
(1,1)
(1,1)
(1,1)
September 15, 2010
Priti Srinivas Sajja
15
1
X1
-1.5

1.0
ƒ
-9.0
1.0
X2
1
X1
-0.5

1.0
ƒ
1.0
X2
K
10
September 15, 2010
wo
.41
w1
w2
-.17
.14
100
.22
-.14
.11
300
-.1
-.008
.07
635
-.49
-.1
.14
Priti Srinivas Sajja
16
Output Layer
O1
O2
Oc
W2ij
Hidden Layer
h1
h2
h3
hB
w1ij
Input Layer
1
x1
x2
x3
x4
xA
……This network has three layers but there may be many.
September 15, 2010
Priti Srinivas Sajja
17
Examples of Multilayer Perceptron
O1
X1 and X2
are two one
/two
digited
positive
numbers
X1
H1
X2
H2
O2
O3
Training Set Data
2, 3, 1, 0, 0
10, 10, 0, 1, 0
90, 90, 0, 0, 1
…… ….. …
September 15, 2010
Priti Srinivas Sajja
18
O1
X1
H1
X2
H2
X3
H3
X4
H4
O2
O3
O4
O5
Training Set Data
2, 3, 0, 0,
1, 0, 0, 0, 0
6, 6, 7, 5,
0, 0, 1, 0, 0
8, 8, 8, 8,
0, 0, 0, 0, 1
…… ….. …
September 15, 2010
Priti Srinivas Sajja
19
O1
X1
H1
X2
H2
X3
H3
X4
H4
O2
O3
Design and Train the above structure using your own
choices considering the following practical situations:
A.
Job At Bengaluru, salary Rs.30, 000 per month of your field
B.
Job At USA, salary Rs.80, 000 per month of other field
C.
Job At Anand, salary Rs.25, 000 per month of your field
September 15, 2010
Priti Srinivas Sajja
20
September 15, 2010
Priti Srinivas Sajja
21
Source:
www.wldelft.nl/cons/area/wds/neural/index.html
September 15, 2010
Priti Srinivas Sajja
22
Sprayer Sensor and Nozzle Element (Zhang, Yang, & ElFaki, 1994)
September 15, 2010
Priti Srinivas Sajja
23

















Optimization, function approximation, time series prediction, and modeling
Classification, pattern matching and recognition (three-dimensional object recognition),
novelty detection, and sequential decision-making
Data processing (including filtering, clustering, blind source separation and compression),
data mining, data compression (speech signal, image—for example, faces), and data
validations
System identification and control (vehicle control, process control) and signal processing
Game playing and decision making (backgammon, chess, racing)
Sequence recognition (gesture, handwritten text recognition)
Medical diagnosis (for example, hepatitis or storing medical records based on case
information)
Financial applications (automated trading systems, time series analysis, stock market
prediction) and customer research
Cognitive science, neurobiology, and the study of models of how the brain works
Biological neural networks, which communicate through pulses and use the timing of the
pulses to transmit information and perform computations
Integration of fuzzy logic and neural networks for applications in automotive engineering,
screening applicants for jobs, controlling a crane, or monitoring a medical condition like
glaucoma
Robotics (navigation and vision recognition)
Speech production and recognition
Vision (face recognition, edge detection, visual search engines)
New topologies and hardware implementations
New learning algorithms
In hybrid systems and soft computing—for example, rule extraction for fuzzy systems,
self-evolving ANNs, and neuro-fuzzy systems
September 15, 2010
Priti Srinivas Sajja
24
Hybrid Systems/Soft Computing
Evolving neurofuzzy systems
Neuro-fuzzy and
fuzzy neural
network
Neural
network
Modular rough
networks
Genetic fuzzy
Fuzzy logic
Soft
computing
Genetic
algorithms
Probabilistic
reasoning
Genetic bayesian
network
Application Layer 1
Application Layer 2
Constituents of soft computing
September 15, 2010
Priti Srinivas Sajja
25
Strength of a Hybrid Soft
Computing System
Field
Strength Offered
Fuzzy logic
Approximate reasoning, impressions
Artificial neural network
Learning and implicit knowledge
representation
Genetic algorithm
Natural evolution and optimization
Probabilistic reasoning
Uncertainty
September 15, 2010
Priti Srinivas Sajja
26
Hybrid Applications
Fuzzy Rules
Fu
zz
y
Int
erf
ac
e
Output
FIS
Data
ANN
Fuzzy Sets
(b) co-operative neuro-fuzzy model
(a) fuzzy neural model
Y1
X1
Output
X2
Data
ANN


FIS
YN
Xn
(c) concurrent neuro-fuzzy model

Y2
(d) hybrid neuro-fuzzy model
Approaches of neuro-fuzzy computing
September 15, 2010
Priti Srinivas Sajja
27
Reference
Knowledge-Based Systems
Rajendra Akerkar and Priti Srinivas Sajja
Jones & Bartlett Publishers
Sudbury, MA, USA (2009)
September 15, 2010
Priti Srinivas Sajja
28
September 15, 2010
Priti Srinivas Sajja
29