Transcript fgdfgdf - 哈尔滨工业大学个人主页

Introduction to Intelligent Systems
and Control
Prof Kang Li
Email: [email protected]
Prof. K. Li
Introduction
1 / 31
Topics
 Introduction to AI, IC, and artificial neural networks (1)
 Neural model and single-layer perceptrons (1)
 BP and Multilayer perceptron (MLP) (2)
 Radial basis function network (RBF) (1)
 Genetic algorithms (1)
Meta-heuristic optimization methods (2)
(1)= one lecture (2)=two lectures (3)=three lectures
Prof. K. Li
Introduction
2 / 31
Today’s Topics
 Getting to know about AI, CI and IC
Biological neuron networks and artificial neural
networks
Representations of artificial neurons and artificial
neural networks
Prof. K. Li
Introduction
3 / 31
What is Intelligence?
 The capacity to acquire and apply knowledge.
 The faculty of thought and reason.
 Superior powers of mind.
Three Questions on AI
1. Can the operations of the brain be simulated on a digital
computer? - yes, if the operation can be described clearly.
2. Is the mind a computer program? - No. Syntactical via semantic
3. Is the brain a digital computer? - ill-defined question
Prof. K. Li
Introduction
4 / 31
Computational Intelligence
- Problem solving methods and approaches that mimic
biologically intelligent behaviour
As a successor of AI, computational intelligence combines
elements of learning, adaptation, evolution and fuzzy logic
(rough sets) to create programs that are, in some sense,
intelligent.
Properties -
 Flexible to changing environments and goals
 Learns from experience
 Make appropriate choices given perceptual limitations
and finite computation
Prof. K. Li
Introduction
5 / 31
Taxonomy of Computational Intelligence
Neural
Networks
Evolutionary
Algorithms
Genetic
Algorithms
Fuzzy
Systems
Genetic
Programming
AI parts for IC
Prof. K. Li
Introduction
6 / 31
Intelligent Control
AI
IC
AC
Op
AI
Prof. K. Li
IC
Fu,
1970
K.S. Fu, Learning Control
Systems--Review and
Outlook, IEEE Transactions
on Automatic Control, 15,
210-221, April 1970.
Saradis,
1979
AC
Introduction
Saridis, G. (1979). "Toward
the Realization of Intelligent
Controls", Proceedings of the
IEEE, Vol. 67, No. 8, August
7 / 31
Intelligent control - the discipline where control methods are
developed that attempt to emulate important characteristics of
human intelligence: adaptation and learning, planning under
large uncertainty, coping with large amounts of data.
Intelligent system - to act appropriately in an uncertain
environment, where an appropriate action is that which increases
the probability of success. In order for a man-made intelligent
system to act appropriately, it may emulate functions of living
creatures and ultimately human mental faculties.
Prof. K. Li
Introduction
8 / 31
Classical control design
Reference
(Goal)
Disturbances
u
Controller
Process
Modelling
“System Identification”
Controller Design
and Synthesis
Mathematical
Model
Prof. K. Li
y
Introduction
Mathematical beauty
9 / 31
An example of Intelligent Control
Reference (Goal)
Disturbances
Manual control/
Human
operator
u
Human-in-loop
y
Process
Knowledge
acquisition
If-then
rules
Reference (Goal)
Design
and Synthesis
Prof. K. Li
.Neuro
/Fuzzy
controller
Introduction
Disturbances
u
Process
y
10 / 31
Biological neuron networks
Motivation from humans brains:
 Are able to process complex task efficiently (perception,
pattern recognition, reasoning etc.)
 Can learn from examples and generalise
 Adapt to new situations
 Robust and fault tolerant (neurons die)
Prof. K. Li
Introduction
11 / 25
Biological neuron networks
Unit nerve cells called neurons;
many different types and extremely
complex; around 1011 neurons in the
brain.
Interaction signal conveyed by
action potentials, interactions could be
chemical (release or receive ions) or
electrical. Each neuron contacts with
around 103 other neurons.
Structure feed-forward, feedback
and self-activation recurrent
Prof. K. Li
Introduction
12 / 25
Nerve cells
A neuron has three parts:
Dendrites - receive
information from another cell and
transmit the message to the cell
body.
Cell body - contains the
nucleus, mitochondria and other
organelles typical of eukaryotic
cells.
Axon- conducts messages away
from the cell body.
Synapse: The junction between a nerve cell and another cell is called a
synapse. Messages travel within the neuron as an electrical action potential.
The space between two cells is known as the synaptic cleft. To cross the
synaptic cleft requires the actions of neurotransmitters. Neurotransmitters are
stored in small synaptic vessicles clustered at the tip of the axon.
Prof. K. Li
Introduction
13 / 25
The artificial neural networks
A network with interactions, attempting to mimic the brain
Unit: artificial neuron (linear or nonlinear input-output unit), much
smaller numbers compared with brain.
Interaction: strength of interaction between artificial neurons are
determined by weights.
Structure: Feed-forward, feedback or recurrent
Prof. K. Li
Introduction
14 / 25
Artificial Neural Networks (cont.)
 ANNs are obtained by connecting artificial neurons together in a
layered structure.
 Networks can be either single or multilayered and be feedforward
or recurrent.
Hidden Layers
Output
Layer
Inputs
O u tpu ts
Input
Layer
Weighted links
Neuron
Single layer perceptrons
Multilayer feedforward network
Prof. K. Li
Introduction
15 / 25
Feedforward Neural Networks
 Feedforward neural networks consist of an input layer, one
or more intermediate layers and an output layer.
 The inputs to each neuron, in a given layer, are the outputs
of all the neurons in the previous layer.
 The network is termed feedforward because there are no
intra-layer connections, and inter-layer connections are
unidirectional.
 The input layer consists of dummy neurons which perform
no function other than to distribute the network inputs to the
next layer.
 The intermediate layers are referred to as hidden layers
because they are not connected directly to either the inputs
or outputs.
Prof. K. Li
Introduction
16 / 25
Recurrent Neural Networks (ANNs)
Recurrent networks are characterised by having at least one
delayed feedback loop.
This leads to nonlinear dynamical behaviour
D
D
Single layer recurrent network
(D = unit delay block)
= input nodes (dummy neurons)
Prof. K. Li
Introduction
17 / 25
Neural network learning
Learning (training from data set, adaptation)
change weights or interaction between neurons
according to examples, previous knowledge ...
The purpose of learning is to minimize
 Training errors on learning data - Training
performance
 Prediction errors on new, unseen data Generalisation performance
Prof. K. Li
Introduction
18 / 25
Learning Methods
1. Supervised learning- Has a teacher,
telling you how to learn. Technically
each example categorized, or
alternatively you receive feedback after
each decision
2. Unsupervised Learning- Learn by
itself. No feedback. The goal is to group
data into similar groups.
3. Other Approaches:
Reinforcement learning: have a
critics, wrong or correct
Prof. K. Li
Introduction
19 / 25
Issues in using neural networks
1. Choose a neural network architecture
– Feed-forward network
– Recurrent network
– Each network has its own characteristics
 Simple networks may not be able to solve a
complex problem
 For example, perceptron can only solve
linearly separable problems
Prof. K. Li
Introduction
20 / 25
Using Neural Networks – cont.
2. Specify the network architecture
– How many layers
– How many neurons at each layer
– A specific connection pattern
3. Choose a learning algorithm
– Specify the parameters of the learning algorithm
– Decide how to train the network
Prof. K. Li
Introduction
21 / 25
Using Neural Networks – cont.
4. Testing
– A trained network will be normally tested on data that
are not used during training
– A network’s ability to process outside training is
called generalization
Generalization is critical for practical applications
Generalization will be discussed later in detail
Prof. K. Li
Introduction
22 / 25
Artificial neuron model (typical representations)
Activation potential
u  1 u1  u n  1 u n 
T
Synapse

x
w
T
 b w 1  w n  1 w n 
Dendrite
Dendritic representation
Synapse
1 
 
u1


u
  
 
u n 
Input layer
Prof. K. Li
y
Summing node
b
w1
x

y
Signal flow graph
wn
Output layer
Introduction
23 / 25
u
n
W
w1
x

y
Block-diagram
T
y  ( w u  b ),
 
 
 
 
w1
u
 
 1
w    , u    
w 
u 
n
 
 n
 
 
Single neuron have 5 components
Input
Prof. K. Li
Weight
Bias
Activation function
Introduction
Output
24 / 25
Some typical activation functions
Activation function can generate either unipolar or bipolar signals
Prof. K. Li
Introduction
25 / 25
Summary
 Getting to know about AI, CI and IC
Biological neuron networks and artificial
neural networks
Representations of artificial neurons and
artificial neural networks
Prof. K. Li
Introduction
31 / 25