Transcript Unit 2

Basic Computer Application
Unit 2:Artificial neural network
Bo Li (李波)
[email protected]
Xi’an Jiaotong University
Basic Computer Application
Content
1.
An Experiment
2.
Biological neurons
3.
Artificial neuron
4.
Perceptron
5.
Artificial neural network
6.
Computational Intelligence
7.
Machine Learning
Basic Computer Application
1. An Experiment
Pigeons
experts
as art
(Watanabe et
al. 1995)
Experiment:
Pigeon
in Skinner box
Present
paintings of
two different artists
(e.g. Chagall / Van
Gogh)
Reward
for pecking
when presented a
particular artist (e.g.
Van Gogh)
Basic Computer Application
Experiment result

Pigeons were able to discriminate between
Van Gogh and Chagall with 95% accuracy


when presented with pictures they had been
trained on
Discrimination still 85% successful for
previously unseen paintings of the artists
Basic Computer Application
Why?

Pigeons do not simply memorise the
pictures

They can extract and recognise patterns
(the ‘style’)

They generalise from the already seen to
make predictions

This is what neural networks (biological
and artificial) are good at (unlike
conventional computer)
Basic Computer Application
2.Biological neurons
Basic Computer Application
Basic Computer Application
Schematic
Basic Computer Application
3.An artificial neuron

An artificial neuron is a mathematical
function conceived as a model of
biological neurons.

Artificial neurons are the
constitutive units in an artificial
neural network.

Depending on the specific model used
they may be called a semi-linear unit,
Nv neuron, binary neuron, linear
threshold function, or McCulloch–Pitts
Basic Computer Application

Neuron
vs.

Node
Basic Computer Application
Structure of a node:
Squashing function limits node output:
Basic Computer Application
Synapse vs. weight
Basic Computer Application
Component


3 Core parts

receives one or more inputs (representing
dendrites)

sums them

produce an output (representing a neuron's
axon).
Usually the sums of each node are
weighted, and the sum is passed
through a non-linear function known as
an activation function or transfer
function.
Basic Computer Application
2 functions

Transfer function



Type：

sigmoid shape

piecewise linear functions

step functions.
monotonically increasing, continuous,
differentiable and bounded.
Thresholding function

is inspired to build logic gates referred to as
threshold logic
Basic Computer Application
4.perceptron
Basic Computer Application
Learning/Training
Train
Set
Initial:
w1
= w2 = w3=0.5
Threshold=0.8
Rules
1
The
weights are
increased by 10% if
the output produced is
less than the output
data
The
weights are
decreased by 10% if
the output produced is
greater than the
output data.
Basic Computer Application
Training and Calculating

Training
1
0.8

Calculating
Basic Computer Application
Difficulty
Two classes of points, and two of the infinitely many linear
boundaries that separate them.
Even though the boundaries are at nearly right angles to one
another, the perceptron algorithm has no way of choosing between
them.
Basic Computer Application
5. Artificial Neural Networks?

Models of the brain and nervous system

Highly parallel

Process information much more like the brain
than a serial computer

Learning

Very simple principles

Very complex behaviours

Applications

As powerful problem solvers

As biological models
Basic Computer Application
ANNs – The basics
ANNs
incorporate
the two
fundamental
components of
biological neural
nets:
1.
Neurones (nodes)
2.
Synapses (weights)
Basic Computer Application
Multi-layer networks &Feed-forward
nets

Several layers of perceptions can be combined to create multilayer neural
networks.

The output from each layer becomes the input to the next layer.

The first layer is called the input layer, the middle layers are called the
hidden layers and the last layer is called the output layer.

Neural networks can be used when enough pre-established inputs and
outputs exist to train the network.
Basic Computer Application
Feeding data through the net
(1  0.25) + (0.5  (-1.5)) = 0.25 + (-0.75)
=
- 0.5
Squashing:
1
 0.3775
0.5
1 e
Basic Computer Application
Data

Data is presented to the network
activations in the input layer

Examples


Pixel intensity (for pictures)

Molecule concentrations (for artificial nose)

Share prices (for stock market prediction)
the
form
Data usually requires preprocessing


in
Analogous to senses in biology
How to represent more abstract data, e.g. a name?

Choose a pattern, e.g.

0-0-1 for “Chris”

0-1-0 for “Becky”
of
Basic Computer Application
Learning algorithms

Weight
settings
determine
behaviour of a network
the
 How can we find the right weights?
Basic Computer Application
Computational intelligence


Artificial Neural Networks

Connectionist Systems

Computational approach

Modeling the way the brain solves problems
Computational intelligence (CI)

the ability of a computer to learn a specific task
from data or experimental observation.

a set of nature-inspired computational
methodologies and approaches to address complex
real-world problems to which mathematical or
traditional modelling can be useless
Basic Computer Application
CI -five techniques


The methods used are close to the human's
way of reasoning

uses inexact and incomplete knowledge

able to produce control actions in an adaptive
way.
CI therefore uses a combination of five
main complementary techniques.

The fuzzy logic

artificial neural networks

evolutionary computing

learning theory

probabilistic methods
Basic Computer Application
Human Beings Intelligence

Computational Intelligence is thus a
way of performing like human beings.

Indeed, the characteristic of
"intelligence" is usually attributed
to humans.

More recently, many products and items
also claim to be "intelligent", an
attribute which is directly linked to
the reasoning and decision making.
Basic Computer Application
Machine learning

scientific discipline

explores the construction and study of algorithms
that can learn from data.

artificial intelligence and optimization

ML algorithms operate by


building a model based on inputs and using that to
make predictions or decisions

rather than following only explicitly programmed
instructions.
Example applications

spam filtering, optical character recognition (OCR),
search engines and computer vision.
Basic Computer Application
Summary
1.
An Experiment
2.
Biological neurons
3.
Artificial neuron
4.
Perceptron
5.
Artificial neural network
6.
Computational Intelligence
7.
Machine Learning
Basic Computer Application
Thank you
Bo Li (李波)
[email protected]
Xi’an Jiaotong University