Transcript Artificial Neural Networks

Biological inspiration
Animals are able to react adaptively to changes in their
external and internal environment, and they use their nervous
system to perform these behaviours.
An appropriate model/simulation of the nervous system
should be able to produce similar responses and behaviours in
artificial systems.
The nervous system is build by relatively simple units, the
neurons, so copying their behavior and functionality should be
the solution.
Biological inspiration
Dendrites
Soma (cell body)
Axon
Biological inspiration
axon
dendrites
synapses
The information transmission happens at the synapses.
Artificial neurons
Neurons work by processing information. They receive and
provide information in form of spikes.
x1
w1
x2
Inputs
xn-1
xn
z   wi xi ; y  H ( z )
w2
x3
…
n
i 1
..
w3
.
wn-1
wn
The McCullogh-Pitts model
Output
y
Artificial neural networks
Inputs
Output
An artificial neural network is composed of many artificial
neurons that are linked together according to a specific
network architecture. The objective of the neural network
is to transform the inputs into meaningful outputs.
Learning in biological systems
Learning = learning by adaptation
The young animal learns that the green fruits are sour,
while the yellowish/reddish ones are sweet. The learning
happens by adapting the fruit picking behavior.
At the neural level the learning happens by changing of the
synaptic strengths, eliminating some synapses, and
building new ones.
Neural network mathematics
Inputs
Output
 y11  2
1
2
 y 32 
 1  y1  f ( y , w1 )
 2
2
3
y 12  f ( x 2 , w12 ) 1  y 2  2
2
1
2
y   1  y 2  f ( y , w2 ) y   y3  yOut  f ( y , w1 )
 2 
y 31  f ( x3 , w31 )
 y3  y 2  f ( y1 , w 2 )
y3 

1
3
3

 y4 
y 14  f ( x 4 , w14 )
y11  f ( x1 , w11 )
Neural network approximation
Task specification:
Data: set of value pairs: (xt, yt), yt=g(xt) + zt; zt is random
measurement noise.
Objective: find a neural network that represents the input /
output transformation (a function) F(x,W) such that
F(x,W) approximates g(x) for every x
Learning with MLP neural
networks
MLP neural network:
with p layers
x
yout
1 2 … p-1 p
1
2
N
(
x
,
y
),
(
x
,
y
),...,
(
x
, yN )
Data:
1
2
Error: E(t )  ( y(t ) out  yt ) 2  ( F ( x t ;W )  yt ) 2
It is very complicated to calculate the weight changes.
Learning with backpropagation
Solution of the complicated learning:
• calculate first the changes for the synaptic weights
of the output neuron;
• calculate the changes backward starting from layer
p-1, and propagate backward the local error terms.
The method is still relatively complicated but it
is much simpler than the original optimisation
problem.
Train = 1
Train = 1
.15
Error = 1-.15=.85
.72
.1*.3+.6*.2=.15
.1
.2*.4+.8*.8=.72
.6
.2
.3*1+.4*0=.3
.4
.2
1
1
0 .8 5
V alue
Weights
P erc ent
E rror
0 .1 5
0 .1
20%
0 .1 7
V alues
Weights
P erc ent
E rror
0 .3
0 .3
100%
0 .1 7
I nputs
.6*0+.8*1=.8
.8
.4
0
T rain
E rror
1
.4
.6
0
0 .6
80%
0 .6 8
0 .4
0%
0
0
.8
.4*0+.4*1=.4
.2*1+.8*0=.2
.3
Error = 1-.72=.28
0 .2
0 .2
100%
0 .6 8
0 .8
0%
0
1
T rain
E rror
1 .0 0
0 .2 8
V alue
Weights
P erc ent
E rror
0 .7 2
0 .2 0
11%
0 .0 3
0 .8
89%
0 .2 5
V alues
Weights
P erc ent
E rror
0 .4
0 .4
0%
0 .0 0
0 .4
100%
0 .0 3
I nputs
0
.8
1
0 .8
0 .6
0%
0 .0 0
0 .8
100%
0 .2 5
Train = 0
1.56
Error = 0-1.56=-1.56
.25*.85+.9*.1.5=1.56
.25 .9
.4*1+.45*1=.85
.6*1+.9*1=1.5
.4
.45
.6
1
.9
1
T rain
E rror
0
- 1 .5 6
V alue
Weights
P erc ent
E rror
1 .5 6
0 .2 5
14%
- 0 .2 1
0 .9
86%
- 1 .3 5
V alues
Weights
P erc ent
E rror
0 .8 5
0 .4
47%
- 0 .1 0
0 .4 5
53%
- 0 .1 1
I nputs
1
1
1 .5
0 .6
40%
- 0 .5 4
0 .9
60%
- 0 .8 1
Artificial Neural Network
Predicts
Structure
at this
point
Danger
You may train the network on your training
set, but it may not generalize to other data
Perhaps we should train several ANNs and
then let them vote on the structure
Profile network from HeiDelberg
family (alignment is used as input) instead of just the new
sequence
On the first level, a window of length 13 around the residue is
used
The window slides down the sequence, making a prediction for
each residue
The input includes the frequency of amino acids occurring in
each position in the multiple alignment (In the example, there
are 5 sequences in the multiple alignment)
The second level takes these predictions from neural networks
that are centered on neighboring proteins
The third level does a jury selection
PHD
Predicts 4
Predicts 5
Predicts 6