Transcript Forward Pass Computations through a Back

Procedure for Training a Child to Identify a Cat using 10,000
Example Cats
For Cat_index  1 to 10000
1. Show cat and describe catlike features (Cat_index)
2. Child adjusts biological neural network in response to
receiving the features of example cat Cat_index
3. Cat_index  Cat_index + 1
Procedure for Testing a Trained Child’s ability to Identify a
new Cat
1. Show new cat and describe catlike features
2. Child processes features with biological neural network in
response to receiving the features of new example cat
3. Output of biological neural network indicates weather or
not new example is a cat
Smoothing function for converting the output of a neuron into the range [0,1]
1/(1 + 1/(e**x))
-10
-9
-8
-7
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
7
8
9
10
4.53979E-05
0.000123395
0.00033535
0.000911051
0.002472623
0.006692851
0.01798621
0.047425873
0.119202922
0.268941421
0.5
0.731058579
0.880797078
0.952574127
0.98201379
0.993307149
0.997527377
0.999088949
0.99966465
0.999876605
0.999954602
Logistics function for neuron activation
1.2
1
activation f(x)
x
0.8
0.6
Series1
0.4
0.2
0
-15
-10
-5
0
x
5
10
15
Forward Pass Computations through a Back-Propagation
Neural Network with three layers having 4, 6, and 8 nodes
INPUT input(1),input(2),input(3),input(4)
For i  1 to 6
middle_in (i)  0
For j  1 to 4
middle_in (i) < middle_in (i) + weight(j,i) * inp8ut (j)
middle_out (i)  Fermi (middle_in(i))
For k  1 to 8
output (k) 0
For i  1 to 6
output (k)  output (k) + weight (i,k) * middle_out (i)
INPUT known_true_value (k)
error (k)  known_true_value (k) – output (k)
General Procedure for training a neural network, then testing
it on new examples
INPUT known true values for each example
For i  1 to number_of_examples_in_input_set
INPUT numbers that measure values of input features for this example
INPUT known true classification values for this example
Do forward neural net computation to get outputs
Compute error by subtracting known true values from outputs
Set error_tolerance_threshold
Repeat until error tolerance <= error_tolerance_threshold
Do backpropagation for an epoch and adjust weights