Transcript Artificial Neural Network Theory and Application

Artificial Neural
Network Theory
and Application
Ashish Venugopal
Sriram Gollapalli
Ulas Bardak
Presentation Overview
Artificial Neural Networks
 NASA profiles as a Pattern
Classification Task
 Implementing Pattern Classification
on the Silicon Recognition Neuron
Board

ANN General Overview

Inspired by biological neuron models
Decision are distributed throughout
system
 Components interacts at several
stages
 Components work in unison to solve
one problem

ANN well suited to problems that don’t
have closed form algorithmic solution
 Black box model – hard to interpret

ANNs Applied

Driving a car. Task involves
determining the direction the road is
curving based on an image
Image pixels are input
 Curve angle is output
 Learned non-linear relationship
between pixels and roads

• hard to understand the model
Theory

Input layer, Hidden layer, Output
Layer
Input – one “neuron” for each pixel
 Hidden – set of neurons to store what
was learned.
 Output layer – one neuron for each of
5 turn directions – ie straight, slight
left, hard right..


But what is a neuron?
Neurons



Math terms: Binary decision makers
English terms: take in some input, decide
whether is “high” or “low”, and pass this
value on.
Similar to neurons in the brain propagating
signals.
How is the model learned?

Once you have training examples
Pair (feature set, classification)
 Run the back propagation algorithms

Learns connection weights between
neurons, as well as within neuron
transfer weights.
 Network topology changes affects
learning behavior!

Unsupervised Networks
So far we have used labeled training
examples to learn relationship
between features and response
variable.
 What if we are just exploring patterns
that exist in the features?
 Unsupervised learning / clustering…

Unsupervised Goal



Find prototypical points within data set that
can be used to approximate distribution of
the whole data set.
Compression / Visualization
Typical methods



Define a measure of similarity between data
points
Select initial prototypes
Move prototypes to best fit the data –
minimize error
Kohonen Net

Unsupervised technique

Each output neuron represents a
prototype point, input layer is
presented with data
Self Organizing Map
Each prototype is pulled towards that
data, and brings its closest prototypes
a little closer as well…
 SOM Demo

How is it relevant to our
problem?

We will go through:
Definition of what we are facing
 How we can use neural nets
 How we can improve on this method

LIDAR in Space

Data from LITE project
Send a laser beam down from a
shuttle and get reflection readings.
 Data gathered for 53 hours in 1994.
 40GB of data

Input
Problem

There is too much data to send and
process
Each pixel is a number to be
transmitted.
 Each reading (one column) contains
3000 pixels.


Need a way to transmit the
information in a more compact
fashion.
Applying Neural Nets
Some readings are very similar to
each other.
 Define classes that will contain
multiple readings.
 Define a representative for the class
which is close enough to all the class
members.

Applying Neural Nets (cont.)

In order to get class
Train Kohonen Nets on data for
specific number of classes.
 Once trained, just pass in a new
reading (3000 data points) and get the
class id. Just transfer the class id.
 Go from 3000x3000 numbers to be
transferred to 3000 numbers

Example
Result – 8 classes
Result – 16 Classes
Result – 64 Classes
Result – 128 Classes
Result – 256 Classes
Problems

Takes a while to train


256 classes took 44 hours on a dual
P3 1GHz computer with 1.5GB of
RAM.
Not optimal

Classifying by the complete reading is
wasteful.
• Need one number for each reading
Improving NN
sky
Improving NN (cont.)

Easier version:
Look at blocks of 50x50 pixels instead
of just 1 pixel.
 Classify as sky, etc.
 Can preprocess using kNN, basic
filters to get rid of noise in data.
 Classes can be formed from
sequence of 50 pixel blocks.

Improving NN (cont.)

Harder version:

Use variable sized mesh
• In parts with just the sky use a big block.
• In detailed parts use smaller blocks.

Something to think about…
Muren Board
Silicon Recognition, Inc.
 2 ZISC078 Chips (156 neurons in
parallel)
 1 Mb of memory
 Up to 1 million recognitions/second

ZISC

ZISC, the cornerstone of the Muren board,
completely overcomes the three major
limitations of software-based pattern
recognition applications:



ZISC fully handles non-linearity and
fuzziness
ZISC is a fully massively parallel processing
system. Recognition speed is consistent,
regardless of the number of stored patterns
ZISC does not use any algorithms for
programming the solution, ZISC is taught
RBF Space mapping
Example
Source picture
Result (after processing)
(circuit board)
(shows good and bad points)
Process
Learning
 Classifying
 Training
 Testing with new data

NASA Data
Further investigation
Can we write directly to the board?
 Ability to preprocess data?

Questions?