Transcript SOLARcief2003

SELF-ORGANIZING LEARNING ARRAY
(SOLAR)
AND ITS APPLICATION TO ECONOMIC AND
FINANCIAL PROBLEMS
by Janusz Starzyk, Zhen Zhu, Haibo He
and Zhineng Zhu
School of EECS
Ohio University, Athens, OH
3rd International Workshop on
Computational Intelligence in Economics and Finance
CIEF'2003
Cary, NC, September 30th, 2003
OUTLINE
– Motivation
– SOLAR and its organization
– Self-organizing principle
– Evolution of SOLAR structure
– Data processing
– Examples of economic and financial
applications
– Future work and conclusion
CIEF'2003
SELF-ORGANIZING LEARNING ARRAY
What is SOLAR?

New Biologically Inspired Network Organization
Basic Fabric:
A fixed lattice of distributed, parallel processing units
(neurons)
Self-organization Algorithm:

Interconnections among neurons are dynamically


CIEF'2003
refined.
Neurons are dynamically re-configured.
Number of neurons used is decided by problem
complexity.
Why do we need SOLAR?










Needed a general purpose learning network
Network that learns without algorithm
Network that runs without software
Network that is data driven
Network that self-organizes
Network that learns through associations
Network that acts with self awareness
Network that scales to a very large system
Network that is fault tolerant
Network that is modular
CIEF'2003
SOLAR-Organization







Neurons organized in a
cell array
Sparse randomized
connections
Local self-organization
Data driven
Entropy based learning
Regular structure
Suitable for large scale
circuit implementation
CIEF'2003
Structure of a single neuron
RPU: reconfigurable processing unit
CU: control unit
DPE: dynamic probability estimator
EBE: entropy based evaluator
DSRU: dynamic self-reconfiguration
memory.
NI/NO: Data input/output
CI/CO: Control input/output
CIEF'2003
Self-organizing connections
Each neuron is pseudorandomly connected to
other neurons or primary inputs.
O: processing unit
: data
: control
CIEF'2003
Learned Structure
Using a entropy-based metric, some of the
connections would be found carrying more
information and thus saved while other cut out.
CIEF'2003
Self-organizing Principle
Information deficiency
Information index

Es
I  1
 1 s c
Emax
Psc log( Psc )   Ps log( Ps )
s
 P log( P )
c
c
c
Es
s 

Emax
p
sc
log( psc )  ps log( ps )
sc
p
c
log( pc )
c
If the space was divided, then the
information index and information
deficiencies are related as
1 I   s
s
When subsequent space partitions take
place, the information deficiency is
expressed as a product of information
deficiencies in each subsequent
partition.
 p   s
p
CIEF'2003
Neuron and its I/O Ports

A priori information – class probabilities
and input information deficiencies
Transformation functions
with thresholds, cut input
space into two sub-spaces.
Neuron self organize to
maximize information
deficiency reduction:
  I * in
Learning stops when:
 in  
CIEF'2003
Neuron’s action

Through learning, one pair of function and
threshold will be chosen and solidified.

This will provide a cut in the input space to
separate different classes.
CIEF'2003
Neuron’s action
Other neurons can provide other cuts.
Most confident neurons vote on final classification.
CIEF'2003
SOLAR Behavior

Behavior of a single neuron:

Calculates its data and control outputs and provides
them as inputs to others.
Computes statistical information (for example, entropy
based information deficiency) in its subspaces.
Makes associations with other neurons.



Behavior of the network:
 Clusters of neurons solve the problem.
 Network connections are active elements of learning.
 The system gathers information from all neurons and
makes final decision based on well trained neurons
CIEF'2003
SOLAR-Data Processing

SOLAR receives input data as a 2-D matrix of
feature samples.
 This matrix is pre-processed to make sure:
– Missing parts are recovered
– Symbolic data is represented in a proper way

Data into each neuron is re-scaled for full
resolution
CIEF'2003
SOLAR-Data Processing
Pre-processing
input data
Initialize SOLAR
Neurons work in
parallel
System collects
information and
makes final
decision
CIEF'2003
Training:
Each neuron selects
most efficient
Testing:
inputs, function,
threshold
and outputs and pass
Neurons
calculate
Testing:
solidify
them.
them
to others.
Neurons calculate outputs and pass
F1=f1(x,y)
them on.
F2=f2(x,y)
F3=f3(x,y)
F4=f4(x,y)
Decision Making
F5=f5(x,y)
SOLAR-Data Processing

We may use an ensemble of SOLAR
networks to vote on the same target:
CIEF'2003
Biologically Inspired NN
NN
CNN
SOLAR
Y
Y
Y
Y
Y
Y
Y
Y
Y
Local Interconnect
Y
Y
Complexity Driven Hardware Use
Y
Y
Massively Parallel
Local Decision Making
Data Driven
Self Organization
CIEF'2003
Y
SOLAR-Examples

Bankruptcy Prediction
– Amir F.Atiya, “Bankruptcy Prediction for Credit Risk Using
Neural Networks: A survey and New Results,” IEEE Trans. on
Neural Networks, Vol. 12, No. 4, July, 2001.

Credit Card Approval Decision
 Loan Decision-Australian Adult Income
Classification
– D. Michie, D. J. Spiegelhalter, and C. C. Taylor, “Machine
Learning, Neural and Statistical Classification” London, U. K.
Ellis Horwood Ltd. 1994.
– ftp at cs.uci.edu (128.195.1.1)
CIEF'2003
SOLAR-Example

The Bankruptcy Dataset:
– 716 solvent US corporations
– 195 defaulted ones (within 1 to 36 months)
– Expanded into 1160 points by taking different instances
before the default
– Over 60 available indicators (features)

Financial Ratio and Equity-based Indicator
System:
– Based on financial ratios and prices
– Proved superior to traditional indicators
– Optimal indicator pool based on traditional ANNs
CIEF'2003
SOLAR-Example
Bankruptcy prediction using SOLAR
Time to default
correct rate %
of SOLAR
6 month or less
Reported
correct
rate %
86.15
85.11
correct rate %
of SOLAR
using all
87.23
6 to 12 months
81.48
84.09
86.36
12 to 18 months
74.60
76.19
90.24
18 to 24 months
78.13
55.17
72.24
more than 24 months
66.67
64.29
75.00
total defaulted
78.13
75.13
83.96
solvent
90.07
92.74
93.42
total
85.50
85.80
90.04
CIEF'2003
SOLAR-Example

Using the same “Financial Ratio and Equitybased Indicator System”, SOLAR shows
equal performance to traditional ANNS.

SOLAR is capable of handling all indicators
and gives even better performance.
CIEF'2003
SOLAR-Example
prewired structure
CIEF'2003
SOLAR-Example
learned connections
CIEF'2003
SOLAR-Example
Detail of prewired structure
Detail of learned connections
CIEF'2003
SOLAR-Examples

Credit Card Approval Decision and Loan
Decision
– Benchmark problems used to compare
performance various methods in literature
– learning algorithms, neural networks, statistical
methods and SOLAR are compared
CIEF'2003
SOLAR-Example
Performance Comparison on
Credit Card Approval Decision
Algorithm
Mis-prob.
Algorithm
Mis-prob.
CAL5
0.131
CART
0.145
RBF
0.145
SOLAR
0.135
Itule
0.137
CASTLE
0.148
DIPOL92
0.141
Naivebay
0.151
Logdisc
0.141
IndCART
0.152
Discrim
0.141
Bprop
0.154
CIEF'2003
SOLAR-Example
Performance Comparison on Loan Decision
Algorithm
Mis-prob.
Algorithm
Mis-prob.
FSS Naive Bayes
0.1405
CN2
0.1600
NBTrees
0.1410
Naive-Bayes
0.1612
C4.5-auto
0.1446
Voted ID3 (0.8)
0.1647
IDTM(Decision table)
0.1446
T2
0.1687
HOODG/SOLAR
0.1482
1R
0.1954
C4.5 rules
0.1494
0.2035
OC1
0.1504
C4.5
0.1554
Nearest-Neighbor
(3)
Nearest-Neighbor
(1)
Pebls
Voted ID3 (0.6)
0.1564
CIEF'2003
0.2142
Crashed
Conclusion

SOLAR is a new biologically inspired learning
network organization.
 Efficient, general purpose, capable of large data
sets, suitable for hardware implementation
 Used to solve financial and economic problems,
prediction, identification, and decision making.
CIEF'2003
Future work

Applications to other fields
 Hardware implementation in FPGA
– real time applications
– modular and expandable structures

Associative learning
 Temporal learning
 Stability issues
 Autonomous, goal driven behavior
CIEF'2003
Questions
?
CIEF'2003