Transcript Neural Computing Applics and Advanced AI

CHAPTER 16
Neural Computing Applications,
and Advanced Artificial Intelligent
Systems and Applications
1
Neural Computing Applications,
and Advanced Artificial Intelligent
Systems and Applications


Several Real-World Applications of ANN Technology
Advanced AI Systems
– Genetic Algorithms
– Fuzzy Logic
– Qualitative Reasoning

Integration (Hybrids)
2
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Areas of ANN Applications:
An Overview
Representative Business ANN Applications






Accounting
Finance
Human Resources
Management
Marketing
Operations
3
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Accounting


Identify tax fraud
Enhance auditing by finding irregularities
4
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Finance








Signatures and bank note verifications
Mortgage underwriting
Foreign exchange rate forecasting
Country risk rating
Bankruptcy prediction
Customer credit scoring
Credit card approval and fraud detection
Stock and commodity selection and trading
5
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Finance 2







Credit card profitability
Forecasting economic turning points
Bond rating and trading
Pricing initial public offerings
Loan approvals
Economic and financial forecasting
Risk management
6
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Human Resources


Predicting employees’ performance and behavior
Determining personnel resource requirements
7
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Management


Corporate merger prediction
Country risk rating
8
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Marketing







Consumer spending pattern classification
Customers’ characteristics
Sales forecasts
Data mining
Airline fare management
Direct mail optimization
Targeted marketing
9
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Operations








Airline crew scheduling
Predicting airline seat demand
Vehicle routing
Assembly and packaged goods inspection
Quality control
Matching jobs to candidates
Production/job scheduling
Factory process control
Many More
10
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Credit Approval
with Neural Networks

Increases loan processor productivity by 25 to
35% over other computerized tools

Also detects credit card fraud
11
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
The ANN Method

Data from the application and into a database

Preprocess applications manually

Neural network trained in advance with many
good and bad risk cases
12
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Neural Network Credit Authorizer
Construction Process

Step 1: Collect data

Step 2: Separate data into training and test sets

Step 3: Transform data into network inputs

Step 4: Select, train, and test network

Step 5: Deploy developed network application
13
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Bankruptcy Prediction
with Neural Networks
Concept Phase

Paradigm: Three-layer network, back-propagation

Training data: Small set of well-known financial ratios

Data available on bankruptcy outcomes

Supervised network

Training time not to be a problem
14
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Application Design

Five Input Nodes
X1: Working capital/total assets
X2: Retained earnings/total assets
X3: Earnings before interest and taxes/total assets
X4: Market value of equity/total debt
X5: Sales/total assets

Single Output Node: Final classification for each firm
– Bankruptcy or
– Nonbankruptcy

Development Tool: NeuroShell
15
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ

Development
– Three-layer network with backpropagation (Figure 16.3)
– Continuous valued input
– Single output node: 0 = bankrupt, 1 = not bankrupt

Training
– Data Set: 129 firms
– Training Set: 74 firms; 38 bankrupt, 36 not
– Ratios computed and stored in input files for:
• The neural network
• A conventional discriminant analysis program
16
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Architecture of the Bankruptcy Prediction
Neural Network
(Figure 16.3)
X1
X2
Bankrupt
0
X3
X4
Not bankrupt 1
X5
17
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ

Parameters
– Learning threshold
– Learning rate
– Momentum

Testing
– Two Ways
• Test data set: 27 bankrupt firms, 28 nonbankrupt firms
• Comparison with discriminant analysis
– The neural network correctly predicted:
• 81.5 percent bankrupt cases
• 82.1 percent nonbankrupt cases
18
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ

ANN did better predicting 22 out of the 27 actual cases

Discriminant analysis predicted only 16 correctly

Error Analysis
– Five bankrupt firms misclassified by both methods
– Similar for nonbankrupt firms

Neural network at least as good as conventional

Accuracy of about 80 percent is usually acceptable for
neural network applications
19
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Stock Market Prediction System with
Modular Neural Networks

Accurate Stock Market Prediction - Complex Problem

Several Mathematical Models - Disappointing Results

Fujitsu and Nikko Securities: TOPIX Buying and Selling
Prediction System
20
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ

Input: Several technical and economic indexes

Several modular neural networks relate past
indexes, and buy/sell timing

Prediction system
– Modular neural networks
– Very accurate
21
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Network Architecture
(Figure 16.4)

Network Model: 3 layers, standard sigmoid function,
continuous output [0, 1]

High-speed Supplementary Learning Algorithm

Training Data
– Data Selection
– Training Data
22
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ

Preprocessing: Input Indexes - Converted into spatial
patterns, preprocessed to regularize them

Moving Simulation Prediction Method (Figure 16.5)

Result of Simulations
– Simulation for Buying and Selling Stocks
– Example (Figure 16.6)
– Excellent Profit
23
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Integrated ANNs and
Expert Systems
1. Resource Requirements Advisor
– Advises users on database systems’ resource requirements
– Predicts the time and effort to finish a database project
– ES shell AUBREY and neural network tool NeuroShell
– ES supported data collection
– ANN used for data evaluation
– ES final analysis
24
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
2. Personnel Resource Requirements Advisor
– Project personnel resource requirements for maintaining networks
or workstations at NASA
– Rule-based ES determines the final resource projections
– ANN provides project completion times for services requested
(Figure 16.7)
25
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
3. Diagnostic System for an Airline
– Singapore Airlines
– Assists technicians in diagnosing avionics equipment
– INSIDE (Inertial Navigation System Interactive Diagnostic Expert)
– Designed to reduce the diagnostic time
(Figure 16.8)
26
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
4. Manufacturing Product Liability
–
–
–
–
United Technologies Carrier
Two ES + ANN
Patterns fed into multilayer feedforward ANN
Integrated with a database into an Automatic Early Warning
System (AEWS)
27
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
5. Oil Refinery Production Scheduling and
Environmental Control
–
–
–
–
–
Citgo Petroleum Corporation
Lower costs
Improved safety
Higher product quality
Higher yields
28
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Genetic Algorithms



Goal (evolutionary algorithms): Demonstrate selforganization and adaptation by exposure to the
environment
System learns to adapt to changes.
Example 1: Vector Game
– Random trial and error
– Genetic algorithm solution


Process (Figure 16.9)
Example: the game of MasterMind
29
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Genetic Algorithm
Definition and Process
Genetic algorithm: "an iterative procedure maintaining a
population of structures that are candidate solutions to
specific domain challenges” (Grefenstette [1982])

Each candidate solution is called a chromosome

Chromosomes can copy themselves, mate, and mutate

Use specific genetic operators - reproduction, crossover
and mutation
30
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Primary Operators of
Most Genetic Algorithms

Reproduction

Crossover

Mutation
31
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Genetic Algorithm Operators
Parent 1
1 0 1 0 1 1 1
Parent 2
1 1 0 0 0 1 1
Child 1
1 0 1 0 0 1 1
Child 2
1 1 0 0 1 1 0
Mutation
32
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
GA Example: The Knapsack Problem









Item:
1 2 3 4 5 6 7
Benefit: 5 8 3 2 7 9 4
Weight: 7 8 4 10 4 6 4
Knapsack holds a maximum of 22 pounds
Fill it to get the maximum benefit
Solutions take the form of a string of 1’s
Solution: 1 1 0 0 1 0 0
Means choose items 1, 2, 5. Weight = 21, Benefit = 20
Evolver solution in Figure 16.10
33
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Genetic Algorithms
Applications and Software

Type of machine learning

Set of efficient, domain-independent search
heuristics for a broad spectrum of applications
34
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Genetic Algorithm Application Areas











Dynamic process control
Induction of rule optimization
Discovering new connectivity topologies
Simulating biological models of behavior and evolution
Complex design of engineering structures
Pattern recognition
Scheduling
Transportation
Layout and circuit design
Telecommunication
Graph-based problems
35
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Business Applications

Channel 4 Television (England) to schedule commercials
Driver scheduling in a public transportation system
Jobshop scheduling
Assignment of destinations to sources
Trading stocks
Productivity in whisky-making is increased

Often genetic algorithm hybrids with other AI methods





36
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Representative Commercial Packages





Evolver (Excel spreadsheet add-in)
Genetic Algorithm User Interface (GAUI)
OOGA (Object-Oriented GA for industrial use)
XperRule Genasys (ES shell with an embedded genetic
algorithm)
Sugal Genetic Algorithm Simulator
37
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Optimization Algorithms

Via neural computing sometimes

Genetic algorithms and their derivatives can optimize
(or nearly optimize) complex problems
38
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Fuzzy Logic

Fuzzy logic deals with uncertainty

Uses the mathematical theory of fuzzy sets

Simulates the process of normal human reasoning

Allows the computer to behave less precisely and
logically
Decision making involves gray areas and the term maybe
39
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Fuzzy Logic Advantages









Provides flexibility
Provides options
Frees the imagination
More forgiving
Allows for observation
Shortens system development time
Increases the system's maintainability
Uses less expensive hardware
Handles control or decision-making problems not
easily defined by mathematical models
40
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Fuzzy Logic Example:
What is Tall?
In-Class Exercise
Proportion
Height
Voted for
5’10”
0.05
5’11”
0.10
6’
0.60
6’1”
0.15
6’2”
0.10

– Jack is 6 feet tall
– Probability theory - cumulative probability
– There is a 75 percent chance that Jack is tall
41
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ




Fuzzy logic - Jack's degree of membership within the
set of tall people is 0.75
We are not completely sure whether he is tall or not
Fuzzy logic - We agree that Jack is more or less tall
Membership Function
< Jack, 0.75  Tall >

Knowledge-based system approach: Jack is tall (CF
= .75)
Belief functions

Can use fuzzy logic in rule-based systems

42
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Membership Functions in Fuzzy Sets
(Figure 16.11)
Short
Medium
Tall
1.0
Membership
0.5
64
74
69
Height in inches (1 inch = 2.54 cm)
43
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Fuzzy Logic Applications and
Software

Difficult to apply when people provide evidence

Used in consumer products that have sensors
–
–
–
–
–
Air conditioners
Cameras
Dishwashers
Microwaves
Toasters

Special software packages

Controls applications
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
44
Examples of Fuzzy Logic
Example 1: Strategic planning
– STRATASSIST - fuzzy expert system that helps small- to
medium-sized firms plan strategically for a single product
Example 2: Fuzziness in real estate
Example 3: A fuzzy bond evaluation system
45
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Fuzzy Logic Software




Fuzzy Inference Development Environment
(FIDE)
Z Search
HyperLogic Corporation demos
Others
46
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Qualitative Reasoning (QR)
– Means of representing and making inferences using
general, physical knowledge about the world
– QR is a model-based procedure that consequently
incorporates deep knowledge about a problem
domain
– Typical QR Logic
• “If you touch a kettle full of boiling water on a stove, you
will burn yourself”
• “If you throw an object off a building, it will go down”
47
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ

But

No specific knowledge about boiling temperature, just
that it is really hot!

No specific information about the building or object,
unless you are the object, or you are trying to catch it
48
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ

Main goal of QR: To represent common sense knowledge
about the physical world, and the underlying
abstractions used in quantitative models (objects fall)

Given such knowledge and appropriate reasoning
methods, an ES could make predictions and diagnoses,
and explain the behavior of physical systems
qualitatively, even when exact quantitative descriptions
are unavailable or intractable
49
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Qualitative Reasoning

Relevant behavior is modeled

Temporal and spatial qualities in decision making are
represented effectively

Applies common sense mathematical rules to variables
and functions

There are structure rules and behavior rules
50
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Some Real-World QR Applications

Nuclear plant fault diagnoses

Business processes

Financial markets

Economic systems
51
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Intelligent Systems Integration

Combine
–
–
–
–


Neural Computing
Expert Systems
Genetic Algorithms
Fuzzy Logic
Example: International investment management-stock selection
Fuzzy Logic and ANN (FuzzyNet) to forecast the
expected returns from stocks, cash, bonds, and other
assets to determine the optimal allocation of assets
52
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ


Global markets
Integrated network architecture of the system
(Figure 16.12)
Technologies



Expert system (rule-based) for country and stock selection
Neural network for forecasting
Fuzzy logic for assessing factors without reliable data
53
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
FuzzyNet Architecture
(Figure 16.12)

Membership Function Generator (MFG)

Fuzzy Information Processor (FIP)

Back-propagation Neural Network (BPN)
54
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Data Mining and Knowledge
Discovery in Databases (KDD)


Hidden value in data
Knowledge Discovery in Databases (KDD)
55
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
The KDD Process
Start with Raw Data and Do
1. Selection to produce target the appropriate data
which undergoes
2. Preprocessing to filter the data in preparation for
3. Transformation so that
4. Data Mining can identify patterns that go through
5. Interpretation and Evaluation resulting in
knowledge
56
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Data Mining

Find kernels of value in raw data ore

Theoretical advances
– Knowledge discovery in textual databases
– Methods based on statistics, cluster analysis, discriminant
analysis, fuzzy logic, genetic algorithms, and neural networks
– Ideal for data mining
57
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
AI Methods and Data Mining
for Search

Neural Networks

Expert Systems

Rule Induction
58
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Data Mining Applications Areas

Marketing

Investment

Fraud detection

Manufacturing
59
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Information Overload

Data mining methods can sift through soft
information to identify relationships automatically

Intelligent agents
60
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ
Important KDD and
Data Mining Challenges

Dealing with larger databases

Working with higher dimensionalities of data

Overfitting--modeling noise rather than data patterns

Assessing statistical significance of results

Working with constantly changing data and knowledge
Continue
61
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ

Working through missing and noisy data

Determining complex relationships between fields

Making patterns more understandable to humans

Providing better user interaction and prior knowledge
about the data

Providing integration with other systems
62
Decision Support Systems and Intelligent Systems, Efraim Turban and Jay E. Aronson
6th ed, Copyright 2001, Prentice Hall, Upper Saddle River, NJ