Transcript Next Generation Techniques: Trees, Network and

Next Generation Techniques:
Trees, Network and Rules
What is a Decision Tree
• Decision tree is a predictive model that can
be viewed as a tree
• Specifically, each branch of the tree is a
classification question, and the leaves of the
tree are partitions of the dataset with their
classification
What is a Decision Tree (cont’d)
• Some interesting things about the tree:
• It divides up the data on each branch point
without losing any of the data
• The number of churners and non-churners is
conserved as you move up or down the tree
• It it pretty easy to understand how the model is
being built
• It is pretty easy to use this model
Applying Decision Tree to Business
• Because of their tree structure and capability to
easily generate rules, decision tree are the favored
technique for building understandable models
• Because of their high level of automation and the
ease of translating decision tree models into SQL
for deployment in relational databases, the
technology has also proven to be easy to integrate
with existing IT processes.
CART – growing a Forest and
picking the best tree
• CART which stands for Classification and
Regression Trees, is a data exploration and
prediction algorithm developed by Leo Breiman,
Jerome Friedman, Richard Olshen, and Charles
Stone
• Predictors are picked as they decrease the disorder
of the data
• In building the CART tree, each predictor is
picked based on how well it teases apart the
records with different predictions.
CHAID
• Another popular decision tree technology is
CHAID or Chi-Square Automatic
Interaction Detector.
• CHAID is similar to CART, but it differs in
the way that it chooses its splits.
What is Neural Network
• When data mining algorithms are talked
about these days, people usually talk about
either decision trees or neural networks.
• Of the two, neural networks have probably
been of greater interest through the
formative stages of data mining technology.
Are Neural Networks Easy to use?
• A common claim for neural networks is that they are
automated so that the user does not need to know that
much about how they work, about predictive modeling, or
even about the database in order to use them
• There are many important design decisions that need to be
made to effectively use a neural network, such as:
– How should the nodes in the network be connected?
– How many neuron-like processing units should be used?
– When should ‘training’ be stopped in order to avoid overfitting?
Applying Neural Networks to Business
• Neural Networks are very powerful predictive modeling
techniques, but some of the power comes at the expense of
ease-of-use and ease-of deployment
• The model itself is represented by numeric value in a
complex calculation that requires all of the predictor values
to be in the form of a number
• The output of the neural network is also numeric and needs
to be translated if the actual prediction value is categorical.
• For example, predicting the demand for blue, white, or
black jeans for a clothing manufacturer requires that the
predictor values blue, black, and white for the predictor
color be converted to numbers.
Applying Neural Networks to Business
(cont’d)
• The neural network model have been successfully
addressed in the following two ways:
– The neural network is packaged up into a complete
solution such as fraud prediction
– The neural network is packaged up with expert
consulting services.
What does a neural network look like
• A neural network is loosely based on the way
some people believe that the human brain is
organized and how it learns.
• There are two main structures of consequence in
the neural network:
– The node, which loosely corresponds to the neuron in
the human brain
– The link, which loosely corresponds to the connections
between neurons (axons, dendrites, and synapses) in the
human brain.
What does a neural network look like
(cont’d)
Rule Induction
• Rule induction is one of the major forms of
data mining and is perhaps the most
common form of knowledge discovery in
unsupervised learning systems. It also
perhaps the form of data mining that most
closely resembles the process that most
people think about when they think about
data mining, namely ‘mining’ for gold
through a vast database.
What to do with Rule
• When the rules are mined out the database, the rules can be
used either for understanding better the business problems
that the data reflects or for performing actual prediction
target.
• Because there is both a left side and right side to a rule
(antecedent and consequence) they can be used in several
ways in your business:
–
–
–
–
–
Target the antecedent
Target the Consequent
Target based on accuracy
Target based on coverage
Target based on ‘Interestingness’
Rules versus Decision Trees
• Decision trees also produce rules, but in a very
different way than rule induction systems.
• The main difference between the rules that are
produced by decision trees and rule induction
systems is as follows:
– Decision trees produce rules that are mutually exclusive
and collectively exhaustive with respect to the training
database
– Rule induction systems produce rules that are not
mutually exclusive and might be collectively exhaustive
Another commonality between decision trees
and rule induction systems
• One other thing that decision trees and rule induction
systems have in common is the fact that they both need to
find ways to combine and simplify rules
• In a decision tree, this can be as simple recognizing that if
a lower split on predictor is more constrained than a split
on the same predictor, both don’t need to be provided to
the user – only the more restrictive one.
• Rules from rule indication systems are generally created by
taking a simple high-level rule, and then adding new
constraints to it until the coverage gets to small so it is not
meaningful.