Transcript 7. Decision Trees and Decision Rules

國立雲林科技大學
National Yunlin University of Science and Technology
Multilevel Category Structure in
the ART-2 Network
Advisor ：Dr. Hsu
Graduate： Yu Cheng Chen
Author: Michael P. Davenport, Albert H.
IEEE Transactions on Neural Network (2004)
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Outline







Motivation
Objective
Introduction
Background
ART2 Neural Network
Analysis of Multilevel Category Structure
Conclusions
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Motivation

Real-world problems require that categorization is
performed on complex data sets, which points to the
need for a modular approach to systems design.

The variations of the ART network have proliferated
during the last decade, very few if any of those
variations have emphasized psychological or
neurobiological principles in their design.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Objective

By exchanging a single node category description for a more
complex category pattern description, the distribution of
information processing is distributed beyond a single ART
module, which is more consistent with studies of
neurobiological organization.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Introduction

These categories are represented by prototype
patterns.

In this work, we present a detailed analysis of an ART
2 network that is specifically tuned to extract
secondary level features from an input data set.

Thus, ART2 network can be used to show not just
what broad category an input belongs to, but also
what features an input has that makes it different or
similar to other inputs.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Background

Review of ART Theory
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Background

The template STM layer forms a prototype or template category
based on the previous experience stored in the LTM weights.

The category output of the network is based on the
interpretation of activity across the template layer (also
referred to as the output layer or the category layer).

Throughout the paper we use the terms single winning
node and multiple winning nodes. These are nodes in
the template or output layer.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Background

The term single winning node indicates that for every
input pattern, the same output node always has the
highest activity level.

The term multiple winning nodes indicates that while
only one output node will have the highest level of
activity for a given input pattern, not all the input
patterns will cause the same output node to be most
active.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Background

MultiLevel Categorization

Several variations of ART networks have been
developed to perform multilevel categorization.

─
SMART、HART
SMART configured each individual ART network to
have a different value for the vigilance parameter.

generate a pair of categories

more general category and more specific subcategory.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Background

In all the above models the ART networks share a
common characteristic: they represent the category
output as a single node — the most active node of the
network.

In this paper, we demonstrate that the elements of a
multilevel categorization can be obtained by using a
single ART 2 network, and that this information is
obtained by considering the pattern of activity across
all the output nodes instead of considering only the
most active.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Background

Psychological and Neurobiological Principles of
Categorization

Psychological Categorization:
─
─
First, any system performing categorization should do so
efficiently; the maximum amount of useful information should be
obtained using a minimal amount of cognitive effort.
second, an organism perceives a high correlational structure among
the features, attributes, objects, events, etc.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Background

Category structures vary across different levels of
abstraction.


Categories within the same abstraction level are also characterized
by not having clearly defined category boundaries which implies
that every pattern is not equally representative of the category to
which it belongs.
We demonstrate in our results that category
representation as an analog pattern of activity can be
described in a similar manner.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Background
Neurobiological
Categorization:

Neurobiological studies indicate that many different and widely
separated neural structures are involved in category learning.

Different neural structures may also exist for learning categories
depending upon the type of category learning.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
ART2 Neural Network
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
ART2 Neural Network

Each individual layer F1 is described by a two-step
process (1)–(3).


First signals are combined and enhanced by gain parameters
Second, the overall activity across the layer is normalized to keep
the total activity of the entire network bounded.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
ART2 Neural Network

The template STM layer in Fig. 1 is represented in
Fig. 2 by the F2 layer (category layer).
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
ART2 Neural Network

Model Parameter Selection



a=0.5,b=0.5,c=0.1,d=0.5,e=0.0001
We have used the “zoo” and the “auto-mpg” databases, which are
available at the University of California, Irvine, Information and
Computing Science.
The dataset used was the “zoo” database consisting of 100 patterns.
Each pattern consists of fourteen binary elements representing
features associated with the animals: hair, feathers, eggs, milk,
airborne, aquatic…
Intelligent Database Systems Lab
Analysis of Multilevel Category Structure

Table 1
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Analysis of Multilevel Category Structure
N.Y.U.S.T.
I. M.

The first case studies patterns generated by ART
network where the vigilance parameter is set low,
enough so that the same node is most active for every
input pattern.

The output patterns corresponding to 43 of the 100
input patterns first case are shown in Fig. 3 (the x axis
represents the ten output nodes, the y axis represents
the analog activity level of a particular output node)
Intelligent Database Systems Lab
Analysis of Multilevel Category Structure
Fig. 3
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Analysis of Multilevel Category Structure

One important characteristic of these activity patterns
is that the feature information is compressed, but still
evident in the shape of the category pattern.

For example, the aquatic animals, essentially the fish
and crustaceans of the second category, all have a
characteristic hump around the nodes seven through
ten.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Analysis of Multilevel Category Structure
Fig. 4
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Analysis of Multilevel Category Structure
Fig. 5
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Analysis of Multilevel Category Structure

N.Y.U.S.T.
I. M.
Fig 6. shows the individual activity patterns generated
when there are four different winning category nodes
associated with the input data set.
Intelligent Database Systems Lab
Analysis of Multilevel Category Structure
Fig. 6
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Analysis of Multilevel Category Structure
Fig. 7
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Analysis of Multilevel Category Structure

There is clearly evidence for a hierarchical category
description moving from general categories to more
specific categories.

The graph in Fig. 7 also shows that it is possible to
combine more specific categories into more general
categories.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
N.Y.U.S.T.
I. M.
Discussion

Traditionally many ART systems have either been modified for
use in specific applications, or have undergone design changes
to build in ever more sophisticated controls that give the ART
network increased ability to do more information processing in
one place.

The category patterns have demonstrated such a variety of
categories using a single network whose parameters do not
need to be tuned or changed during learning and recall in order
to make such category information available.
Intelligent Database Systems Lab
N.Y.U.S.T.
I. M.
Conclusions

We have presented a novel approach to analyzing categories and
multilevel category structure in the output patterns of an ART 2
network.

We have shown how general and specific categories can be
derived from the same set of category data, which is not
possible when defining a category simply by the winning node.

Finally, these findings have been compared favorably with
principles of categorization from different disciplines.
Intelligent Database Systems Lab