Transcript Advanced Diagnostics Algorithms in Online Field Device Monitoring

Advanced Diagnostics Algorithms in
Online Field Device Monitoring
Vagan Terziyan (editor)
http://www.cs.jyu.fi/ai/Metso_Diagnostics.ppt
“Industrial Ontologies” Group: http://www.cs.jyu.fi/ai/OntoGroup/index.html
“Industrial Ontologies” Group, Agora Center, University of Jyväskylä, 2003
Contents
 Introduction: OntoServ.Net – Global “HealthCare” Environment for Industrial Devices;
 Bayesian Metanetworks for Context-Sensitive
Industrial Diagnostics;
 Temporal Industrial Diagnostics with
Uncertainty;
 Dynamic Integration of Classification
Algorithms for Industrial Diagnostics;
 Industrial Diagnostics with Real-Time NeuroFuzzy Systems;
 Conclusion.
Oleksandr
Kononenko
Vagan
Terziyan
Andriy
Zharko
Oleksiy
Khriyenko
Web Services for Smart Devices
Smart industrial devices can be also
Web
Service
“users”.
Their
embedded agents are able to monitor
the state of appropriate device, to
communicate and exchange data
with another agents. There is a good
reason to launch special Web
Services for such smart industrial
devices to provide necessary online
condition monitoring, diagnostics,
maintenance support, etc.
OntoServ.Net: “Semantic Web Enabled Network of Maintenance Services
for Smart Devices”, Industrial Ontologies Group, Tekes Project Proposal,
March 2003,
Global Network of Maintenance Services
OntoServ.Net: “Semantic Web Enabled Network of Maintenance Services
for Smart Devices”, Industrial Ontologies Group, Tekes Project Proposal,
March 2003,
Embedded Maintenance Platforms
Embedded
Platform
Host Agent
Maintenance
Service
Service Agents
Based on the online
diagnostics, a service
agent, selected for the
specific emergency
situation, moves to the
embedded platform to
help the host agent to
manage it and to carry
out the predictive
maintenance activities
OntoServ.Net Challenges
 New group of Web service users – smart industrial
devices.
 Internal (embedded) and external (Web-based) agent
enabled service platforms.
 “Mobile Service Component” concept supposes that any
service component can move, be executed and learn at
any platform from the Service Network, including
service requestor side.
 Semantic Peer-to-Peer concept for service network
management assumes ontology-based decentralized
service network management.
Agents in Semantic Web
1. “I feel bad, pressure
more than 200,
headache, … Who can
advise what to do ? “
3. “Wait a bit, I
will give you
some pills”
Agents in Semantic Web supposed
to understand each other because
they will share common standard,
platform, ontology and language
2. “ I think you
should stop drink
beer for a while “
4. “Never had such
experience. No idea
what to do”
The Challenge: Global Understanding
eNvironment (GUN)
How to make entities from our
physical world to understand
each other when necessary
?..
… Its elementary ! But not easy !!
Just to make agents from them !!!
GUN Concept
2. “I have some
pills for you”
1. “I feel bad,
temperature 40, pain in
stomach, … Who can
advise what to do ? “
Entities will interoperate
through OntoShells, which
are “supplements” of these
entities up to Semantic Web
enabled agents
Semantic Web: Before GUN
Semantic Web Applications
Semantic Web applications
“understand”, (re)use, share,
integrate, etc. Semantic Web
resources
Semantic Web Resources
GUN Concept:
All GUN resources “understand” each other
Real World
objects
Real World Object +
+ OntoAdapter +
+ OntoShell =
= GUN Resource
GUN
OntoShells
OntoAdapters
Real World objects
of new generation
(OntoAdapter inside)
Read Our Recent Reports
 Semantic Web: The Future Starts Today

(collection of research papers and presentations of Industrial Ontologies
Group for the Period November 2002-April 2003)
V. Terziyan
 Semantic Web and Peer-to-Peer:
Integration and Interoperability in Industry
A. Zharko
 Semantic Web Enabled Web Services:
State-of-Art and Challenges
O. Kononenko
 Distributed Mobile Web Services Based on Semantic Web:
Distributed Industrial Product Maintenance System
O. Khriyenko
 Available online in: http://www.cs.jyu.fi/ai/OntoGroup/index.html
Industrial Ontologies Group
Vagan Terziyan
Oleksandra Vitko
Example of Simple Bayesian Network
P(X)
X
P(Y|X)
Y
P(Y)-?
Conditional (in)dependence rule
n
P( X 1 , X 2 ,..., X n )   P( X i | Parents( X i ))
i 1
P(Y  y j , X  xi )  P( X  xi )  P(Y  y j | X  xi )
P(Y  y j )   P( X  xi )  P(Y  y j | X  xi )
Joint probability rule
Marginalization rule
i
P( X  xi | Y  y j ) 
P( X  xi )  P(Y  y j | X  xi )
P(Y  y j )
Bayesian rule
Contextual and Predictive Attributes
air pressure
dust
humidity
temperature
Machine
emission
Environment
Sensors
X
x1
x2
x3
predictive attributes
x4
x5
x6
x7
contextual attributes
Contextual Effect on Conditional
Probability
X
x1
x2
x3
x4
xk
x6
x7
contextual attributes
predictive attributes
Assume conditional
dependence between
predictive attributes
(causal relation between
physical quantities)…
x5
xt
xr
… some contextual
attribute may effect
directly the conditional
dependence between
predictive attributes but
not the attributes itself
Contextual Effect on Conditional
Probability
•X ={x1, x2, …, xn} – predictive attribute with
n values;
•Z ={z1, z2, …, zq} – contextual attribute with q
values;
•P(Y|X) = {p1(Y|X), p2(Y|X), …, p r(Y|X)} –
conditional dependence attribute (random
variable) between X and Y with r possible
values;
•P(P(Y|X)|Z) – conditional dependence
between attribute Z and attribute P(Y|X);
r
n
P(Y  y j )   { pk (Y  y j | X  xi )  P( X  xi ) 
k 1 i 1
q
  [ P( Z  z m )  P( P(Y | X )  pk (Y | X ) | Z  z m )]}
m 1
Contextual Effect on Unconditional
Probability
X
x1
x2
x3
x4
X
xk
x7
xt
P(X)
x1 x2 x3 x4
x6
contextual attributes
predictive attributes
Assume some predictive
attribute is a random
variable with appropriate
probability distribution
for its values…
x5
… some contextual
attribute may effect
directly the probability
distribution of the
predictive attribute
Contextual Effect on Unconditional
Probability

X ={x1, x2, …, xn} – predictive attribute with n
values;
· Z ={z1, z2, …, zq} – contextual attribute with q values
and P(Z) – probability distribution for values of Z;
• P(X) = {p1(X), p2(X), …, pr(X)} – probability
distribution attribute for X (random variable) with r
possible values (different possible probability
distributions for X) and P(P(X)) is probability
distribution for values of attribute P(X);
· P(Y|X) is a conditional probability distribution of Y
given X;
· P(P(X)|Z) is a conditional probability distribution for
attribute P(X) given Z
r
n
P(Y  y j )   {P(Y  y j | X  xi )  pk ( X  xi ) 
k 1 i 1
q
  [ P( Z  z m )  P( P( X )  pk ( X ) | Z  z m )]}
m 1
Bayesian Metanetworks for Advanced Diagnostics
Two-level Bayesian Metanetwork for
managing conditional dependencies
Two-level Bayesian Metanetwork for
managing conditional dependencies
Contextual level
A
X
Q
B
Y
X
A
Predictive level
S
R
Q
B
S
Y
R
2-level Bayesian Metanetwork for
modelling relevant features’ selection
Contextual level
Predictive level
Terziyan V., Vitko O., Probabilistic Metanetworks for Intelligent Data Analysis, Artificial Intelligence,
Donetsk Institute of Artificial Intelligence, Vol. 3, 2002, pp. 188-197.
Terziyan V., Vitko O., Bayesian Metanetwork for Modelling User Preferences in Mobile Environment, In:
German Conference on Artificial Intelligence (KI-2003), Hamburg, Germany, September 15-18, 2003.
Two-level Bayesian Metanetwork for
managing conditional dependencies
Contextual level
P(B|A)
P(Y|X)
A
B
X
Predictive level
Y
Causal Relation between Conditional
Probabilities
xm
xn
P(P(Xn| Xm))
P(Xn| Xm)
P1(Xn|Xm) P2(Xn|Xm) P3(Xn|Xm)
P(P(Xr| Xk))
P(P(Xr| Xk)|P(Xn| Xm))
P(Xr| Xk)
P1(Xr|Xk) P2(Xr|Xk)
xk
There might be causal relationship
between two pairs of conditional
probabilities
xr
Example of Bayesian Metanetwork
The nodes of the
2nd-level network
correspond to the
conditional
probabilities of the
1st-level network
P(B|A) and P(Y|X).
The arc in the 2ndlevel network
corresponds to the
conditional
probability
P(P(Y|X)|P(B|A))
P(Y  y j )  { pk (Y  y j | X  xi )  P( X  xi ) 
i
k
  [ P( P(Y | X )  pk (Y | X ) | P( P( B | A)  pr (Y | X )) P( P( B | A)  pr ( B | A))]}.
r
Other Cases of Bayesian Metanetwork (1)
a)
Contextual level
P(A)
P(X)
Predictive level
X
A
Unconditional probability distributions associated
with nodes of the predictive level network depend
on probability distributions associated with nodes
of the contextual level network
Other Cases of Bayesian Metanetwork (2)
c)
Contextual level
P(A)
P(Y|X)
Predictive level
X
A
Y
The metanetwork on the contextual
level models conditional dependence
particularly between unconditional
and conditional probabilities of the
predictive level
Other Cases of Bayesian Metanetwork (3)
e)
P(A)
P(B)
Contextual level
P(Y|X)
Predictive level
X
A
B
Y
The combination of cases 1 and 2
2-level Relevance Bayesian Metanetwork
(for modelling relevant features’ selection)
Contextual level
Predictive level
Simple Relevance Bayesian Metanetwork
We consider relevance as
a probability of importance
of the variable to the
inference of target attribute
in the given context. In
such definition relevance
inherits all properties of a
probability.
P(Y ) 
Probability to have this model
is:
P((X)=”no”)= 1-X
1
  P(Y | X )  [nx  X  P( X )  (1   X )].
nx X
P(X)
P0(Y)
X
Probability to have this
model is:
P((X)=”yes”)= X
P(Y|X)
Y
Y
Example of 2-level Relevance Bayesian
Metanetwork
In a relevance network
the relevancies are
considered as random
variables between
which the conditional
dependencies can be
learned.
P(Y ) 
1
  {P(Y | X )  [nx  P( X )  P( X |  A )  P( A )  (1  X )]}.
nx X
A
More Complicated Case of
Managing Relevance (1)
1
Probability
Probability
P(X)
P(Z)
X
Z
Probability of this case is equal to:
P((X)=”yes”)×P((Z)=”yes”) =
= X·Z
P(Y|X,Z)
Y
2
3
Probability
P(X)
X
4
Probability
P(Z)
Probability of this case is equal to:
P(Y|X)
Z
Probability of this case is equal to:
P((X)=”no”)×P((Z)=”no”) =
= (1-X)·(1-Z)
P((X)=”no”)×P((Z)=”yes”) =
= (1-X)·Z
Probability of this case is equal to:
P0(Y)
P(Y|Z)
P((X)=”yes”)×P((Z)=”no”) =
= X·(1-Z)
Y
Probability
Y
Y
More Complicated Case of
Managing Relevance (2)
nx
nz
P(Y )   X  Z   P(Y | X  xi , Z  z k )  P( X  xi )  P( Z  z k ) 
i 1 k i
1 nx nz
  X  (1  Z )    P(Y | X  xi , Z  z k )  P( X  xi ) 
nz i 1 k i
1 nx nz
 (1  X )  Z    P(Y | X  xi , Z  z k )  P( Z  z k ) 
nx i 1 k i
nx nz
1
 (1  X )  (1  Z ) 
  P(Y | X  xi , Z  z k ),
nx  nz i 1 k i
General Case of Managing Relevance (1)
Predictive attributes:
X1 with values {x11,x12,…,x1nx1};
X2 with values {x21,x22,…,x2nx2};
…
XN with values {xn1,xn2,…,xnnxn};
Target attribute:
Y with values {y1,y2,…,yny}.
Probabilities:
P(X1), P(X2),…, P(XN);
P(Y|X1,X2,…,XN).
Relevancies:
X1 = P((X1) = “yes”);
X2 = P((X2) = “yes”);
…
XN = P((XN) = “yes”);
Goal: to estimate P(Y).
General Case of Managing Relevance (2)
P(Y ) 
1
N
 nxs
s 1
  ... [ P(Y | X 1, X 2,... XN ) 
X1 X 2
XN
nxr 


r ( ( Xr )" yes ")
Xr
 P( Xr ) 
(1  


q ( ( Xq )"no")
Xq
)]
Example of Relevance Metanetwork
a)
A
X
Q
B
Y
S
Predictive level
b)
A
R
X
c)
Q
B
S
Y
R
Relevance level
Combined Bayesian Metanetwork
Contextual level A
Contextual level B
Predictive level
In a combined Metanetwork two controlling
(contextual) levels will effect the basic level
Learning Bayesian
Metanetworks from Data
 Learning Bayesian Metanetwork structure
(conditional, contextual and relevance
(in)dependencies at each level);
 Learning Bayesian Metanetwork parameters
(conditional and unconditional probabilities
and relevancies at each level).
Vitko O., Multilevel Probabilistic Networks for Modelling Complex
Information Systems under Uncertainty, Ph.D. Thesis, Kharkov National
University of Radioelectronics, June 2003. Supervisor: Terziyan V.
When Bayesian Metanetworks ?
1.
2.
Bayesian Metanetwork can be considered as
very powerful tool in cases where structure
(or strengths) of causal relationships between
observed parameters of an object essentially
depends on context (e.g. external environment
parameters);
Also it can be considered as a useful model
for such an object, which diagnosis depends
on different set of observed parameters
depending on the context.
Vagan Terziyan
Vladimir Ryabov
Temporal Diagnostics of Field Devices
• The approach to temporal diagnostics uses the
algebra of uncertain temporal relations*.
• Uncertain temporal relations are formalized
using probabilistic representation.
• Relational networks are composed of uncertain
relations between some events (set of symptoms)
• A number of relational networks can be
combined into a temporal scenario describing
some particular course of events (diagnosis).
• In future, a newly composed relational network
can be compared with existing temporal
scenarios, and the probabilities of belonging to
each particular scenario are derived.
* Ryabov V., Puuronen S., Terziyan V., Representation and Reasoning with
Uncertain Temporal Relations, In: A. Kumar and I. Russel (Eds.), Proceedings of
the Twelfth International Florida AI Research Society Conference - FLAIRS-99,
AAAI Press, California, 1999, pp. 449-453.
Conceptual Schema for Temporal Diagnostics
Generating temporal scenarios
Recognition of temporal scenarios
N
N2
N1
N3
D N ,S n
DN ,S1
D N ,S 2
S
S1
N5
S2
… Sn
N4
• We compose a temporal scenario
combining a number of relational
networks consisting of the same set of
symptoms and possibly different
temporal relations between them.
Temporal scenarios
• We estimate the probability of
belonging of the particular relational
network to known temporal scenarios.
Terziyan V., Ryabov V., Abstract Diagnostics Based on Uncertain Temporal
Scenarios, International Conference on Computational Intelligence for Modelling
Control and Automation CIMCA’2003, Vienna, Austria, 12-14 February 2003, 6 pp.
Industrial Temporal Diagnostics
(conceptual schema)
Temporal
data
Estimation
Recognition
Diagnosis
Relational
network
Industrial object
DB of
scenarios
Learning
Ryabov V., Terziyan V., Industrial Diagnostics Using Algebra of Uncertain
Temporal Relations, IASTED International Conference on Artificial Intelligence
and Applications, Innsbruck, Austria, 10-13 February 2003, 6 pp.
Imperfect Relation Between Temporal
Point Events: Definition
< a1; a2; a3 > - imperfect temporal relation
Event 1
between temporal points (Event 1 and Event 2):
 P(event 1, before, event 2) = a1;
< a1 ; a 2 ; a 3 >
 P(event 1, same time, event 2) = a2;
 P(event 1, after, event 2) = a3.
Event 2
Ryabov V., Handling Imperfect Temporal Relations, Ph.D. Thesis, University of
Jyvaskyla, December 2002. Supervisors: Puuronen S., Terziyan V.
Example of Imperfect Relation
 < 0.5; 0.2; 0.3 > - imperfect
temporal relation between
temporal points:
Event 1
 P(event 1, before, event 2) = 0.5;
 P(event 1, same time, event 2) = 0.2;
< 0.5; 0.2; 0.3 >
 P(event 1, after, event 2) = 0.3.
1
Event 2
<
=
>
R(Event 1,Event 2)
Operations for Reasoning with
Temporal Relations
Composition
a
ra ,b
rb,a = ~
a
b
ra,b
rb,c
c
r a,c = r a,b  r b,c
b
ra,b
r2a,b
r1a,b
Inversion
a
ra ,b  r1a ,b  r 2 a ,b
Sum
b
Temporal Interval Relations
 The basic interval relations are the thirteen
Allen’s relations:
A
A
A
B
A before (b) B
B after (bi) A
B
A meets (m) B
B met-by (mi) A
B
A overlaps (o) B
B overlapped-by (oi) A
B
A starts (s) B
B started-by (si) A
A
B
A during (d) B
B contains (di) A
A
B
A finishes (f) B
B finished-by (fi) A
A
B
A equals (eq) B
B equals A
A
Imperfect Relation Between
Temporal Intervals: Definition
< a1; a2;… ; a13 > - imperfect temporal
interval 1
< a1; a2 ;… ; a13 >
interval 2
relation between temporal intervals (interval 1
and interval 2):
 P(interval 1, before, interval 2) =
a1;
 P(interval , meets, interval 2) =
a2;
 P(interval 1, overlaps, interval 2) = a3;
…
 P(interval 1, equals, interval 2) =
a13;
Industrial Temporal Diagnostics
(composing a network of relations)
Sensor 1
Estimation of
temporal
relations between
symptoms
Sensor 2
Industrial object
Sensor 3
Relational network
representing the
particular case
Industrial Temporal Diagnostics
(generating temporal scenarios)
Object B
Object A
N1
Object C
N2
1. for i=1 to n do
2. for j=i+1 to n do
3.
if (R1) or…or (Rk) then
4.
begin
5.
for g=1 to n do
6.
if not (Rg) then Reasoning(, Rg)
7.
// if “Reasoning” = False then (Rg)=TUR
8.
( R) = Å ( Rt), where t=1,..k
9.
end
10.
else go to line 2
N3
Scenario S
Generating the
temporal
scenario
for “Failure X”
DB of
scenarios
Recognition of Temporal Scenario
Temporal
data
Estimation
Recognition
Diagnosis
Relational
network
Industrial object
DB of
scenarios
Learning
m
D N,S 
w d
i 1
m
i
w
i 1
i
Probability
value
i
b
o
m
fi
di
si
e
q
d
s
f
oi
m
i
bi
d R A,B ,RC ,D  Bal( R A,B )  Bal( R C,D )
12
1

Bal(RA,B) =  i  e Ai,B1
12 i  0
wb
=0
weq
=0.5
Balance point for
RA,B
wf
=0.75
Balance point for
RC,D
wbi
=1
When Temporal Diagnostics ?
1.
2.
3.
Temporal diagnostics considers not only a static
set of symptoms, but also the time during which
they were monitored. This often allows having a
broader view on the situation, and sometimes
only considering temporal relations between
different symptoms can give us a hint to precise
diagnostics;
This approach might be useful for example in
cases when appropriate causal relationships
between events (symptoms) are not yet known
and the only available for study are temporal
relationships;
Combination of Bayesian (based on probabilistic
causal knowledge) and Temporal Diagnostics
would be quite powerful diagnostic tool.
Vagan
Terziyan
Terziyan V., Dynamic Integration of Virtual Predictors, In: L.I. Kuncheva, F. Steimann,
C. Haefke, M. Aladjem, V. Novak (Eds), Proceedings of the International ICSC Congress
on Computational Intelligence: Methods and Applications - CIMA'2001, Bangor, Wales,
UK, June 19 - 22, 2001, ICSC Academic Press, Canada/The Netherlands, pp. 463-469.
The Problem
During the past several years, in a variety of
application domains, researchers in machine
learning, computational learning theory, pattern
recognition and statistics have tried to combine
efforts to learn how to create and combine an
ensemble of classifiers.
The primary goal of combining several classifiers is to
obtain a more accurate prediction than can be
obtained from any single classifier alone.
Approaches to Integrate Multiple
Classifiers
Integrating Multiple Classifiers
Combination
Selection
Decontextualization
Global
(Static)
Local
(Dynamic)
Local
Global
(“Virtual”
(Voting-Type) Classifier)
Inductive learning with
integration of predictors
 xt1, xt 2 ,..., xtm 
Sample Instances
Learning Environment
Predictors/Classifiers
 xr1, xr 2 ,..., xrm  yr 
P1
P2
...
yt
Pn
Virtual Classifier
Virtual Classifier is a group of seven cooperative agents:
Constant Team Members ElectiveTeam Members

 
 TC,
TM,
TP,
TI
,
FS,
DE,
CL

 

Team Instructors
Classification Team
TC - Team Collector
FS - Feature Selector
TM - Training Manager
DE - Distance Evaluator
TP - Team Predictor
CL - Classification Processor
TI - Team Integrator
Classification Team:
Feature Selector
Constant Team Members ElectiveTeam Members

 
 TC,
TM,
TP,
TI
,
FS
,
DE,
CL

 

Team Instructors
ClassificationTeam
FS - Feature Selector
Feature Selector:
finds the minimally sized feature subset that is sufficient for
correct classification of the instance
Sample Instances
Sample Instances
 Χr  yr 
 Χ'r  yr , Χ' r  Χr
Classification Team:
Distance Evaluator
Constant Team Members ElectiveTeam Members

 
 TC,
TM,
TP,
TI
,
FS,
DE
,
CL

 

Team Instructors
DE - Distance Evaluator
ClassificationTeam
Distance between Two Instances with
Heterogeneous Attributes (example)
i  d ( xi , yi )
D( X , Y ) 
2
i , xiX , yiY
where:

0, if xi  yi
if i  th attribute is nominal - 

1, otherwise
d ( xi , yi )  
else : xi  yi

rangei

d (“red”, “yellow”) = 1
d (15°, 25°) = 10°/((+50°)-(-50°)) = 0.1
Distance Evaluator:
measures distance between instances based on
their numerical or nominal attribute values
 xi1, xi 2 ,..., xim 
 x j1, x j 2 ,..., x jm 
Distance Evaluator
dij
Classification Team:
Classification Processor
Constant Team Members ElectiveTeam Members

 
 TC,
TM,
TP,
TI
,
FS,
DE,
CL

 

Team Instructors
ClassificationTeam
CL - Classification Processor
Classification Processor:
predicts class for a new instance based on its selected
features and its location relatively to sample instances
 xi1, xi 2 ,..., xim 
Sample Instances
Feature
Selector
Classification
Processor
Distance
Evaluator
yi
Team Instructors:
Team Collector
Constant Team Members ElectiveTeam Members

 
 TC,
TM,
TP,
TI
,
FS,
DE,
CL


 
Team Instructors
ClassificationTeam
TC - Team Collector completes
Classification Teams for training
Team Collector
completes classification teams for future training
Distance Evaluation
functions
Classification
rules
Feature Selection
methods
Team Collector
FSi
DEj
CLk
Team Instructors:
Training Manager
Constant Team Members ElectiveTeam Members

 
 TC,
TM
,
TP,
TI
,
FS,
DE,
CL

 

Team Instructors
ClassificationTeam
TM - Training Manager trains all
completed teams on sample instances
Training Manager
trains all completed teams on sample instances
Training Manager
Sample Instances
 xr1, xr 2 ,..., xrm  yr 
FSi1
DEj1
CLk1
FSi2
DEj2
CLk2
FSin
DEjn
Sample Metadata
 xr1, xr 2 ,..., xrm  wr1, wr 2 ,..., wrn 
CLkn
Classification Teams
Team Instructors:
Team Predictor
Constant Team Members ElectiveTeam Members

 
 TC,
TM,
TP
,
TI
,
FS,
DE,
CL

 

Team Instructors
ClassificationTeam
TP - Team Predictor predicts weights for
every classification team in certain location
Team Predictor
predicts weights for every classification
team in certain location
Predicted weights
of classification teams
Location
 xi1, xi 2 ,..., xim 
Team Predictor:
e.g. WNN algorithm
Sample Metadata
 xr1, xr 2 ,..., xrm  wr1, wr 2 ,..., wrn 
 wi1, wi 2 ,..., win 
Team Prediction:
Locality assumption
Each team has certain subdomains in the space
of instance attributes, where it is more reliable
than the others;
This assumption is supported by the experiences,
that classifiers usually work well not only in certain
points of the domain space, but in certain
subareas of the domain space [Quinlan, 1993];
If a team does not work well with the instances
near a new instance, then it is quite probable that
it will not work well with this new instance also.
Team Instructors:
Team Integrator
Constant Team Members ElectiveTeam Members

 
 TC,
TM,
TP
,
TI
,
FS,
DE,
CL

 

Team Instructors
ClassificationTeam
TI - Team Integrator produces classification
result for a new instance by integrating
appropriate outcomes of learned teams
Team integrator
produces classification result for a new instance by
integrating appropriate outcomes of learned teams
Weights of classification teams
in the location of a new instance
 xt1, xt 2 ,..., xtm 
 wt1, wt 2 ,..., wtn 
FSi1
DEj1
CLk1
yt1
FSi2
DEj2
CLk2
yt2
FSin
DEjn
CLkn
yt1
Classification teams
Team Integrator
New instance
yt
Static Selection of a Classifier
 Static selection means that we try all teams
on a sample set and for further
classification select one, which achieved
the best classification accuracy among
others for the whole sample set. Thus we
select a team only once and then use it to
classify all new domain instances.
Dynamic Selection of a Classifier
 Dynamic selection means that the team is being
selected for every new instance separately
depending on where this instance is located. If it
has been predicted that certain team can better
classify this new instance than other teams, then
this team is used to classify this new instance. In
such case we say that the new instance belongs
to the “competence area” of that classification
team.
Conclusion
 Knowledge discovery with an ensemble of classifiers is known
to be more accurate than with any classifier alone [e.g.
Dietterich, 1997].
 If a classifier somehow consists of certain feature selection
algorithm, distance evaluation function and classification rule,
then why not to consider these parts also as ensembles making a
classifier itself more flexible?
 We expect that classification teams completed from different
feature selection, distance evaluation, and classification methods
will be more accurate than any ensemble of known classifiers
alone, and we focus our research and implementation on this
assumption.
Yevgeniy Bodyanskiy
Volodymyr Kushnaryov
Online Stochastic Faults’ Prediction
Control Systems Research Laboratory,
AI Department, Kharkov National University of
Radioelectronics. Head: Prof. E. Bodyanskiy. Carries
out research on development of mathematical and
algorithmic support of systems for control, diagnostics,
forecasting and emulation:
1. Neural network architectures and real-time
algorithms for observation and sensor data processing
(smoothing, filtering, prediction) under substantial
uncertainty conditions;
2. Neural networks in polyharmonic sequence
analysis with unknown non-stationary parameters;
Bodyanskiy Y., Vorobyov S, Recurrent Neural Network
Detecting Changes in the Properties of Non-Linear
Stochastic Sequences, Automation and Remote Control, V.
1, No. 7, 2000, pp. 1113-1124.
Bodyanskiy Y., Vorobyov S., Cichocki A., Adaptive Noise
Cancellation for Multi-Sensory Signals, Fluctuation and
Noise Letters, V. 1, No. 1, 2001, pp. 12-23.
Bodyanskiy Y., Kolodyazhniy V., Stephan A. An Adaptive
Learning Algorithm for a Neuro-Fuzzy Network, In: B.
Reusch (ed.), Computational Intelligence. Theory and
Applications, Berlin-Heidelberg-New York: Springer, 2001,
pp. 68-75.
3. Analysis of chaotic time series; adaptive algorithms
and neural network architectures for early fault
detection and diagnostics of stochastic processes;
4. Adaptive multivariable predictive control
algorithms for stochastic systems under various types
of constraints;
5. Adaptive neuro-fuzzy control of non-stationary
nonlinear systems;
6. Adaptive forecasting of non-stationary nonlinear
time series by means of neuro-fuzzy networks;
7. Fast real-time adaptive learning procedures for
various types of neural and neuro-fuzzy networks.
Existing Tools
Most existing (neuro-) fuzzy systems used for fault
diagnosis or classification are based on offline learning
with the use of genetic algorithms or modifications of
the error back propagation. When the number of features
and possible fault situations is large, tuning of the
classifying system becomes very time consuming.
Moreover, such systems perform very poorly in high
dimensions of the input space, so special modifications
of the known architectures are required.
Neuro-Fuzzy Fault Diagnostics
Successful application of the neuro-fuzzy synergism to
fault diagnosis of complex systems demands development
of an online diagnosing system that quickly learns from
examples even with a large amount of data, and maintains
high processing speed and high classification accuracy
when the number of features is large as well.
Challenge: Growing (Learning)
Probabilistic Neuro-Fuzzy Network (1)
input layer,
n inputs
1-st hidden layer,
N neurons
2-nd hidden layer,
(m+1) elements
output layer,
m divisors
Bodyanskiy Ye., Gorshkov Ye., Kolodyazhniy V., Wernstedt J., Probabilistic Neuro-Fuzzy Network with
Non-Conventional Activation Functions, In: Knowledge-Based Intelligent Information & Engineering
Systems, Proceedings of Seventh International Conference KES’2003, 3–5 September, Oxford, United
Kingdom, LNAI, Springer-Verlag, 2003.
Bodyanskiy Ye., Gorshkov Ye., Kolodyazhniy V. Resource-Allocating Probabilistic Neuro-Fuzzy Network,
In: Proceedings of International Conference on Fuzzy Logic and Technology, 10–12 September, Zittau,
Germany, 2003.
Challenge: Growing (Learning)
Probabilistic Neuro-Fuzzy Network (2)
Tested on real data in comparison with
classical probabilistic neural network
Unique
combination
of features
 Implements fuzzy reasoning and classification (fuzzy classification
network);
 Creates automatically neurons based on training set (growing
network);
 Learns free parameters of the network based on training set
(learning network);
 Guarantees high precision of classification based on fast learning
(high- performance network);
 Able to perform with huge volumes of data with limited
computational resources (powerful and economical network);
 Able to work in real-time (real-time network).
Tests for Neuro-Fuzzy Algorithms
Industrial Ontologies Group (Kharkov’s Branch), Data
Mining Research Group and Control Systems Research
Laboratory of the Artificial Intelligence Department of
Kharkov National University of Radioelectronics have
essential theoretical and practical experience in implementing
neuro-fuzzy
approach
and
specifically
Real-Time
Probabilistic Neuro-Fuzzy Systems for Simulation,
Modeling, Forecasting, Diagnostics, Clustering, Control .
We are interested in cooperation with Metso in that area
and we are ready to present the performance of our
algorithms on real data taken from any of Metso’s products
to compare our algorithms with existing in Metso
algorithms.
Inventions we can offer (1)
 Method of intelligent preventive or predictive
diagnostics and forecasting of technical condition of
industrial equipment, machines, devices, systems, etc.
in real time based on analysis of non-stationary
stochastic signals (e.g. from sensors of temperature,
pressure, current, shifting, frequency, energy
consumption, and other parameters with threshold
values).
 The method is based on advanced data mining
techniques, which utilize fuzzy-neuro technologies, and
differs from existing tools by flexible self-organizing
network structure and by optimization of computational
resources while learning.
Inventions we can offer (2)
 Method of intelligent real-time preventive or predictive
diagnostics and forecasting of technical condition of
industrial equipment, machines, devices, systems, etc.
based on analysis of signals with non-stationary and
non-multiplied periodical components (e.g. from
sensors of vibration, noise, frequencies of rotation,
current, voltage, etc.).
 The method is based on optimization of computational
resources while learning because of intelligent reducing
of the number of signal components being analyzed.
Inventions we can offer (3)
 Method and mechanism of optimal control of
dosage and real-time infusion of anti-wear oil
additives into industrial machines based on its
real-time condition monitoring.
Summary of problems we can solve
 Rather global system for condition monitoring and
preventive maintenance based on OntoServ.Net
(global, agent-based, ontology-based, Semantic Web
services-based,
semantic
P2P search-based)
technologies, modern and advanced data-mining
methods and tools with knowledge creation,
warehousing, and updating during not only device’s
lifetime, but also utilizing (for various maintenance
needs) knowledge obtained afterwards (various
testing and investigations techniques other than
information taken from “living” device’s sensors)
from broken-down, worn out or aged components of
the same type.
Recently Performed Case Studies (1)
Semen
Simkin
 Ontology Development for Gas Compressing Equipment
Diagnostics Realized by Neural Networks
 Available in: http://www.cs.jyu.fi/ai/OntoGroup/docs/July2003.pdf
NN and Ontology using for Diagnostic
Neural Network
SENSOR
SIGNAL
Diagnostic out
Training
The creating ontology classes
instance program

Diagnosing
12
The subclasses and their slots forming and
instances filling by the information is
carried out automatically with the program
on Java. The filling occurs from RDBMS
Oracle, which contains in the actualized
base using in ”UkrTransGas”.
Oracle
Java
Program
Ontology
15
Volodymyr
Kushnaryov
Recently Performed Case Studies (2)
Konstantin
Tatarnikov
 The use of Ontologies for Faults and State Description of GasTransfer Units
 Available in: http://www.cs.jyu.fi/ai/OntoGroup/docs/July2003.pdf
GTU
Repair-Reason
Trend
Analog
Signal
Launch
Signal-Types
PARAMETER
Compute
Variable
ACTIONS
GTUMAINTENANCE
Shutdown
Control-type
Oil-temperature
deviation
Period
Subsystem
Mid-life Repair
GTU
GTU
GTU
SITUATIONS
Axle-shear
REPAIR
GTU
Support-History
GTU-State
Major Repair
Vibration
GTU-Node
Rise-oftemperature
Current Repair
Planned Repair
Compressor
station
SCADA
SCADA
SCADA
SCADA
Volodymyr
Kushnaryov
Agent
Diagnosist
Diagnosist
Ontology
for agent communication
Agent
Conclusion
 Industrial Ontologies Research Group (University
of Jyvaskyla), which is piloting the OntoServ.Net
concept of the Global Semantic Web - Based System
for Industrial Maintenance, has also powerful
branches in Kharkov (e.g. IOG-Kharkov’s Branch,
Control Systems Research Laboratory, Data Mining
Research Group, etc.) with experts and experiences
in various and challenging data mining and
knowledge discovery, online diagnostics, forecasting
and control, models learning and integration, etc.
methods, which can be and reasonable to be
successfully utilized within going-on cooperation
between Metso and Industrial Ontologies Group.