Transcript Quantification of the non parametric continuous BBNs with

Quantification of the nonparametric
continuous BBNs with expert
judgment
Iwona Jagielska
Msc. Applied Mathematics
Outline of the presentation
I PART
1. Introduction
II PART
2. Method of eliciting conditional rank correlations
3. Comparison of algorithms to calculate multivariate normal
probabilities
4. Presentation of elicitation software UniExp
III PART
5. Building the Maintenance Performance Model
Model variables
Dependence relation
6. Results
7. Conclusions and recommendations
1. Introduction
CATS – casual model for Air Transport Safety
– motivation and purpose
- three sectors of human performance
ATC Model,
Flight Crew Performance Model
Maintenance Performance Model
2. Way of assessing dependence relations
• Conditional Rank correlations  Conditional probabilities of exceedence
“Suppose that the variable X3 was observed above its qth quantile.
What is the probability that also X4 will be observed above its qth
quantile? “
P1= P ( FX4(X4) > q | FX3(X3) > q )
• Why normal copula?
C(u1 ,...,uN )   N (1 (u1 ),...,1 (uN ))
- Advantages
• known relation between partial and rank correlation
• equal conditional and partial correlations
• possess zero independent property
- Disadvantages
• no analytical form for multivariate cumulative distribution function
2. Way of assessing dependence relations
• we can calculate relationship between rank correlation and conditional probability
4,3  P1 ( 4,3 )
4,3  r4,3
r4,3  P1 (r4,3 )
To see the conditional probability as a function of rank correlation we integrate bivariate
1
2
normal density over the given region [ (q), ] .
P1e1  0.7

e1
r4,3
 0.57
2. Way of assessing dependence relations
“ Suppose that not only variable X3 but also X2 was observed
above their qth quantile. What is the probability that also X4 will be
observed above its qth quantile? ”
P2= P ( FX4(X4) > q | FX3(X3) > q, FX2(X2) > q )
P2 ( 4,2 )  4,2  4,2|3  r4,2|3
r4,2|3  P2 (r4,2|3 )
P2e1  0.8 
To find the conditional probability we
integrate trivariate normal density over the
1
3
given region [ (q), ] with covariance
matrix
.
 1

  4,3
  4,2
 4,3
1
3,2
e1
r4,2|3
 0.37
We assess the higher order conditional rank correlation in the similar way.
4,2 

3,2 
1 
3.1. Algorithms to calculate multivariate normal
probabilities
F ( a, b) 
b1

a1
b2
bn
 ...
a2
an
1
 ( 2 )
n
e  x '
1
x
dx
Proposed numerical integration methods:
• Algorithm I and II – by Genz
- first we apply transformation to simplify integration region
- later randomized quasi Monte Carlo method is used
- different choice of quasi points
- in algorithm I we specify number of points; algorithm II assign number of points, s.t. the
requested accuracy is provided
• Algorithm III and IV
- based on successive subdivisions of integration region, where each subdivision is
used to provide a better approximation of the integrand
- polynomial rule is used to approximate integrand on each subregions
- error estimate – difference between two polynomial rules of different order
- algorithm IV may involve some simplification routines (change of
variables)
3.2. Numerical Comparison
TAi – time of calculation for algorithm i
PAi – probability obtained by algorithm i
EAi – estimated error of approximation provided by algorithm i
700, 1500 – number of quasi random point in Alg I; 10-5 requested accuracy for Alg II
dimension = 4, determinant = 0.5271
dimension = 7, determinant = 0.489
3. Numerical Comparison – brief summary
• Algorithms III and IV are unpractical for large scale applications since they require long
time for numerical calculations
- time for hypercube [0.5, inf]7 is more than 700seconds
- when the procedure of subdivision of integration region is applied, algorithm do not
provide the total error
• In Algorithm I user needs to specify number of quasi random points used to calculation;
there is no control of provided error of estimation; time of calculation depends on the
number of points, not of covariance matrix
• Time of calculation for Algorithm II is sometimes grater than for Algorithm I; time depends
on covariance matrix; number of quasi random points depends on requested accuracy of
solution
At this moment Algorithm II is used in the software UniExp as the most accurate one; Algorithm I
also has future potencial for implementation.
4. Software elicitation tool - UniExp
1 Step – input of nodes and connections
4. Software elicitation tool - UniExp
2 Step – elicitation of conditional rank correlations
4. Software elicitation tool - UniExp
Values of Rank Correlations can be found in
RankCorrelationValues.txt file
5. Maintenance Performance Model
5. Maintenance Performance Model – dependence relation
Elicitation with single expert
we asked –
4 questions about marginal distributions – classical method of
expert judgment
7 questions about conditional probabilities of exceedance
All variables are negatively correlated with variable human error
5. Maintenance Performance Model
At the bottom of each histogram the expectation and standard derivation are shown.
Unconditional expected value of human error is 0.266/10000
6. Maintenance Performance Model - conditioning
Number of years of experience = 3
expected value of human error increases 0.266/10000 -> 0.309/10000
6. Maintenance Performance Model - conditioning
Requiring at least 6 hours of sleep provides decrease of expected human error
from 0.266/10000 to 0.152/10000
Moreover E(HE|WorkCond=1,Alert=6) = 0.248/10000 while
E(HE|WorkCond=1)=0.398/10000
7. Conclusions and recommendations
•
Calculation of multivariate normal probabilities is not an easy task in case of
high dimension; there is still need to develop more fast (and also accurate)
algorithm for higher dimension
•
Include Algorithm I in UniExp software; together with making UniExp to worked
outside the Matlab environment
•
Combining experts opinion to obtain better results
•
Collect data describing to marginal distribution in Maintenance Performance
Model
•
Discover other possible influential factors in Maintenance Performance Model
Any other propositions?
Questions ???
Additional
Slides
A1. Covariance matrixes used in numerical tests
Dimension 4
determinant = 0.5271
determinant = 0.1099
A1. Covariance matrixes used in numerical tests
Dimension 7
determinant = 0.4890
determinant = 0.1102
A2. Determinant of covariance matrix as the measure
of spread from distribution
Dimension 2
determinant = 1
A2. Determinant of covariance matrix as the measure
of spread of distribution
Dimension 2
determinant = 0.51, =0.714
A2. Determinant of covariance matrix as the measure
of spread of distribution
Dimension 2
determinant = 0.0199, =0.141
3.2. Numerical Comparison
TAi – time of calculation for algorithm i
PAi – probability obtained by algorithm i
EAi – estimated error of approximation provided by algorithm i
700, 1500 – number of quasi random point in Alg I; 10-5 requested accuracy for Alg II
dimension = 4
dimension = 7
Test 1 – identity covariance matrix
3. Numerical Comparison
Test 2 – covariance matrix with determinant 0.5
dimension = 4, determinant = 0.5271
dimension = 7, determinant = 0.489
3. Numerical Comparison
Test 3 – covariance matrix with determinant 0.1
dimension = 4, determinant = 0.1099
dimension = 7, determinant = 0.1102
5. Maintenance Performance Model
-Motivation –
- part of CATS model
- build to describe the causal factors influencing the maintenance
crew
Methodology
- non-parametric BBN
Quantification
- Nodes – variables which can influence the human performance among the
maintenance crew; marginal distribution – data or Classical Method (Expert
Judgment)
- Conditional rank correlations – obtained from experts through the
dependence probabilities of exceedance
5. Maintenance Performance Model – model variables
Variable
Definition
Source of marginal
distribution
1. Job Trainings
average number of training per year
Expert judgment
2. Alertness
average number of hours an aircraft mechanic
sleeps of per day
Data
3. Communication
current information transfer procedure in use,
distinguishing: 1. only paper notes, 2. paper
notes with oral feedback
Expert judgment
4. Experience
average number of years a person worked as
aircraft mechanic
Data
6. Aircraft Generation
aircraft generation in scale from 1 to 4 where 4
is the most recent generation
Data
5. Working Conditions
average number of maintenance operations
needed to be performed 1.out-side /
2. inside the hangar per 10,000
maintenance operations
Expert Judgment
7. Human Error
number of maintenance human errors that
might lead to hazardous situations per 10,000
maintenance tasks
Expert Judgment