Transcript metric analysis results

THE INTELLIGENCE
SYSTEM OF SOFTWARE
COMPLEXITY AND
QUALITY EVALUATION
AND PREDICTION
Oksana Pomorova,
Tetyana Hovorushchenko
Khmelnitsky National University
Safety Case Methodology
The main task of Safety Case methodology is the
automating of the creation of:
• software
requirements
profile
(including
standards for software development, subject domain
standards and customer requirements);
• software analysis results profile - metric analysis
results, source code and software test results;
• evaluation of results profile accordance to
requirements profile.
Our task - automating of the metric analysis
results processing.
Unsolved Tasks of Metric Analysis
Results Processing:
• absence of unified standards for metrics, which leads to
subjective selection of quality evaluation methods;
• difficulty of interpretation the metrics values, which is
caused by individual projects features and absence of
metrics standard values;
• absence of criterion to compared essentially new and
previous projects, which leads to the impossibility of
interpretation of obtained metrics for new project;
• basic parameters in the selection of software realization
versions are the design cost and time and software company
reputation, but the decisions, taken on the basis of these
parameters, not guarantee software quality.
On the basis of the above the need
and actuality of scientific research in
development of new effective
methods of software quality
evaluation and prediction arises.
The intelligent methods, in particular
artificial neural net's method of
software quality evaluation and
prediction, are perspective today.
Metrics of Software Design Stage
The Structure of Intelligence System
of Software Complexity and Quality
Evaluation and Prediction (ISCQEP)
ISCQEP Structure
ISCQEP Components
• The dialog (interface) module visualizes the functioning of
module of data collection and communication, displays the
system functioning and produces the messages to user in an
understandable form for him.
• The module of data collection and communication reads the
user information about the quantitative values of exact and
predicted metrics of software design stage, saves the obtained
information in the knowledge base and transmits its to the
module of ANN input vectors forming.
• Knowledge base contains the quantitative values of exact and
predicted metrics of software design stage, the ANN input
vectors and the rules of ANN results processing.
• The module of ANN input vectors forming prepares the
metrics values of the knowledge base for the ANN inputs.
• The artificial neural network provides the approximation of
software design stage metrics and gives the quantitative
evaluation of project complexity and quality and prediction of
designed software complexity and quality characteristics.
Input data for ANN are the set of the design stage metrics with
the exact values and the set of the design stage metrics with
the predicted values. If a certain metric was not determined,
the proper element of set will be equal -1. Multilayer
perceptron is ANN for solving of task of the metrics analysis
and software quality characteristics prediction. This ANN has
24 neurons of the input layer, 14 neurons of approximating
layer and 8 neurons of the adjusting layer and 4 neurons of the
output layer. Realized neural network was trained with
training sample of 1935 vectors and tested with testing sample
of 324 vectors by one step secant backpropagation method
(OSS). The training performance is 0,102197.
ANN architecture in
Simulink
Structural scheme of
ANN layers
Structural scheme of
ANN 1-st layer
Structural scheme of
ANN 3-rd layer
Structural scheme of
ANN 2-nd layer
Structural scheme of
ANN 4-th layer
ANN evaluations:
–
–
–
–
project complexity estimate;
project quality evaluation;
software complexity prediction;
software quality prediction
are values in the range [0, 1], where 0 - proper metrics
were not determined, approximately 0 - the project or
designed software has a high complexity or low quality and
1 - the project or software is simple or high quality.
• The module of ANN results processing makes the
conclusions about the project quality and complexity and the
expected quality and complexity of designed software on the
basis of an analysis of 4-th obtained results.
Processing of Stage Design Metrics Using ISCQEP
The project has low complexity and
high quality
The designed software will has low
complexity and high quality
The project has significant complexity
and low quality
The designed software will has
significant complexity and low quality
The project has medium complexity
and medium quality
The designed software will has medium
complexity and quality
The project has low complexity and
high quality
The designed software will has low
complexity and high quality
The project has significant complexity
and low quality
The designed software will has
significant complexity and low quality
On the basis of ANN results, design cost and time the
choice of project version was performed.
Both versions have approximately the same design
cost and time, but significantly different estimates of
project complexity and quality and prediction of
designed software complexity and quality. On the basis
of only cost and time software company can make a
false conclusion about selection of the project version.
ANN evaluations help to make the right selection.
ACKNOWLEDGMENT
The necessity and actuality of scientific research in
software quality evaluation and prediction comes
from the results of the software metric evaluation
methods analysis.
The main parameters in the selection of software
project version are the design cost and time and
designing company reputation, but a decisions on the
basis of these parameters are not always guarantee
the proper software quality.
Predicted evaluations of designed software
complexity and quality give the prediction about
complexity and quality of concrete project version
realization and allow to compare the different project
versions, when the cost and time is approximately
equal.
The proposed intelligence system of software
complexity and quality evaluation and prediction
provides the motivated and grounded decision about
selection of the project version on the basis not only
cost and time, but also considering quality
characteristics.
Problems:
• metric information lack to increasing of the
training and testing samples size;
• need the such diverse utilities to comparing of
metric information processing results of this project;
• need the development of designed software
complexity evaluation metrics from the viewpoint of
the maintenance difficulty or simplicity, usability
and the effectiveness of the methods chosen to solve
the task.
References
• Bishop P. A Methodology for Safety Case Development / P. Bishop. - 1998
• Kelly T. Arguing Safety – A Systematic Approach to Managing Safety Cases / T.
Kelly. - 1998
• A. Gordeyev, V. Kharchenko, A. Andrashov, B. Konorev, V. Sklyar,
A.
Boyarchuk. Case-Based Software Reliability Assessment by Fault Injection
Unified Procedures // Proceedings of the 2008 International Workshop on
Software Engineering in East and South Europe – Germany, Leipzig, 2008. – pp.
1-8
• Pomorova O.V., Hovorushchenko T.O. Analysis of Methods and Tools of
Software Systems Quality Evaluation // Radioelectronic and Computer Systems –
Kharkiv: KhAI, 2009 – N6, pp.148-158
• Pomorova O.V., Hovorushchenko T.O. Intelligence Method of Design Results
Evaluation and Software Quality Characteristics Prediction // Radioelectronic and
Computer Systems – Kharkiv: KhAI, 2010 – N6, pp.211-218
• Pomorova O.V., Hovorushchenko T.O., Tarasek S.Y. Analysis and Processing of
Software Quality Metrics on the Design Stage // Transactions of Khmelnitsky
National University – Khmelnitsky: KhNU, 2010. - N1, pp.54-63
Our Contacts
29016, Ukraine, Khmelnitsky, Institutska str., 11
Khmelnitsky National University
Department of system programming
Oksana Pomorova
Doctor of Technical Sciences, Professor,
Head of System Programming Department
[email protected]
Tetyana Hovorushchenko
Ph.D., Senior Researcher, Associate Professor,
Lecturer of System Programming Department
[email protected]