Transcript EEL6883_SEII_Spr07_ProjectTeam2

The Organizational Impacts on Software
Quality and Defect Estimation
Stephen Lopez-Couto
Discussion Topics


Introduction
Defect Concepts



Defect Estimation Methods






Definition of a defect
Static vs. Dynamic Discovery
Linear Regression
Capture/Recapture
AI
Defect Estimation Variables
Further Research
Conclusion
Introduction

Defect Estimation


The act of “guessing” the number of
defects that exist in a current software
baseline
Purpose of this paper is to determine
the usefulness of metrics that are not
explicitly tied to the code to make the
estimates
Introduction

Organizational Elements:
Development Team Size
 Developer Experience
 Institutional Processes

 CMMI
level for example
Development Tools
 Development Schedule
 Programming Language
 Software Architecture

Defect Concepts: What is a
Defect?
There is no common definition of a
defect
 Consider the following:


Program is supposed to add the
variables ‘y’ and ‘i’ together and put the
value into ‘x’
int i = 5;
int y = 6;
int x = y+1
Print(x);
Defect Concepts: What is a
Defect?
There is no common definition of a
defect
The programmer introduced a defect
 Consider the
following:
Correct
code: int x = y + i


Program is supposed to add the
variables ‘y’ and ‘i’ together and put the
value into ‘x’
int i = 5;
int y = 6;
int x = y+1
Print(x);
Defect Concepts: What is a
Defect?

Two things occur before a defect
can be detected
1.
2.
Fault – Incorrect value in the internal
state of the program
Failure – When a fault is realized as
output
int i = 5;
int y = 6;
int x = y+1 // Causes a Fault
Print(x); // Causes a Failure
A Failure may not be detected until well after the fault has occurred!
Defect Concepts: Static Vs.
Dynamic Discovery

Static – Does not utilize executing
code




Software Inspections
Walkthroughs
Complexity Mapping
Dynamic – Utilizes executing,
compiled code


Runtime tests
Automated defect discovery tools
Defect Estimation Methods



Estimating the number of defects is by no
means an exact science
There are lots of different ways to
determine an estimate
Most of the methods are highly tied to a
specific organization or software baseline


Not good enough for general use
Three general methods will be discussed



Linear Regression
Capture/Recapture
Artificial Intelligence


Bayesian Belief Networks
Neural Networks
Defect Estimation Methods
Linear Regression

Method:


Determines a mathematical
expression that relates some
number of independent variables
(the input metrics) and the
dependent variable (defect
estimation)
Variable Type: Numeric Only
Defect Estimation Methods
Capture/Recapture

Method:


Some number of defects are
seeded into the software at
development time. When testing
occurs ratio of seeded to
unseeded defects found is used to
estimate the total number.
Variable Type: N/A. Only track
defect counts.
Defect Estimation Methods
AI: Bayesian Belief Networks

Method:


A network that weighs the
relationship among different
variables is created (using expert
knowledge) and processed using
Bayesian probabilities to
determine the quality of the
software.
Variable Type: any
Defect Estimation Methods
AI: Neural Networks

Method:


A multilayer perceptron using the
back propagation algorithm is
trained on legacy defect data and
then provides estimates based on
input data.
Variable Type: Numeric Only
Defect Estimation Variables
Further Research

Three main areas
1. Transformation of non numerical
data into a numerical form
2. Additional estimation methods
that do not utilize numeric only
data
3. Determination of common
relationships among the input
metrics, independent of which
method is used
Conclusion



Organizational Element Data is
useful when paired with the proper
estimation method
Limitations of the most common
method (linear regression) has led
to a general shunning of these
metrics
Defect Estimation approaches are
not ready for real world use

They are too tied to specific cases
Questions
Questions?