Transcript Evaluation of Safety Critical Software by D. Paranas, A. J. van

Evaluation of Safety Critical
Software
by D. Parnas, A. J. van Schouwen, and P. Kwan
Presented by:
Hatem Halaoui
Presentation Outline

Paper
Introduction
 Why is software used?
 Software controllers and other controllers
 Software concerns
 Modular Structure
 Reliability Assessment for Safety-critical software
 Conclusions

Uncovered issues
 Statistical software crisis information due to bad software
engineering

Introduction
The failure of programmable computer applications could
be life threatening. Computers now have safety-critical
functions.
 To reduce application failures these questions need some
answers:

what standards must a software product satisfy?
 What documentation is needed?
 How much testing is needed?
 How should software be structured?

Why is software a special concern?
Many tragedies were caused by software failures.
 Software systems do not work well until they have been
used and have failed repeatedly
 Software products may fail in their first use because
situations that were anticipated by the programmers were
also overlooked by the test planners
 The use of randomly generated data reduces the likelihood
of shared oversight

Why is software used?
 Software
makes it practical to build more logic
into the system. They can distinguish a large
number of situations and provide suitable outputs.
 Logic implemented in software is easier to change
than that implemented in hardware
 Software provide more information in more
suitable form
How are software controllers like
other controllers?

Similar to other controllers (hardware), software
controllers has the following properties:
1.
2.
3.
4.
Inputs can be described as mathematical functions
Outputs can be described as mathematical functions
of the inputs
Values of the variables can also be described as
mathematical function of the controller output
Relations between variables can be described
How are software controllers different
from other controllers?

Complexity:




Error Sensitivity: software is much more sensitive to small errors.
Hard to test:



Huge number of testing cases in software
what about points between the testing points
Correlated failures:



Programs need more precise documentation which might fill a book case
programs need much more time (years ) before being trusted
Assumptions for hardware design are invalid for software: like assuming that a
hardware failures are not strongly correlated
In contrast to hardware, duplicating software components does not imply higher
reliability
Lack of professional standards


No professional software engineers
No agreement on the skills and knowledge that a software engineer should have
Software Testing Concerns





We cannot test software for correctness: large number of states
It is difficult to make accurate predictions of software reliability and
availability
It is not practical to measure the trustworthiness of software: a
software is trustworthy if the probability of having a potentially
catastrophic flaw is acceptably low.
There is a role for testing: some scientist argue that one should test
rather than spending this time review and does mathematical
verifications
There is a need for an independent validation agency: it is difficult
to test one’s own design in an unbiased way
Software Reviewability Concerns
 Why
is the reviewability concern for software?
 Before,
there were not too much care about software,
its documentation, and its trustworthiness
 Now, much more software controllers in big
equipments and industry
 Reviews are very important to increase the correctness
of software
 Engineers are required to do documentations
What Reviews are needed
Review for the correct intended function
 Review for maintainable, understandable, well
documented structure.
 Review each module to verify the algorithm and date
structure design are consistent with the specified behavior
 Review the code for consistency with the algorithm and
data structure design.
 Review test adequacy: was the testing sufficient to provide
sound confidence in the software functioning.

What documentation is required to
review functional requirements?


Should be done by engineers and scientists who understand the
situation to be monitored and devices to be controlled.
The functional requirement can be stated by giving three
mathematical functions:
1.
2.
3.

The required values of the controlled variables in terms of values of the
relevant observable environmental parameters
The computer inputs in terms of those observable environmental variables
The values of the controlled environmental variables in terms of the
computer output
This can be given as a set of tables and formulae
What documentation is required to
review software structure?


Documents that describe the breakdown of the program into
modules (each module is a unit which could be a set of programs).
The purpose of this review is to make sure that:
1.
2.
3.
4.

The structure allow independent development and change
All programs need are included once and only once
Module interfaces are defined
Modules are compatible to each other and meet the requirements
Three types of documents are needed:
1.
2.
3.
Requirements and specifications
Informal document describing responsibilities of each module
Module specification (black box description)
What documentation is required to
review module’s internal design?
 Design
documentation which include description
of two types of mathematical functions: program
functions and abstraction functions
 Programs must be described in a hierarchical way
where each small program or subprogram is
described
 This needs software engineers and specialist in the
areas that the software is dealing with
What documentation is required to
review the code?
 Algorithms
and data structures
 The reviewers should be experienced users of the
hardware and compilers
What documentation is required to
review the test plan review?
 Mathematical
verification of the code
 The test plan should be described in a document
not available for designers
 Random testing is required
 Should be reviewed by specialists in software
testing and application area
Reviewing relationships between
these documents
Relationship between these documents should be verified
 Main issues about documents relationships

Module guide should show no ambiguity about module’s
responsibilities
 Each module design document should show arguments that
verify the module specifications
 The module design document, which describes the algorithms
should be mapped on the code
 Test plans must show how the tests are derived and how they
cover the requirements

Why is configuration management
essential for rigorous reviews
 All
documents must be kept consistent when
changes made.
 If document is changed, all related documents
must be reviewed and possibly change.
 Designers and reviewers should be notified of
changes and have the latest versions of documents
Modular Structure
Software should be organized in accordance with the
modern principles of “information hiding”, “Object
Orientation”, “Separation of Concerns”, “Encapsulation”,
“Data Abstraction”, etc.
 Large programs should be organized in smaller and
simpler assignments known as modules.
 Software engineers should be trained to use abstract
mathematical specifications
 Details that change a lot should be hidden

Reliability Assessments for SafetyCritical Software
 Some
engineers argue that the correctness of a
software is not probabilistic
 Not true!! Software appears to exhibit stochastic
properties
 We can never say that a software is 100% reliable
What should be measuring?
Encountering the sequence of input that lead to failures
 Different kinds of software needs different measuring
 Example



In Safety-critical applications errors are not acceptable
We cannot predict the failure rate from failure rates of
individual lines or subprograms (a failure in a subprogram
might lead into a failure in others)
The finite state machine model of
programs
Software is a machine that is described by two functions:
Next State and Output
 Software can be viewed as a finite state machine described
by large tables
 Loading a program in the machine selects a terminal
submachine consisting of all states can be reached from
the initial state
 It helps to know the number of tests we need to perform

Use of hypothesis testing
 We
need to confirm that failure probability is
below upper bound
 Assume the upper bound probability of a failure is
1/h => reliability is 1-1/h
 For N random tests the probability there will be no
failure during testing is M=(1-1/h)N
Hypothesis testing example
Reliability estimates of programs
Three classes of programs:

Memoryless batch programs (each run is independent of others)
Batch programs with memory (a run might depend on previous one)
Real time programs (run continuously)
1.
2.
3.



Reliability of memoryless batch programs : tests consist of single run for each
set of inputs
Reliability of batch programs: test consist of single run of input data and
internal state
Reliability of Real time programs:



when testing the concept of inputs must be replaced with a multidimensional
trajectory.
Each trajectory gives the input values as continuous functions of time
Length of trajectories should be estimated
Conclusions
Information hiding should be used
 Documentation should be complete, precise and making
use of mathematical notations rather than natural language
 Mathematical verification techniques must be used
 An independent agency should test the software using
valid random testing to get estimates of the reliability of
the software at critical situations.
 Hypothesis testing should be used to allow us to know the
probability that the software meet the requirements

Uncovered Issues
 One
of the most important issues that lead into
software crisis is the errors resulting from
discreatizing continuous data
 Such data will not be accurate
 Operations applied on multiple of such data will
lead to high loss of precision
Software Crisis Examples


Source: http://www.unt.edu/benchmarks/archives/1999/july99/crisis.htm
Software crisis:






The cost of owning and maintaining software in the 1980’s was twice as expensive as
developing the software.
During the 1990’s, the cost of ownership and maintenance increased by 30% over the
1980’s.
In 1995, statistics showed that half of surveyed development projects were operational,
but were not considered successful.
The average software project overshoots its schedule by half.
Three quarters of all large software products delivered to the customer are failures that
are either not used at all, or do not meet the customer’s requirements.
Main reasons


Software development is seen as a craft, rather than an engineering discipline.
The approach to education taken by most higher education institutions encourages that
"craft" mentality.