Transcript White-Box Testing

Chapter : 18
Testing Conventional
Applications
Software Testing Fundamentals
Testability
•
•
Software testability is simply how easily a
computer program can be tested.
The following characteristics lead to testable
software:
1) Operability : The better it works, the more
efficient it can be tested.
2) Observability : What you see is what you test
3) Controllability : The better we can control the
software, the more the testing can be automated
and optimized.
4) Decomposability : By controlling the scope of testing,
we can more quickly isolate problems and perform
smarter retesting.
5)Simplicity : The less there is to test, the more quickly
we can test it.
6)Stability : The fewer the changes, the fewer the
disruptions to testing
7)Understandability : The more information we have, the
smarter we will test.
Test Characteristics
Following are attributes of good test:
•
•
•
•
A good test has a high priority of finding errors.
A good test is not redundant.
A good test should be “best of breed”.
A good test should neither too simple nor too
complex.
Internal And External Views of
Testing
•
Any engineered product can be tested in one of
two ways:
1) Knowing the specified function that product
has designed to perform (external view)
2) Knowing internal working of a product
(internal view)
White-Box Testing
•
•
White-box testing, also called glass-box testing, is a
test-case design philosophy that uses the control
structure described as a part of component-level
design to derive test cases.
Using white-box testing methods, you can derive test
cases that
1) guarantee that all independent paths within a
module have been exercised at least once,
2) exercise all logical decisions on their true and
false sides,
3) execute all loops at their boundaries and within
their operational bounds,
4) exercise internal data structures to ensure their
validity.
Basis Path Testing
•
Basis path testing, a white-box testing technique,
enables the test-case designer to derive a logical
complexity measure of a procedural design and use
this measure as a guide for defining a basis set of
execution path.
•
Test cases derived to exercise the basis set are
guaranteed to execute every statement in the
program at least one time testing.
Flow Graph Notation
•
•
•
•
•
•
The flow graph depicts logical control flow using
some notations.
Figure : 18.1
Each circle in flow graph is called flow graph node
and represents one or more procedural statements.
The arrows on flow graph is called edges or links
and represents flow of control.
Areas bounded by edges and nodes are called
regions.
When counting regions, we include the area outside
the graph as a region.
•
•
•
•
Figure 18.2
A compound condition occurs when one or more
Boolean operators (OR,AND) is present in a
conditional statement.
Figure 18.3
Each node that contains a condition is called a
predicate node and is characterized by two or more
edges emanating from it.
Independent Program Paths
•
•
•
•
•
An independent path is any path through the
program that introduces at least one new set of
processing statements or a new condition.
In terms of a flow graph, an independent path must
move along at least one edge that has not been
traversed before the path is defined.
Example
These paths constitute a basis set for flow graph
18.2
The basis set is not unique. A number of different
basis sets can be derived for a given procedural
design.
•
•
•
•
The computation of cyclomatic complexity helps to
look for number of paths.
Cyclomatic complexity is a software metric that
provides a quantitative measure of the logical
complexity of a program.
In context of basis path testing method, the value
computed for cyclomatic complexity defines the
number of independent paths in the basis set of a
program.
Cyclomatic complexity is computed in one of three
ways:
1)The number of regions of the flow graph corresponds
to the cyclomatic complexity.
2)Cyclomatic complexity V(G) for a flow graph G is
defined as
V(G) = E – N + 2
where E is number of flow graph edges and N is
number of flow graph nodes.
3)Cyclomatic complexity V(G) for flow graph G is defined
as
V(G) = P + 1
where P is number of predicate nodes.
Deriving Test Cases
Steps :
1)Using the design or code as a foundation, draw a
corresponding flow graph.
2)Determine the cyclomatic complexity of the resultant
flow graph.
3)Determine a basis set of linearly independent paths.
4) Prepare test cases that will force execution of each
path in the basis : Data should be chosen so that
conditions at the predicate nodes are appropriately
set as each path is tested. Each test case is
executed and compared to expected results. Once
all test cases have been completed, the tester can
be sure that all statements in the program have been
executed at least once.
Graph Matrices
•A data structure, called a graph matrix, can be quite
useful for developing a software tool that assists in
basis path testing.
•A graph matrix is square matrix whose size (i.e. no
of rows and columns) is equal to the no of nodes on
the flow graph.
•Each row and column corresponds to an identified
node.
•Matrix entries corresponds to connection (an edge)
between nodes.
•Figure
•
•
•
•
•
•
1.
2.
3.
4.
A Each node on flow graph is identified by no.
Each edge is identified by letters.
A letter entry is made in the matrix to
corresponds to a connection between two
nodes.
A link weight can also be added to each matrix
entry.
If the link weight is 1 then connection exists. If it
is 0 then connection does not exist.
A link weights can be assigned other properties
The probability that a link will execute.
The processing time expended during traversal
of a link.
The memory required during traversal of link.
The resources required during traversal of link.
Control Structure Testing
•
It improves quality of white box testing.
(1) Condition testing
(2) Data flow testing
(3) Loop testing.
Condition Testing
•
•
•
•
•
It is test case design method that exercise the logical
condition contained in the program module.
A relational expression takes the form
E1 <relational operator> E2
Compound condition is composed of two or more
conditions, Boolean operators and parentheses.
If a condition is incorrect, then at least one
component of condition is incorrect. Types of errors
in condition include Boolean operator errors
(incorrect/missing/extra Boolean operators) ,
Boolean variable errors, parenthesis errors,
relational operator errors and arithmetic expression
errors.
The condition testing method focuses on the testing
each condition in the program to ensure that it does
not contain errors.
Data Flow Testing
• The data flow testing method selects test paths of a
program according to the locations of definitions and
uses of variables in the program.
• Assume that we are assigning unique statement no to
statements and each function does not modify its
parameters or global variables.
• For a statement with s as its no,
DEF(s) = {x| statement s contains a definition of x}
USE(s) = {x| statement s contains a use of x}
• If statement S is an if or loop statement, its DEF set is
empty and its USE set is based on the condition of the
statement s.
• The definition of variable x at statement S is said to be
live at statement S’ if there exists a path from
statement S to S’ that contains no other definition of x.
• A definition-use (DU) chain of variable x is of the form
[x,s,s’], where s and s’ are statement numbers, x is in
DEF(S) and USE(S’), and the definition of x in
statement S is live at statement S’.
Loop Testing
• It is white box testing technique that focuses on the
validity of loop constructs.
• Four classes of loops can be defined:(1)Simple
(2)Nested
(3)Concatenated loops
(4)Unstructured
•
Simple loop:- The following set of tests can be
applied to simple loops. ( n is maximum no of
allowable passes through the loop).
(1)
(2)
(3)
(4)
(5)
Skip the loop entirely.
Only one pass through the loop.
Two pass through the loop.
m passes through the loop where m<n.
n-1, n, n+1 passes through the loop.
• Nested loop:- Beizer suggests an approach that will
help to reduce the no of tests:
(1)Start at the innermost loop. Set all other loops to
minimum value.
(2)Conduct simple loop tests for the innermost loop
while holding the outer loops at their minimum
parameter ( loop counter) value.
(3)Work outward, but keeping all other outer loops at
minimum values.
(4) Continue until all loops have been tested.
• Concatenated loops:- It can be tested using the
approach defined for simple loops , if each loop
is independent of the other. When loops are not
independent, the approach applied to nested
loop is recommended.
• Unstructured loop:- This class of loop should
be redesigned to reflect the use of the structured
programming language.
Black Box Testing
• Also known as functional testing and behavioral
testing, Black-box testing focuses on the functional
requirements of the software.
• The test designer selects valid and invalid input and
determines the correct output. There is no knowledge
of the test object's internal structure.
• Black Box Testing is testing without knowledge of the
internal workings of the item being tested. For
example, when black box testing is applied to software
engineering, the tester would only know the "legal"
inputs and what the expected outputs should be, but
not how the program actually arrives at those outputs.
• The tester does not ever examine the programming
code.
• The types of testing under this strategy are totally
based/focused on the testing for requirements and
functionality of the work product/software
application.
• Black box testing is sometimes also called as
"Opaque Testing", "Functional/Behavioral Testing" and
"Closed Box Testing".
• It attempts to find errors in following categories:
incorrect or missing functions, interface errors, errors
in data structures or external database access
Graph Based Testing Method
•
•
•
Software testing begins by creating graph of
important objects and their relationship and then
devising a series of test that will cover the graph so
that each object and relationship is exercised and
errors are uncovered.
S.w eng begins by creating graph- collection of
nodes that represent objects; links that represent the
relationship between objects; node weight that
represent properties of node; and link weights that
describe some characteristics of link.
Link may take no of different forms:-
(1) Directed link:- represented by arrow, indicates
relationship moves in only one direction.
(2) A bidirectional link:- also called symmetric link,
implies that relationship applies in both direction.
(3) Parallel link:- it is used when no. of different
relationship are established between graph nodes.
• Figure
• Behavior Testing Methods that make use of graph
1) Transaction flow modeling :-nodes represent steps in
some transactions (e.g. steps required to make
airline reservation using online service) and the links
represent the logical connection between steps.
2)Finite state modeling:-nodes represent different user
observable states of the s.w (e.g. each of the
“screens” that appear as an order entry clerk takes a
phone order) , and the links represent the transitions
that occur to move from state to state.
3)Data flow modeling:- nodes are data objects and the
links are the transformation that occur to translate one
data object into another.
4)Timing modeling:- The nodes are program objects and
links are sequential connections between those
objects
Equivalence Partitioning
•
Divides input domain of program into classes of data
from which test cases can be derived.
• Goals:1) To reduce the number of test cases to a necessary
minimum.
2) To select the right test cases to cover all possible
scenarios.
• Test-case design for equivalence partitioning is
based on an evaluation of equivalence classes for
input condition.
• If a set of objects can be linked by relationships that
are symmetric, transitive, and reflexive, an
equivalence class is present.
•
An equivalence class represents a set of valid or
invalid states for input condition.
• An input condition is either a specific numeric value,
a range of values, a set of related values, or a
Boolean condition.
• Equivalence classes may be defined according to
the following guidelines:
1. If an input condition specifies a range, one valid and
two invalid equivalence classes are defined.
2. If an input condition requires a specific value, one
valid and two invalid equivalence classes are
defined.
3. If an input condition specifies a member of a set, one
valid and one invalid equivalence classes are
defined.
4) If input condition is Boolean. One valid and one
invalid classes are defined.
• Example : An input has certain ranges, one valid and
two invalid equivalence classes are defined. This may
be best explained at the following example of a
function which has the pass parameter "month" of a
date. The valid range for the month is 1 to 12,
standing for January to December. This valid range
is called a partition. In this example there are two
further partitions of invalid ranges. The first invalid
partition would be <= 0 and the second invalid partition
would be >= 13.
• .... -2 -1 0 1 .............. 12 13 14 15 .....
--------------|-------------------|--------------------invalid partition 1
valid partition
invalid partition 2
• (one valid and two invalid equivalence
classes are defined)
BVA (Boundary Value Analysis)
• The boundaries of software component input
ranges are areas of frequent problems.
• A greater no of errors occurrs at the boundries of input
domain rather than in the “center”.
• BVA leads to selcetion of test cases that exercise
bounding values.
• if (month > 0 && month < 13)
• But a common programming error may check a wrong
range e.g. starting the range at 0 by writing:
• if (month >= 0 && month < 13)
• For more complex range checks in a program this may
be a problem which is not so easily spotted as in the
above simple example.
• To set up boundary value analysis test cases you first
have to determine which boundaries you have at the
interface of a software component. This has to be
done by applying the equivalence partitioning
technique. Boundary value analysis and equivalence
partitioning are inevitably linked together.
• .... -2 -1 0 1 .............. 12 13 14 15 .....
--------------|-------------------|--------------------invalid partition 1
valid partition
invalid partition 2
• Applying boundary value analysis you have to
select now a test case at each side of the
boundary between two partitions. In the above
example this would be 0 and 1 for the lower
boundary as well as 12 and 13 for the upper
boundary
• The boundary value analysis can have 6 text
cases. n, n-1,n+1 for the upper limit and n, n1,n+1 for the lower limit.
• Rather than selecting any element of
equivalence class, BVA leads to selection of test
cases at the “edges” of class.
• Rather than focusing solely on input conditions,
BVA derives test cases from the output domain
also.
Guidelines:(1)If an input condition specifies a range bounded
by values a and b, test cases should be
designed with values a and b as well as just
above and just below a and b.
(2)If an input condition specifies a number of values, test
cases should be developed that exercise the minimum
and maximum numbers. Values just above and below
maximum and minimum are also tested.
(3)Apply 1 and 2 to output condition. E.g. assume that
temperature vs. pressure table is required as output
from an engineering analysis program. Test cases
should be designed to create an output report that
produces the maximum (and minimum) allowable
number of table entries.
(4)If internal program data structure have prescribed
boundaries (e.g. an array has defined limit of 100
entries), be certain to design a test case to exercise
the data structure at its boundary.
Testing For Specialized
Environments, Architectures,
And Applications
Testing GUI
• GUI will present you with interesting testing
challenges.
• Because reusable components are a common
part of GUI development environment, the
creation of the UI has become less time
consuming but the complexity of GUI has grown
leading to more difficult in the design and
execution of test cases.
• Finite-state modeling graphs may be used to
derive a series of tests that address specific data
and program objects that are relevant to the
GUI.
• Because of the large number of permutations
associated with GUI operations, GUI testing
should be approached using automated tools.
Testing of Client-Server Architecture
• The distribution nature of client-server
environments, the performance issues
associated with transaction processing, the
complexity of network communication all
combine to make testing of client-server
architectures and the software that resides
within them considerably more difficult than
stand-alone application.
• The testing of client-server software occurs at
three different levels:
1)Individual client applications are tested in a
“disconnected” mode; the operation of the server
and the underlying network are not considered.
2)The client software and associated server
applications are tested in concert (performance),
but network operations are not explicitly
exercised.
3) The complete client-server architecture,
including network operation and performance is
tested.
• Testing Approaches for client-server architecture
1) Application function tests : The functionality of
client application is tested. The application is
tested in stand-alone fashion to uncover an
errors in its operations.
2) Server tests : The coordination and data
management functions of the servers are tested.
3) Database tests : The accuracy and integrity of
data stored by the server is tested. Transactions
posed by the client applications are examined to
ensure that data are properly stored, updated,
and retrieved.
4) Transaction tests : A series of tests are
created to ensure that each class of transaction
is processed according to requirements.
5) Network communication tests : These tests
verify that communication among the nodes of
network occurs correctly and that message
passing transactions, and related network traffc
occur without errors.
• To accomplish these testing approaches,
develop operational profiles derived from clientserver usage scenarios.
• An operational profile indicates how different
types of users interoperate with the client-server
system.
• To develop operational profile, it is necessary to
derive a set of scenarios.
• Each usage scenario addresses who, where,
what, and why.
• Scenarios can be derived using requirement
elicitation techniques.
Testing Documentation and Help Facilities
• Errors in documentation can be devastating
(disturbing) as errors in data or source code.
• Nothing is more frustrating than following a user guide
or an online help facility exactly and getting results or
behavior that do not coincide with those predicated by
the documentation.
• Documentation testing can be approached in two
different phases.
• The first phase, technical review, examines the
document for editorial clarity.
• The second phase, live test, uses the documentation
in conjunction with the actual program.
Testing for Real-Time Systems
• Four step strategy :
1) Task testing : The first step is to test each task
independently. Task testing uncovers errors n logic
and function but not timing and behavior.
2)Behavioral Testing : Using system models created with
automated tools, it is possible to simulate the behavior
of a real-time system and examines its behavior as a
consequence of a external events. Using a technique
that is similar to equivalence partitioning, events are
categorized. Each of these events is tested
individually, and the behavior of the executable system
is examined to detect errors.
3)Intertask testing : Now testing shifts to time-related
errors. Asynchronous tasks that are known to
communicate with one another are tested with
different data rates and processing load to determine if
intertask synchronization errors will occur.
4)System Testing : Software and hardware are
integrated, and full range of system tests are
conducted in an attempt to uncover errors at the
software-hardware interface.