Transcript Testing Interactions

Software Testing and Quality
Assurance
Testing Interactions
Reading Assignment

John McGregor and David A. Sykes, A Practical
Guide to Testing Object-Oriented Software,
Addison-Wesley, 2001, ISBN: 0-201-325640.

Chapter 6: Testing Interactions
2
Objectives


To understand different types of interactions.
Learn techniques on how to select test cases
to run.
3
Topics covered








Introduction
Object interactions
Testing object interactions
Sampling test cases
Testing off-the-shelf components
Protocol testing
Test patterns
Testing exceptions
4
Object Interactions


An instance of a trusted primitive class, for
example, may contain no faults, but if the
services of that instance are not used
correctly by other program components, then
the program contains faults.
Thus the correct collaboration—or
interaction—of objects in a program is critical
to the correctness of the program.
5
Introduction


Object-oriented program comprises a collection
of objects that collaborate to solve some
problem.
Most classes have collaborators:



The methods in the class interact with instances of
other classes.
The interactions being tested are between
objects at runtime.
The focus of interaction testing is ensuring that
messaging occurs correctly with objects whose
classes have already been tested separately.
6
Object interactions


An object interaction is simply a request by one object
(the sender) to another (the receiver) to perform one of
the receiver’s operations and all of the processing
performed by the receiver to complete the request.
Object interactions include:



Messages between an object and its components
Messages between an object and other objects with which it is
associated.
We want to consider the impact of these interactions
both on the internal state of the receiving object and on
those objects with which it has an association
7
Partial Class Testing


Partial class testing is a result of the iterative,
incremental development approach.
In general,

It is not possible to sequence the development of a
class so that all the classes it needs to interact with
are totally developed and tested.

A project's schedule is usually based on delivery of
end-user functionality (use cases).
8
Partial Class Testing (cont.)

Lower level— Primitive classes are more
likely to be completely developed at one time
and tested as a complete unit.

Other classes are therefore developed and
tested incrementally.
9
Object Interactions: Identifying Interactions

A primitive class is one that can be used
without any collaboration with other objects.

A nonprimitive class is one that requires
collaboration with other objects in order to
accomplish its objectives.
10
Object Interactions: Identifying Interactions
(cont...)

Possible collaborations:




Parameter: public operation collaborates with an
object passed in as an argument (formal
parameter).
Return: public operation returns an object with
which it collaborates.
Internal: a method of a class creates an instance
of another as part of its implementation.
Global: reference to a global instance of some
class (undesired).
11
Object Interactions: Identifying Interactions
(cont...)

Categories of nonprimitive classes based on
degree of interaction:

Collection classes: maintain references to
instances of other classes, but do not interact with
those instances.

Collaborating classes: interacts directly with its
associated classes classes that are not
collection classes.
12
Object Interactions: Identifying
Interactions— Collection Classes

Collection classes: Use objects in their
specifications but never actually collaborate
with any of them.

Instead they do one of the following:




Store references
Create instances
Delete instances
Examples: lists, stacks, and queues.
13
Object Interactions: Identifying
Interactions— Collaborating Classes

Collaborating Classes



Non-primitive classes that are not collection classes.
They are classes that use other objects in one or
more operations as part of their implementation.
When a post condition of an operation in a class's
interface

refers to the state of an instance of an object and/or

specifies that some attribute of that object is used or
modified,
then that class is a collaborating class.
14
Object Interactions: Identifying
Interactions— Collaborating Classes (cont...)

The number of potential collaborations can
become impossibly large quickly .

Often the bugs that are most serious do not
arise from the interaction of two objects, but
from the interactions between a complete set
of objects.
15
Object Interactions: Identifying
Interactions— Collaborating Classes (cont...)

Factors that determine the size for testing:


Composition and association:

The interaction of the composing object with its
composed attributes is tested.

The interaction between an object and its associated
objects are tested as successive layers of aggregation
are integrated.
The number of layers of aggregations created
between interaction tests is closely related to the
visibility of defects.
16
Object Interactions: Identifying
Interactions— Collaborating Classes (cont...)

Object complexity: the more complex the objects,
the fewer that should be integrated prior to a
round of testing.




number of parameters for each method,
number of methods,
number of attributes in each object.
Trying to test a chunk that is too complex often
results in defects that successfully hide from the
tests.
17
Testing Object Interactions: Testing
Collection Classes

Collection classes are tested using techniques
for primitive classes:

A test driver will create instances that are passed as
parameters in messages to a collection being tested.

Test cases center around ensuring that those
instances are correctly incorporated into and
removed from the collection class

If defensive design has been used then include
negative test cases in the test suite.
18
Specifying Interactions


Assume that operations defined by a class are
specified by preconditions, post conditions, and
class invariants.
It is important to know whether defensive design
or design by contract has been used in creating
the specification of the particular interface to be
tested.
19
Specifying Interactions (Cont.)

Defensive design assumes that little or no
checking of parameter boundaries occurs
prior to a message being sent.



reduces the number of clauses in preconditions,
requires checks internally for violations of attribute
constraints,
increases the number of clauses in post
conditions.
20
Specifying Interactions (Cont.)

Design by contract assumes that appropriate
preconditions are checked prior to a message
being sent and that the message is not sent if
any of the parameters are outside acceptable
limits.
 increases the number of clauses in
preconditions,
 requires no checking internally for violations of
attribute constraints, and
 reduces the number of clauses in the post
condition clause.
21
Testing Object Interactions: Testing
Collaborator Classes


The complexity of testing a collaborating
class is greater than that of testing a
collection class or a primitive class.
For a collaborator class under test:


Identify the collaborating classes
Determine the sequence of testing



Collaborating classes,
Next level of collaboration
Then the next level of collaboration and so on
22
The Interaction between Testing and
Design Approach




Contract design places more responsibility on the
human designer than on error-checking code.
This reduces the amount of class-level testing since
there are fewer paths due to a smaller amount of
error-checking code.
At the interaction level, there is more testing
required in order to be certain that the human
designer has complied with the client side of the
contract using precondition constraints.
A focus of interaction testing is whether the
preconditions of methods in a receiving object are
being met by the sending object.
23
The Interaction between Testing and
Design Approach (Cont.)



It is not legitimate to build test cases that
violate these preconditions.
The intention is to determine that the sending
object checks the preconditions of the
receiving object before sending the message
inappropriately.
The test should also check whether the
sending object aborts correctly, probably by
throwing an appropriate exception.
24
Sampling Test Cases

When the number of test cases is too large to
handle reasonably

A systematic technique is needed for
determining which ones to use.

We are interested with a testing approach in
which the level of coverage can be increased
systematically.
25
Sampling Test Cases



Population: all possible test cases that can be executed.
It includes all preconditions and all possible combinations
of input values.
A sample is a subset of a population that has been
selected based on some probability distribution.
Possibilities for determining which test cases to select

Based on probability distribution


A probability distribution defines, for each data value in a population,
a set of allowable values and the probability that value will be
selected.
Uniform probability distribution

Each value in the population is assigned the same selection
probability
26
Sampling Test Cases (cont...)




A stratified sample is a set of samples in which
each sample represents a specific subpopulation
Stratified sample tests are selected from a series
of categories.
A population of tests is divided into subsets so
that a subset contains all of the tests that
exercise a specific component. Sampling occurs
on each subset independent of the others
Example: select a sample of test cases from the
use cases of each actor
27
Sampling Test Cases (cont...)



Stratifying the test case samples by each
actor provides an effective means of
increasing the reliability of the system.
Running the selected tests uses the system
the way that it will be used in typical
situations and finds those defects that are
most likely to be found in typical use.
Removing these defects produces the largest
possible increases in reliability with the
smallest effort.
28
Sampling Test Cases (cont...)

Use of random number generator in sampling

Advantage: all values have equal probability

Disadvantage: test cases cannot be reproduced.

Example Velocity class, set direction: values
between 0 and 359, inclusive.
int (random() * 360).
29
Sampling Test Cases (cont...)

The first dimension in sampling is considering
the class family

Class family: a set of classes related by
inheritance.


Example, Sprite is the base class in a very large class
family, which is a set of classes related by inheritance.
An object from any one of the classes in the family can
be substituted for the sprite parameter.
30
Sampling Test Cases (cont...)

The second dimension for sampling



Consider that each member of the family may
have different states that can cause two objects
from the same class to behave differently.
There will be the same number of states or more
states in the derived class as there are in the base
class.
Cover the states defined for each class with
special emphasis on the new states added at that
level in the inheritance hierarchy.
31
Sampling Test Cases: Orthogonal Array
Testing (OATS)




Orthogonal arrays provide a specific sampling
technique that seeks to limit the explosion by
defining pair-wise combinations of a set of
interacting objects.
An orthogonal array is an array of values in
which each column represents a factor (a variable
in an experiment).
It represents a specific class family (a class and its
children)
E.g. 3 factors with 3 levels each (27 possibilities),
with pair-wise only 9 possibilities
32
Sampling Test Cases: Orthogonal Array
Testing (OATS)
This is a systematic way of reducing the number of test
cases.
•
• Most of the errors that are encountered are between
pairs of objects rather than among several objects.
33
Sampling Test Cases: Orthogonal Array
Testing (OATS)



Decide how many independent variables will be
tested for interaction - Factors of the array.
Decide the maximum number of values that
each independent variable will take on - the
Levels of the array.
Find a suitable orthogonal array
- has at least as many Factors as needed
- has at least as many levels for each of those factors


Map the Factors and values onto the array.
Transcribe the Runs into test cases.
34
Sampling Test Cases: OATS Example 1

Number of states:






Class A: 2
Class B: 3
Class P: 2
Class C: 2
Class D: 3
Class E: 3
* State transition diagrams are not shown
35
Sampling Test Cases: OATS Example 1

Step 1: Identify all factors:
sending hierarchy,
 receiving hierarchy,
 parameter position in the message
six factors in the example:






class A hierarchy,
class P hierarchy,
class C hierarchy and
factors for the states associated with each class hierarchy
Step 2: Determine levels for each factor by considering
the set of possible values:



One factor has one level: the parameter class family only has one
member: P
Two factors have a maximum of two levels; A and C; the maximum
number of states for a class in the P family is two.
Three factors have a maximum of three levels: B, D and E.
36
Sampling Test Cases: OATS Example 1



Step 3: Locate a standard orthogonal array that fits the
problem.
6 factors
3 levels
37
Sampling Test Cases: OATS Example 1

Step 4: Establish a mapping from each factor onto
the integers in the array so that standard array can
be interpreted






The first column in L18 can be used to represent the sender class
family which has two classes: A and B (1 corresponds to A class and
2 corresponds to class B)
The second column in L18 when there is a difference in the number of
levels (class A has 2 states and class B has 3 states).
The third column in L18 represents the parameter hierarchy that only
has one class, P. (any value in the third column represents P).
The fourth column represents the state of P (which there are two).
The fifth column represents the class C hierarchy, which has three
members.
The sixth column represents the states of the C, D, and E classes.
38
Sampling Test Cases: OATS Example 1
2
39
Sampling Test Cases: OATS Example 1

Step 5: Construct test cases based on the
mapping and the rows in the table.

Each row in the orthogonal array specifies one
specific test case.
40
Sampling Test Cases: OATS Example 1—
Result Interpretation

The 10th column
represents:



An instance of B in
state 1
Is to send a
message by
passing an
instance of class P
in state 2
To an instance of
class E in state 2.
41
Sampling Test Cases: OATS Example 2

Consider a web page with three distinct
sections (Top, Middle, and Bottom) that can
be individually shown or hidden by the user.
You wish to test the interactions of the
different sections.
42
Sampling Test Cases: OATS Example 2

There are three independent variables (the
sections of the page).

Each variable can take on two values (hidden
or visible).

An L4(23) orthogonal array will do the job —
two levels for the values and three factors for
the variables.

Mapping the values onto the array where
Hidden=0 and Visible=1
43
Sampling Test Cases: OATS Example 2
OA before mapping factors
Factor 1
Factor 2
Factor 3
Run 1
0
0
0
Run 2
0
1
1
Run 3
1
0
1
Run 4
1
1
0
OA after mapping factors
Top
Middle
Bottom
Test case 1
Hidden
Hidden
Hidden
Test case 2
Hidden
Visible
Visible
Test case 3
Visible
Hidden
Visible
Test case 4
Visible
Visible
Hidden
44
Sampling test cases: Adequacy Criteria for
OATS





Exhaustive: all possible combinations of all factors
are considered.
Minimal: only the interactions between the base
classes from each hierarchy are tested.
Random: the tester haphazardly selects cases from
several of the classes.
Representative: a uniform sample that ensures that
every class is tested to some level.
Weighted Representative: Add cases to the
representative approach based on relative
importance or risk associated with the class.
45
Testing off-the-shelf components





Functionality can be added to an application by
purchasing “Components”.
Quality of components varies from vendor to
another.
Acceptance testing for the component in the context
in which it will be used is necessary.
Stress tests: extreme, may be correct or incorrect
values e.g., moving mouse a lot to generate a large
number of mouse move events, making multiple
menu selections, etc.
Tests are normally based on the main class.
46
Testing off-theshelf
components:
example ―a
component test
plan for Grid
47
Protocol testing




Protocol testing investigates whether the
implementation of a class satisfies its specification
Protocol: sequence of method invocations by
combining a method whose postconditions enable
the precondition of another method
Each protocol corresponds to a sequence of
states beginning with initial states of the two
objects, a sequence of states for each object, and
ending with terminal states (denoted in the state
diagram)
A test case takes the two objects through one
complete sequence of methods
48
Protocol testing: Example
Timer maintains list of observers to notify when a timer event occurs.
When it is enabled, it notifies the observers
49
Protocol testing: Example

A protocol can be found by tracing through
the state diagram.



One protocol would be to create the object then
sending one or more attach(...) messages
followed by the enable() message, the disable
message, and finally, the delete() message.
This provides a life-cycle test case.
This provides an effective test of the object in the
ways that it will interact with its client objects.
50
Test patterns




Test patterns are design patterns for test
software.
A design pattern: captures and reuses design
knowledge that has gained widespread
application in the OO software development
community, e.g., Grid component is based on
the Listener design pattern
A pattern is a specific configuration of
interactions among a set of objects that form
some cluster in the overall design.
For a design pattern there can be a test pattern
51
Testing exceptions


An exception: provides an alternative route
for returning from a method that does not
necessary move to the next statement after
the method is invoked.
Exceptions are powerful:

The exceptional return value is an object and can
be arbitrary complex.

The points at which an exception is thrown varies
based on the depth of the aggregation hierarchy.
52
Testing exceptions (cont...)

Testing exceptions provides two perspectives:

At the class level, whether each of the methods in that class
does throw the exceptions that it claims to in the appropriate
circumstances.


During integration, interaction testing will determine whether
the exceptions that are being thrown at correct time are being
caught at correct place.


The coverage criteria requires that a class throws every exception
contained in the specifications of the methods. There would be at
least one test case per exception.
The coverage criteria for this level of testing is to be certain that
every exception thrown is caught in the correct location.
Both of these points of view can be tested very early in
the development.
53
Testing exceptions: testing interactions at the
system level


When the components become complex it is
easier to test them in the context of the
application itself instead of in the environment
provided by a test driver.
The interactions that can be tested at the
system level are only those that can be
verified at the system level (e.g. user
interface):
54
Key points




Primitive and nonprimitive classes.
Possible collaborations: parameter, return,
internal, global.
Categories of nonprimitive classes: collection
and collaborating classes
Factors that determine the size for testing



Composition and association:
aggregations
Object complexity
55
Key points (cont...)




Possibilities for determining which test cases to
select: based on probability distribution, uniform
probability distribution and probability distribution
based on the use profile.
A stratified sample is a set of samples in which each
sample represents a specific subpopulation
An orthogonal array is an array of values in which
each column represents a factor (a variable in an
experiment).
Adequacy criteria for OATS: exhaustive, minimal,
random, representative, and weighted
representative.
56
Key points (cont...)




Stress tests: extreme, may be correct or incorrect
values
Protocol: sequence of method invocations by
combining a method whose postconditions enable the
precondition of another method
A design pattern: captures and reuses design
knowledge that has gained widespread application in
the OO software development community
An exception: provides an alternative route for returning
from a method that does not necessary move to the
next statement after the method is invoked.
57