SWE 637: Here! Test this!
Download
Report
Transcript SWE 637: Here! Test this!
CS 217
Software Verification and
Validation
Week 7, Summer 2014
Instructor: Dong Si
http://www.cs.odu.edu/~dsi
REVIEW OF BB-testing
Black-box Testing
1.
Input Space Partitioning
2.
Boundary Value Analysis
3
Example 1: compare two numbers
– p50 of week3
Function ‘Compare (x, y)’
Inputs: Two numbers – x and y
Outputs: A larger number between x and y
(x, y)
z
Compare (x, y) = z
4
– p51 of week3
• Equivalence Classes:
{ (x, y) | x < y }
{ (x, y) | x > y }
Valid inputs
{ (x, y) | x = y }
{ input other than a pair of numbers,
Invalid inputs
“as&%dfget^$(&w” }
5
– p52 of week3
Valid (x, y) Input Space
Three test cases:
(1, 2) --- 2
x<y
(8, 8) --- 8
(100, 30) --- 100
x>y
Plus one test cases:
(^&%*) --- ERROR
6
Example 2: Loan application
- p53 of week3
Customer Name
Account number
Loan amount requested
Term of loan
Monthly repayment
2-64 chars.
6 digits, 1st
non-zero
$500 to $9000
1 to 30 years
Term:
Repayment:
Minimum $10
Interest rate:
Total paid back:
Choosing (or defining) partitions seems easy, but is easy to get wrong…
7
Customer name
Number of characters:
invalid
Valid characters:
Conditions
Customer
name
1
2
valid
A-Z
-’ a-z
space
Valid
Invalid
Partitions
Partitions
2 to 64 chars < 2 chars
valid chars
> 64 chars
invalid chars
64 65
invalid
Any
other
Valid
Invalid
Boundaries
Boundaries
2 chars
1 chars
64 chars
65 chars
0 chars
8
Loan amount
499
invalid
Conditions
Loan
amount
Valid
Partitions
500 - 9000
500
9000
valid
Invalid
Partitions
< 500
>9000
0
non-numeric
null
9001
invalid
Valid
Invalid
Boundaries
Boundaries
500
499
9000
9001
9
Black-box Testing
1.
Input Space Partitioning
2.
Boundary Value Analysis
10
Example 3 - Bonus
Consider a program IsAnOddNumber(x) which output “Yes”
or “No” to the user to tell if the input integer x is a odd
number or not.
If you are planning to classify all possible user inputs into
two classes - valid and invalid inputs
1)
How do you partition the valid input space (the integer
number space)?
2)
Can you give an example of invalid test data?
11
Example 3 - Bonus
If your software manager asks the program MUST ONLY
take input real number x as: -100<x<100
1)
How do you partition all possible user inputs into two
classes - valid and invalid inputs now?
2)
If the software manager further asks you to do some
Boundary Value Analysis, then what do you plan to do for
the BVA analysis?
12
Example 4 - Bonus
Consider a program Absolute(x) which calculate the absolute
value of a input real number x.
If you are planning to classify all possible user inputs into
two classes - valid and invalid inputs
1)
How do you partition the valid input space (the real
number space)?
2)
Can you give an example of invalid test data?
13
Example 4 - Bonus
If your software manager asks the program MUST ONLY
take input real number x within the range of:
-321<x<-123, 123<x<321
1)
How do you partition all possible user inputs into two
classes - valid and invalid inputs now?
2)
If the software manager further asks you to do some
Boundary Value Analysis, then what do you plan to do for
the BVA analysis?
14
Today’s topic
WHITE-BOX TESTING
Styles of Testing
Testing traditionally can be conducted in two styles
Black-Box testing
Try to choose "smart" tests based on the requirements,
without looking at the code.
White-Box testing
Try to choose "smart" tests based on the structure of the
code, with minimal reference to the requirements.
16
Definition of White-Box Testing
Testing based on analysis of internal logic (design, code,
etc.). (But expected results still come from requirements.)
Use different techniques to check every visible path of the
source code to minimize errors.
It is the ability to know which line of the code is being
executed and being able to identify what the correct
output should be.
17
Characters of WB testing
Exploits the structure, control or data flow of programs in
order to design test cases.
Typically performed during coding.
Allows to test parts of the program, since you know the
structure. (with black‐box is not possible)
Allows to cover systematically every part of the program.
18
WB Testing Techniques
Logic coverage: (learned in previous classes)
Statement coverage
Branch coverage
Condition coverage
…
Dataflow based testing / Path coverage: all program paths
have been traversed at least once
19
Pseudo code & Control/Dataflow Graphs
“nodes”
“edges”
Input
x
Absolute (x)
{
IF (x>=0)
THEN y = x;
IF (x>=0)
ELSE IF (x<0)
y = -x;
y = x;
ELSE IF (x<0)
THEN y = -x;
y
Output y;
}
output
20
Can you draw a graph?
Absolute (x)
{
IF (x>0)
THEN y = x;
ELSE IF (x==0)
THEN y = x;
ELSE IF (x<0)
THEN y = -x;
Output y;
}
21
Example
Fun (x, y)
{
z = 0;
IF ((x>z) OR (y>z))
THEN z = 2;
ELSE
z = 3;
Test cases:
1. Fun (0, 0)
2. Fun (2, 0)
3. Fun (0, 2)
4. Fun (2, 2)
5. Fun (8, 9)
Output z;
}
22
Control/Dataflow of the example
Test cases:
1.
Fun (0, 0)
2.
Fun (2, 0)
3.
Fun (0, 2)
4.
Fun (2, 2)
5.
Fun (8, 9)
Input x, y
Fun (x, y)
{
z = 1;
z = 1;
IF ((x>z) OR (y>z))
ELSE
IF ((x>z) OR (y>z))
THEN z = 2;
ELSE
z = 3;
z = 2;
z = 3;
Output z;
}
Output z;
23
Statement coverage
Has each statement in the program been executed?
√
√
√
24
Branch coverage
Has each branch of each control structure been executed?
For example, given an if statement, have both the T and F
branches been executed?
Another way of saying this is, has every edge in the
Control/Dataflow graph been executed?
25
Condition coverage
Has each Boolean sub-expression evaluated both to true
(T) and false (F) ?
Fun (x, y)
{
z = 1;
Test cases:
1.
Fun (0, 0)
2.
Fun (2, 0)
3.
Fun (0, 2)
4.
Fun (2, 2)
5.
Fun (8, 9)
IF ((x>z) OR (y>z))
THEN z = 2;
ELSE
z = 3;
X>1
Y>1
T,F?
T,F?
Output z;
}
26
Exercise
Consider a program ‘Compare’ which compare the value
between two numbers. The inputs are two real numbers x
and y, the program outputs the larger number between x
and y.
Compare (x, y)
{
IF (x>y)
THEN z = x;
ELSE IF (x==y)
THEN z = x;
ELSE IF (x<y)
THEN z = y;
Output z;
}
27
Exercise - Control/Data Flow Graphs
I have provided you the Pseudo code,
Can you draw a Control/Dataflow Graphs?
Try it! This is a bonus question with +5 points towards to
FINAL!!
28
Exercise - Statement Coverage
Statement Coverage requires that each
statement will have been executed at least
once.
What is the minimum number of test cases
required to achieve statement coverage for the
program Compare (x, y) ?
29
Exercise - Branch Coverage
Branch Coverage requires that each branch
will have been traversed at least once.
What is the minimum number of test cases
required to achieve branch coverage for the
program Compare (x, y) ?
30
Condition Coverage
Condition Coverage requires that each
condition (sub-expression) will have been True
at least once and False at least once.
What is the relationship between Branch and
Condition Coverage?
Condition / branch coverage
IF (
X>0
AND
Y>0
)
T,F?
T,F?
T
F
THEN
…
ELSE
…
32
Condition Coverage example –
Combination !
A
B
Branch
IF A or B
THEN s1;
test 1
T
F
true
test 2
F
F
false
ELSE
test 3
T
T
true
test 4
F
T
true
s2;
Combinations of condition values:
TT, TF, FT, FF
Dataflow based testing -> Path coverage
All program paths have been traversed at least once.
Input x, y
z = 1;
IF ((x>z) OR (y>z))
z = 2;
ELSE
Two paths !
z = 3;
Output z;
34
Path coverage - Example 1
if a then s1
else if b then s2
else if c then s3
else s4
end_if_then_else
end_if_then_else
end_if_then_else
# Paths: ___
# Branches: ___
35
WB Testing Techniques
Logic coverage: (learned in previous class)
Statement coverage
Branch coverage
Condition coverage
…
Dataflow based testing / Path coverage: all program paths
have been traversed at least once
36
Coming Up Next week…
Lab 2: White-box Testing
Preparation:
Go over the slides of WB testing
Understand “Code coverage” & “Data flow / Path”