Transcript test case

Software Development Tools
COMP220
Seb Coope
Ant, Testing and JUnit (2)
These slides are mainly based on “Java Development with Ant” - E. Hatcher & S.Loughran. Manning Publications,
2003 and “JUnit in Action”, 2nd edition – P. Tahchiev, F. Leme, V. Massol, G. Gregory, Manning Publications, 2011
JUnit Primer (with ANT) continued
More on writing a test case
Recall that a test case is


a class which imports some JUnit packages
import org.junit.Test;
import static org.junit.Assert.*;
...
and implements one or more no-argument
void @Test methods testXXX()
(See 11. Ant, Testing and JUnit, JUnit Primer,
Slides 5, 6.)

where each testXXX() consists of several
assertion methods typically comparing
expected and actual values.
3
More on writing a test case:
setUp() and tearDown()
Additionally, to prevent one @Test method from
affecting the behaviour of another we can
a. annotate by @Before and implement a method
public void setUp() which
- initializes objects under test
- by storing a state (fixture) as member variables to the test case
b. Optionally, annotate by @After and implement a method
public void tearDown() which
- releases objects/resources under test.
c.
For this to work, we also need to import from Junit:
import org.junit.Before;
import org.junit.After;
The names setUp and tearDown are traditional after JUnit3.
JUnit4 annotations @Before and @After make the behaviour of
setUp and tearDown as required. (See also next slides.)
4
Running a test case with
setUp() and tearDown()
Test Runner calls the same setUp() and
tearDown() methods before and after each @Test
method testXXX() is invoked.
Thus, each @Test method testXXX() runs

under one and the same fixture created by

hence, with no assumptions about the order
in which test methods are called.
setUp()(despite this fixture could probably be
changed by other test methods testYYY()),
This prevents one test from affecting the behaviour of
another.
The following example illustrates this behaviour by
using System.out.println("some message").
5
Test Case setUpTearDownTest.java :
package org.example.antbook.junit;
Put this file under ch04\test according
//import JUnit4 classes:
package declaration.
import static org.junit.Assert.*;
Use your personalised package name!
import org.junit.Test;
//
import org.junit.Before;
//import org.junit.* also would work
import org.junit.After;
//
to the
Here setUp() and tearDown()
methods do nothing besides informing
that what they intend to do.
public class setUpTearDownTest{
@Before
//Runs before each @Test method
public void setUp(){ System.out.println("setUp sets up a fixture"); }
@After
//Runs after each @Test method
public void tearDown(){ System.out.println("tearDown releases fixture"); }
@Test
public void testA(){
System.out.println("testA runs");
@Test
assertTrue("MULTIPLICATION FAILED!!!", 4 == (2 * 2));
methods
}
@Test
//Each method annotated by @Test runs
public void testB(){
System.out.println("testB runs");
assertSame("ADDITION FAILED!!!", 4, 2 + 2);
}
public void SomeTestC(){
System.out.println("SomeTestC runs");
Should it
assertSame("ADDITION FAILED!!!", 5, 2 + 2);
}
fail/run ???
}
6
Compile and Run:
C:\Antbook\ch04>javac -d build\test
test\org\example\antbook\junit\setUpTearDownTest.java
Do not forget to recompile each time!!!
C:\Antbook\ch04>java -cp build\test;C:\JAVA\junit4.8.2\junit-4.8.2.jar
org.junit.runner.JUnitCore org.example.antbook.junit.setUpTearDownTest
JUnit version 4.8.2
.setUp sets up a fixture
testA runs
setUp and tearDown
tearDown releases fixture
surround each
.setUp sets up a fixture
@Test annotated method.
testB runs
tearDown releases fixture
Time: 0.016
OK (2 tests)
Note that someTestC was not run, unlike testA and testB! WHY??
Because only @Test annotated methods should be run by Junit4 test runners.
testA() and testB(), succeeded, so, setUpTearDownTest succeeded too (OK).
Replace 4 by 5 in testA() and recompile and run it again.
7
What will be changed? Will setUpTearDownTest succeed?
Compile and Run:
Do not forget to recompile each time!!!
C:\Antbook\ch04>java -cp build\test;C:\JAVA\junit4.8.2\junit-4.8.2.jar
org.junit.runner.JUnitCore org.example.antbook.junit.setUpTearDownTest
JUnit version 4.8.2
.setUp sets up a fixture
testA runs
tearDown releases fixture
E means that the previous testA
E.setUp sets up a fixture
failed
testB runs
tearDown releases fixture
Time: 0.015
There was 1 failure:
1) testA(org.example.antbook.junit.setUpTearDownTest)
java.lang.AssertionError: MULTIPLICATION FAILED!!!
<many lines skipped>
FAILURES!!!
Tests run: 2, Failures: 1
Your message
to yourself
Add @Test annotation before someTestC, and compile and run it again.
What will be the result?
8
Finally, compile and run this by Ant from mybuild.xml like in Part 11, Slide 14.
Writing a Test Suite
What if we need to run many JUnit test cases?
JUnit, test classes can be grouped into a suite
Just use appropriate JUnit imports and annotations
@SuiteClasses
and
@RunWith(value=Suite.class).
Grouping tests


lets you build several individual test cases for a particular
subsystem , and
write an all-inclusive Test Suite that runs them all .
See AllTests.java below.
Note that Suite of Suites can also be created in the
same way.
9
An example of a Test Suite
C:\Antbook\ch04\test\org\example\antbook\AllTests.java
package org.example.antbook;
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
import org.junit.runners.Suite.SuiteClasses;
@RunWith(value=Suite.class)
@SuiteClasses(value=
{org.example.antbook.junit.SimpleTest.class,
org.example.antbook.junit.setUpTearDownTest.class,
org.eclipseguide.persistence.FilePersistenceServicesTest.class
}
)
End of file
public class AllTests{}
Test Suite to run test methods of three test cases: SimpleTest,
setUpTearDownTest and FilePersistenceServicesTest.
We can also compose suites of suites in the same way.
EXTEND target junit-TestRunner in mybuild.xml to RUN
10
test suite AllTests.java.
Test Suites and Ant??
Don’t need to bother with test suites
when running JUnit tests using Ant:
 Ant itself can list a group of TestCase
classes in a much more powerful
way – all test cases from a directory(!),
and

run it as a batch from the build file itself.
 But <java> task is not
appropriate for this purpose.
11
The JUnit task
<junit>
The JUnit task <junit>
The Ant's task <junit> is much better for running JUnit
test cases than <java> which we used till this point:
<junit> runs one or more (even all from a directory)
JUnit test cases
Collects and displays test results in one or more formats
Provides a way either to fail or continue a build when a
test fails

Recall the unnatural situation when BUILD SCCESSFUL while some
tests failed.
No need to mention explicitly any Test Runner (as it was
necessary with <java> task)
13
The JUnit task <junit> (cont.)
In order to execute some existing test cases by Ant,
 let us declare a special <junit> task with
 the names of the test cases and
 appropriate classpath :
<target name="test-brief" depends="test-compile">
<junit>
<classpath refid="test.classpath"/>
<classpath> with id
test.classpath
formerly used for running
test cases by <java> and
some <javac> tasks
<test name =
"org.eclipseguide.persistence.FilePersistenceServicesTest"/>
<test name="org.example.antbook.junit.SimpleTest"/>
</junit>
One or more test cases to run
</target>
Extend mybuild.xml by this target. You can also add:
<test name="org.example.antbook.junit.setUpTearDownTest"/>14
The JUnit task <junit> (cont.)
TRY it:
H:\Antbook\ch04>ant -f mybuild.xml test-brief
Buildfile: C:\Antbook\ch04\mybuild.xml
[SOME MESSAGES OMITTED]
test-brief:
[junit] Test org.eclipseguide.persistence.
FilePersistenceServicesTest FAILED
BUILD SUCCESSFUL
<junit> TEST FAILED, but BUILD SUCCESSFUL (?!)
It would be more natural to see here also BUILD FAILED...
But let us wait for a more advanced build with <junit>.
Nothing above about org.example.antbook.junit.SimpleTest
because it does not fail.
15
TRY to confirm this by appropriate experiment.
The JUnit task <junit> (cont.)
Test failure enforcing build failure
There are two issues to note about these results:
1. no details were provided about which testXXX()
method in FilePersistenceServicesTest
failed and why,


not so bad – no cluttering messages,
But, on the other hand, insufficiently informative
2. the build completed successfully (?!) despite the
test failure – looks not natural.
However, ADD to the task <junit> two attributes:
(a default value)
<junit haltonfailure="false"
printsummary="true">
We now get more informative and coherent output:
16
The JUnit task <junit> (cont.)
Test failures enforcing build failures
Now it is more informative:
test-brief:
[junit] Running
org.eclipseguide.persistence.FilePersistenceServicesTest
[junit] Tests run: 5, Failures: 2, Errors: 0, Time elapsed: 0.031 sec
[junit] Test org.eclipseguide.persistence.FilePersistenceServicesTest
FAILED
[junit] Running org.example.antbook.junit.SimpleTest
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.016 sec
BUILD SUCCESSFUL
Total time: 1 second
•printsummary="true" prints one-line statistics for each test case.
•Taking haltonfailure="true" gives results only up to the first failure:
test-brief:
[junit] Running
org.eclipseguide.persistence.FilePersistenceServicesTest
[junit] Tests run: 5, Failures: 2, Errors: 0, Time elapsed: 0.031 sec
BUILD FAILED
C:\Antbook\ch04\mybuild.xml:140: Test
org.eclipseguide.persistence.FilePersistenceServicesTest failed
Now BUILD FAILED because of first failing test case(!!)
17
Structure directories to accommodate testing
See also Part 11, slide 13
Recall and extend the directory structure in ch04:
C:\Antbook\ch04
- base directory (basedir= ".")
C:\Antbook\ch04\src
- source directory (${src.dir})
C:\Antbook\ch04\test
- test directory (${src.test.dir})
containing (deeper) JUnit test classes
C:\Antbook\ch04\build
- build directory (${build.dir})
C:\Antbook\ch04\build\classes - for compiled source files
(${build.classes.dir})
C:\Antbook\ch04\build\test
- for compiled JUnit test cases
(${build.test.dir} )
C:\Antbook\ch04\build\data
- for test repots in XML format
C:\Antbook\ch04\build\report - for test repots in HTML format
(new directories data and report to be considered later)
First 3 red coloured directories and their contents are created by yourself.
18
Other directories should be created automatically by your Ant build file.
Four main steps to do in Ant build file
related to JUnit testing
Adding testing into our build process is straightforward:
Add a few targets or tasks:
1. target test-init to initialize (<mkdir>) the
testing directory structure (except directories
created by yourself manually; these are last three
directories on the previous slide),
2. target test-compile to compile the test code with
<javac>,
3. target test (or the like) to execute the tests with
<junit> (or <java>),
4. <junitreport> and <report> tasks to generate
the test reports (to be considered later).
Note that besides testing we need as usually also steps and
targets dealing with source code in src not mentioned
above.
19
Ant build process related to JUnit testing
Some more details
Before above four steps, we should start with assigning
properties to various directories used by our (compile
and) test targets:
<property name="src.dir" value="src"/>
C:\Antbook\ch04\src
<property name="src.test.dir" value="test"/>
C:\Antbook\ch04\test
<property name="build.dir" location="build"/>
More properties...
C:\Antbook\ch04\build
Benefits of presentation of directories by properties:
If you want to change your system of directories, this can
be done only in one place of build file.
20
Ant build process related to JUnit testing
Some more details
Benefits of presentation of directories by properties:
Also, we can easily control e.g. where Ant places test
reports by
 overriding test.reports.dir in command line as
-Dtest.reports.dir=C:\REPORTS
to temporary change the standard (default)
behaviour described in our Ant build file.
Thus, in mybuild.xml you should present all
directories by properties
(except those directories related to package
declarations which you create manually for your source
files and test cases and need not mention in mybuild.xml)!
21
Ant build process related to JUnit testing
Some more details
For compiling and running source files and tests cases we need
classpath which has been already introduced into mybuild.xml in
11. Ant, Testing and JUnit , Slides 14 and 29 (and repeated
now):
<path id="test.classpath">
<pathelement location="${build.test.dir}"/> <!-- build/test -->
<pathelement location="C:\JAVA\junit4.8.2\junit-4.8.2.jar"/>
<pathelement location="${build.classes.dir}"/>
<!-- build/classes -->
</path>
CHECK all targets, tasks, properties and paths in mybuild.xml
required for running test-brief target (and for other future test
targets).
22
Ant build process related to JUnit testing
Some more details
In mybuild.xml, testing can possibly be preceded by
a <copy> task
C:\Antbook\ch04\build\test
<copy todir="${build.test.dir}">
<fileset dir="${src.test.dir}" excludes="**/*.java,**/*.class"/>
</copy>
C:\Antbook\ch04\test
This brings over all non-.java resources:



For example, assume that an html file is required as an
argument of a Java program which should do something with
this file (read it, parse it, search some information in it, etc).
Then we need an example of such file test.html somewhere
under C:\Antbook\ch04\test , and so
test.html, should be copied into the ${build.test.dir},
to allow our compiled JUnit test cases to access test data file
23
– test.html – easily .