Transcript Testing Cactus and JUnit By Bill Dudney and Jonathan Lehr

Testing Cactus and JUnit
By Bill
Dudney and Jonathan Lehr
Neal O’Brien 10/27/04
Table of Contents








Overview
Article Outline
Thesis
JUnit Background
Cactus Background
Pitfalls 1-6
Comparable Products
Conclusion
Overview






Junit and Cactus are popular tools for automating
testing of Java classes and web-based components
Automation accomplished using ANT (Java Make
utility) and often tested every time project built
These tools enable testers to “verify” the code is
written correctly – it was built right
Automated tests are very useful to show code works
correctly esp. after refactoring which is central to XP
I picked this article because deals with JUnit &
Cactus, offers a critique of the tools, and was
practical, not just theoretical
As this article describes, the tests themselves must be
built right in order to validate the code being tested
Article Outline







Theme: Pitfalls when testing with JUnit & Cactus
Pitfall 1: No assert
Pitfall 2: Unreasonable assert
Pitfall 3: Console-Based Testing
Pitfall 4: Unfocused Test Method
Pitfall 5: Failure To Isolate Each Test
Pitfall 6: Failure to Isolate Subject
Thesis – Analysis of article






Thesis: critique was correct but wordy, repetive, and
incomplete – missing some bigger pitfalls
Pitfalls 1-3 are all the same – use assert correctly
Pitfall 4 is programming rule – write focused method
Pitfall 5 is JUnit rule – Use setUp and tearDown
Pitfall 6 is incomplete analysis of Cactus vs MockObj
Overview of Junit and Cactus, then discuss pitfalls,
and finally look at unmentioned pitfalls & issues
JUnit Overview








Popular and simple Java framework / library for
automating testing
Integrates well with ANT – Java Make utility
General idea: write one test class per testee
Write one method to verify each main feature
Test class must extend TestCase and each test
method must start with “test”
Order of test method execution varies
Use assertTrue() and assertEquals() to verify code
Use setUp() & tearDown() prepare testcase testfixture
JUnit code example

Following test cases test the collection methods, isEmpty() and add()

import junit.framework.*;

public class SimpleTest extends TestCase {

private java.uti.Collection collection;

protected void setUp() { collection = new ArrayList(); } // instantiates collection test fixture

protected void tearDown() { collection.clear(); }

public void testEmptyCollection() {

assertTrue(collection.isEmpty());

}

public void testOneItemCollection() {

collection.add("itemA");

assertEquals(1, collection.size());
}


}
Cactus Overview





Built on Junit framework
Intended to test JSP, Servlets, EJBs, Filters, and
custom tags
Complex architecture that has client JVM call the
J2EE application server JVM via redirector
Testcase classes must reside on client and server
Adds two methods to Junit architecture, beginXX()
and endXX() which get called on client, rest on server
Cactus System Diagram
Cactus Sequence Diagram – Jakarta website
Pitfalls 1-3 – No assert, Unreasonable
assert, Console-Based Testing


Pitfalls 1-3 are all the same – use assert correctly
Very important and author points out a major guideline
–
–



test what is written in the javadocs for the testee
implies javadocs must be up-to-date with requirements
Author also points out to write a test case if encounter
a defect before it is corrected
However, using assertTrue() and assertEquals() is
obvious
These are the prominent features of the JUnit
Pitfall 4 – Unfocused Test Methods



Writing focused tests is really just writing good code
General rule of programming to make methods
succinct, this applies equally to test methods
Writing focused test methods is the whole point
Pitfall 5 – Failure To Isolate Each Test

Pitfall 5 is really saying to use setUp() and tearDown() to prepare/release test
fixture, an obvious suggestion – example from JUnit site

Bigger pitfall is automating creation of the test fixture in distributed environments

import junit.framework.*;

public class SimpleTest extends TestCase {

private java.uti.Collection collection;

protected void setUp() { collection = new ArrayList(); } // instantiates collection test fixture for 2 tests

protected void tearDown() { collection.clear(); }

public void testEmptyCollection() {
assertTrue(collection.isEmpty());


}

public void testOneItemCollection() {

collection.add("itemA");

assertEquals(1, collection.size());
}


}
Pitfall 6 – Failure to Isolate Subject


Author points out one drawback of Cactus is that does
not isolate test case as MockObjects does
MockObjects simulates the Servlet container
–
–

Mock Object framework does isolate test but at big expense
Massive amount of stubs needed, more code to maintain
While Cactus may be better than MockObjects, it may
NOT be better than HttpUnit, why not compare these?
My pitfalls






Does not mention major issues like creating this
complex automated testing API is itself a software
project and has high upfront resource commitments
It will slow the development process
Upkeep of test code is costly
Tests must be documented, reside in source control,
and be included in the project planning
Junit testcases must reside in same package or
parrallel package to access private/package
methods – complicating code structure
Testers often analysts, not programmers
My Pitfalls – continued


Cactus can only test J2EE web objects - JSP/EJB/etc,
cannot test ASP.NET, PHP, etc.
Goal of Cactus not clear – Web or Unit tests?
–

Cactus is unnecessarily complex
–
–

JUnit is better for unit testing business objects
mucky interface: test code must reside on two JVMs
Should call via web server like user does, else white box
** Many business websites may have distributed
nature of data/objects to be tested, it may be very
difficult to setUp the test cases
–
Possibly data to be tested is on server not owned/managed
by the testing team, data could be volatile
Alternative tools


HttpUnit cleaner, simpler tool than Cactus
HttpUnit is black box testing by calling webserver
–





Test code resides ONLY on client JVM
Various interfaces like JWebUnit (Java API) and
WebTest (XML) integrate well with ANT
Use Junit for unit tests and HttpUnit for functional
Features to analyze HTML, ie. table element tests
Features to input HTML form elements
http://www.junit.org/news/extension/index.htm
JWebUnit example –
JWebUnit.sourceforge.net

// The tests perform a google search for the HttpUnit home page, navigate to that page from Google,

// and validate that there is a link to the user manual on the HttpUnit home page.


package net.sourceforge.jwebunit.sample;

import net.sourceforge.jwebunit.WebTestCase;

public class JWebUnitSearchExample extends WebTestCase {

public JWebUnitSearchExample(String name) {

public void setUp() {

public void testSearch() {
super(name); }
getTestContext().setBaseUrl("http://www.google.com"); }

beginAt("/");

setFormElement("q", "httpunit");

submit("btnG");

clickLinkWithText("HttpUnit");

assertTitleEquals("HttpUnit");

assertLinkPresentWithText("User's Manual");
}


}
Webtest – XML interface to HttpUnit, easy to
automate test by inlining with ANT, verifies logon


This XML file can easily be called from ANT
Generic interfaces like XMLSPY can be used by non
programmers to create the XML test files
<target name="login" >
<testSpec name="normal" >
<steps>
<invoke
stepid="get Login Page"
url="login.jsp" />
<verifytitle
stepid="we should see the login title" text="Login Page" />
<setinputfield stepid="set user name" name="username“ value="scott" />
<setinputfield stepid="set password" name="password" value="tiger" />
<clickbutton stepid="Click the submit button" label="let me in" />
<verifytitle
stepid="Home Page follows if login ok" text="Home Page" />
</steps>
</testSpec>
</target>
Conclusion

Article lists some useful guidelines & pitfalls in an wordy fashion

Many pitfalls were obvious and important ones not mentioned
Important pitfalls not mentioned include

–
–
–
–
–
–



Cost, complexity, difficulty of distributed tests not mentioned
Performs white box tests, yet, JUnit already does this
Does not test HTTP interface (tests presentation layer poorly)
Test code must reside in same package as testee & both JVMs
Testers must be programmers
JWebUnit & WebTest better for web unit testing
At times unclear when addressing Junit vs. Cactus and unnecessarily
complex coding examples
However, automating testing can save time and money in the long run
These tools, while not perfect, are major players for automated Java
testing and can verify functionality during development and refactoring