Transcript A Novel Approach to Unit Test: The Aspect

A Novel Approach to Unit Test: The
Aspect-Oriented Way
Guoqing Xu and Zongyuan Yang
Software Engineering Lab (SEL)
East China Normal University
http://www.cs.ecnu.edu.cn/sel/~harryxu
Oct.21 2004 at ISFST
Background
Test Oracle problem
-- how to identify oracles in unit test?
 Automating the test oracle generation
-- Manually writing test codes is a labor-intensive job.
-- Cheon and Leavens automatically generate JUnit test
codes and use JML runtime assertions as test oracle to
test the Java programs annotated with JML
specifications [CL02].
 Automating the test case generation
-- Korat generates test cases based on java predicts.
[BKM02]
-- JMLAutoTest generates test cases based on JML class
invariants specified in the program. [XY03]

Problems



Current test automating process relies on formal
assertions and predicts.
Traditional specifications only focus on
programs’ functional behaviors, with little
support for specifying their non-functional
behaviors, e.g. temporal logic, performance...
How to find oracles to test non-functional
aspects? How to automate this process?
Our approach


In Aspect-Oriented Programming (AOP),
crosscutting properties are monitored to model
the program from different aspects, a lot of
tasks which have been difficult to be handled in
traditional ways are easily done.
Using a crosscutting property of the program as
the criterion to check the correctness of the
application in the corresponding aspect is well
suited to the unit testing problems.
Application-specific Aspect

A new notion: application-specific aspects (ASS)
-- top-level application related aspects.
-- established at the design level.
-- may be picked up from low language level
aspects.
-- all the ASS for the same use share some
common features, e.g. testing ASS, tracing ASS...
-- may be translated into language level aspects.
Aspect-Oriented Test Description
Language
How to describe ASS?
-- a formal way is needed.
-- can not be too complicated.
 Aspect-Oriented Test Description Language(AOTDL)
-- uses the first order predict logic as part of its syntax
-- used by the designer at design level
-- can be translated into AspectJ aspects.
 Basic units in AOTDL
-- Utility unit
-- MeaninglessCase Advice unit
-- Error Advice unit
advicetype (arguments): pointcuts: conditions: message

AOTDL (cond.)
TestingAspect tempLogic{
Class Stack{
public void
init(){...}
public void push
( Node n){...}
...
}
Utility{
protected boolean isInitialized = false;
//push is reached
pointcut pushReached(Stack st):
target(st)&&call(void Stack.push(Integer));
//init is reached
pointcut initReached(Stack st):
target(st)&&call(void Stack.init(void));
// after advice
after(Stack st):initReached (st){
isInitialized = true;
}
}
AOTDL (cond.)
MeaninglessCase
Advice{
/*advices for specifying
criteria of meaningless
test cases */
before(Stack s) :
pushReached(s) :
s.getSize()
>=MAX : ”Overflow”;
...
}
Error Advice{
/*advices for specifying
criteria of test errors */
before(Stack s):
pushReached(s):
! isInitialized : ”Not
Initialized” ;
...
}
}
AOTDL (cond.)
//error advices
before(Stack s) throws TestErrorException:
pushReached(s){
if (!isInitialized){
TestErrorException ex =new
TestErrorException(“Not Initialized”);
ex.setSource(“TempLogic”);
throw ex;
}
}
JAOUT: Automated Generation of
AO Testing Framework
AOTDL
Testing
Aspects
Jaout/
Tranlator
translate
Aspects
AspectJ
AspectJ
in
Java
Java
programs
Generate
Arj
Weave and
Compile
Bytecode
files
Jaout/
Generator
Junit test
classes
Junit test
runner
JMLAutoTest
Test Case Supply
Run the test
Test
Results
Fig.1. An Overview of the basic technique
JAOUT: Automated Generation of
AO Testing Framework

JAOUT takes Java class M.java as the input,
and automatically generate JUnit test
framework.
-- Aspect_M_Test.java, the JUnit unit test
class.
-- Aspect_M_TestCase.java, the test case
provider.
-- Aspect_M_TestClient.java, JMLAutoTest
test case generation class.
Test Suite Definition
For class C and its method M(A1 a1, A2 a2…An
an), the generated test suite is defined as
--- C[ ] receivers
-- A1[ ] vA1; ... ; An[ ] vAn;
 There is a corresponding init_Ai method for
each type Ai and a method init_receiver in test
case provider to initialize test cases.
 Testers use APIs provided by JMLAutoTest to
generate test case space in test client, and pass it
to the test case provider.

Generated Test Method
public void testM(){
catch (TestErrorException e) {
for (int i0 = 0; io < receiver.length; i0++){
String msg = e.getSource();
fail(msg + NEW LINE +
e.getMessage());
} catch (java.lang.Throwable e) {
continue;
} finally {
setUp(); // restore test cases
}
for (int i1 = 0; i1 < a1.length; i1++){
…
try {
receiver[i0].M(a1[i1], : : :, an[in]);
} catch (MeaninglessCaseException e)
{
/* ... tell framework test case was
meaningless ... */
continue;
}
}
}
Test Result



















...in push
false
F
Time: 0.06
There was 1 failure:
1) testPush (sample.Stack_Aspect_TestCase)junit.framework.AssertionFailedError: In Testing
Aspect TempLogic: Not Initilized!
at ample.Stack_Aspect_Test.testPush
(Stack_Aspect_Test.java:155)
at sun.reflect.NativeMethodAccessorImpl.invoke0
(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at sample.Stack_Aspect_Test.run(Stack_Aspect_Test.java:26)
at sample.Stack_Aspect_TestCase.main
(Stack_Aspect_TestCase.java:24)
FAILURES!!!
Tests run: 3, Failures: 1, Errors: 0, Meaningless:0
Related Work (Spec-based test)





TestEra – Automating OO test generation. [MK01]
MIT
JMLUnit – Generating test oracles from JML runtime
assertions. [CL02] Iowa State Univ.
Korat – Generating test case based on Java predicts.
[BKM02] MIT.
JMLAutoTest – Generating test framework from JML
runtime assertions and test cases based on class
invariants. [XY03] ECNU.
Jov -- java automated unit test based on inferred
program properties. [XN03] Univ. of Washington.
Conclusions





Traditional formal predicts tend not to deal with
non-functional properties of the program.
AOP is well suited to the unit test problems.
Designers use AOTDL to build ApplicationSpecific Testing Aspects.
JAOUT translates Testing Aspects to AspectJ
aspects automatically.
JAOUT automatically generates JUnit test
framework and uses the runtime messages
thrown by Testing Aspects as test oracles.
Thank you… Questions?