Transcript WP4_Framework

WP4 Test and Metrics Plugin
Framework
(WP4)
Eva TAKACS
www.eu-etics.org
ETICS All Hands meeting
Bologna, October 23-25, 2006
Overview
• Why a plugin framework
• Advantages of a Plugin Framework in general
• Core functionality and first steps of the Test and
Metrics Plugin Framework
• Basic architecture of a plugin framework in general
• Java Plugin Framework – implementation of a plugin
architecture
• User scenarios
• Basic architecture of the Test and Metrics Plugin
Framework
• Conclusion
2
Advantages of a plugin
framework in general
Making an application extensible using the
plugin system has strong advantages over
other techniques:
• It allows third-party developers to easily enhance
software in a way that is loosely coupled: only the
plugin API is required to remain stable.
• It allows new plugins to be discovered flexibly.
More precisely,
– plugins can be loaded and saved when a program is first run, or
re-discovered each time the program starts up
– allows the discovery of new plugins installed after the program
has started.
3
Why a Plugin Framework for
ETICS?
The rational behind using „Eclipse style” plug-in architecture is
•
that most of present and/or future ETICS users will with some
tests and testing tools already in place and of course they would
like to use them when working with ETICS
On the other hand
• they should be provided with „default” testing tools when they do
not have their own but would like to use something
4
Core functionality of the Test and
Metrics Plugin Framework
Core functionality of the framework is consisted of
• Existing tools (default plug-ins)
– TestManager – a testdriver used for gLite testing already in use
– Two coverage tools for Java testcases
– Emma
– JavaCov
– other (Cobertura ?)
• A plugin manager system able to handle the plugins and user
profile settings from the web interface and/or command-line
interface
5
Plugin Framework – First steps 1
It was decided that as a first step
– TestManager a testdriver developed in CERN and used for gLite
functional testing
– Two coverage tools
– EMMA, an open source tool requested by future ETICS users
– JavaCov, a commercial tool providing a very strong MC/DC
coverage criterion
will be taken into account.
6
Plugin Framework – First steps 2
To examine how these tree tools can be integrated in a
future plugin environment
• to „modularise” TestManager
• to have a closer look to the running ways of coverage
tools
7
EMMA
EMMA – is an open source coverage tool
• Supported coverage types
–
–
–
–
class
method
line
basic block (branch)
• Supported Java version: 1.5.*
• Running way:
– configurable command line
– ANT integration
– no Eclipse plugin
8
EMMA - JUnit
EMMA – Junit integration
• EMMA does not support implicitely "JUnit integration„
• and not assume any particular testing methodology
• But after doing some configuration can be used for
measuring the coverage of Junit test cases
9
JavaCov
• JavaCov
• Supported coverage types
–
–
–
–
–
class
method
line
basic block (branch)
MC/DC (refined version of branch, required in safety-critical
systems)
• Supported Java version: 1.4.* ?????????
• Running way:
– ANT integration
– Eclipse plugin
– no separate command line
10
JavaCov - JUnit
• JavaCov – JUnit integration
• supports implicitely "JUnit integration„ without extra
configuration
• at the same time does not assume any particular test
method
11
JavaCov ? Cobertura
Because of not supporting java 1.5 and the company is
not going to upgrade JavaCov this year it can be
replaced by an other coverage tool.
One candidate can be Cobertura – an other open source
product but without supporting MC/DC criterion.
12
Terms refering to a Plugin
Architecture
Basic terms:
• a runtime engine that dynamically discovers and loads "plug-ins„
• A plug-in is a structured component that contributes code and
resources to the „core” system and describes itself to runtime engine
using a "manifest file„ usually an xml.
• Runtime engine maintains a registry of available plug-ins and the
functions they provide (via extension points and extensions)
• extension points, well-defined methods that can be extended by other
plug-ins.
• extension to the system – when an implementatin is added to an
extension poit
13
Requirements
• Important requirement of a plugin framework
• Because of memory and performance issues the installed plug-ins
should not be loaded (instantiated) until they are called. They are
added to a central registry at application start-up and are loaded when
necessary.
14
Basic Plug-in architecture
Manifest1.xml
extension
point
Plugin Manager
- loads
and activates plugins
- maintains a plug-in registry
- finds the physical location of
plugins
API for defining
Manifest files
Plugin_1
•Plugin_11
Manifest2.xml
Plugin_2
Manifestn.xml
Plugin_n
15
Java Plugin Framework
JPF (Java Plugin Framework) is a general-purpose open source plug-in
framework intended to build extendable Java applications
Main features:
• Open framework architecture but having a „default” implementation
• Lazy plug-in activation (Plug-in classes are loaded into memory only
when they are actually needed )
• "On the fly" plug-in registration and activation
• Xml based manifest files (Eclipse now uses OSGI)
16
OSGI
The OSGi framework specification forms the basis of the
Eclipse Runtime. As of Eclipse 3.0, the Runtime is fully
based on the OSGi notion of bundle (equivalent to
Eclipse plug-ins). The OSGi framework implementation
in Eclipse 3.0 is fully compliant with the OSGi
framework specification R3.0. The OSGi
implementation in Eclipse 3.1.2 and the upcoming
Eclipse 3.2 fully implement the OSGi R4.0 framework
specification.
17
User scenario in ETICS 1
Use case: the user specifies in his/her profile that the
required coverage tool during the build is EMMA
• Test and Metrics Plug-in Framework’s runtime engine
discovers dynamically and loads the „EMMA plug-in„
(For example scans a directory named „plugins” looking for
EMMA)
• In „plugins” directory the engine discovers „emma.jar”
• When the build process reaches the „coverage” target the
‘emma.jar’ will be run using the configuration described in
the manifest (xml) file of the EMMA plugin
18
User scenario in ETICS 2
Use case : ETICS user wants to use TestManager to run gLite
testcases outside the build process but using his/her own
charting library to create reports
• In his profile (or somehow through the user interface)
or from the user interface of the Plugin Framework he
can chose TestManager between the available test
drivers
• The plugin manager observes that TestManager should
be loaded
• Also observes that when executing the reporting
module in TestManager an extension of it e.g the
charting system of the user should be loaded
19
User scenario in ETICS 3
Use case: ETICS user (manager) wants to know how many
open bugs were fixed during a specific build
• In his profile (or somehow through the user interface)
he specifies the required metric
• The Plug-in Frameworks runtime engine discovers the
required metric calculation utility
• When the build process reaches the target „metrics”
the selected metric calculation utility will be run.
20
Test and Metrics Plug-in
Architecture 1
Manifest1.xml
TestManager
Plugin Manager
Manifest2.xml
- loads
and activates plugins
- maintains a plug-in registry
- finds the physical location of plugins
API for defining
Manifest files
EMMA
Manifestn.xml
JavaCov/
Cobertura
JPF
21
Test and Metrics Plug-in
Architecture 2
Manifest1.xml
Plugin Manager
- loads and activates plugins
- maintains a plug-in registry
- finds the physical location of plugins
API for defining
Manifest files
TestManager
extension point
Charting
system
Manifest2.xml
EMMA
Manifestn.xml
JavaCov/
Cobertura
Java Plugin Framework
Build system - (build.xml)
22
Conclusion
2006
• Specification on conceptual level
• Feasibility study
• Architectural specification
• Integration of the plugin system in the ETICS
framework
2007
• Implementation
23