Transcript JAIDA, AIDA-JNI, JAS3 Status and Plans

JAS3 – Current Status and
Prospects
by Victor Serbo, SLAC
Contents
 Introduction to JAS3 (Java Analysis Studio)
 Basic: Opening Files, Plots, Printing, etc.
 Web Browser, Editor, Java Compiler, Scripting
 Miscellaneous: Spreadsheet, Configuration Save/Restore,
Preferences, etc.
 Analysis
 AIDA – Abstract Interfaces for Data Analysis
 JAIDA – Java Implementation of AIDA
 AIDA JNI – using JAIDA from C++
 Analysis with JAIDA and JAS3
 JAS3 Plugins
 What is Plugin, examples
 Demo: AmbientPlugin and Geant4 Plugin
 Demo: JAIDA Web Interface
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Overview
 A completely new version of JAS
 Design based on Application Shell, into which many (optional)
modules can be plugged
Highly customizable for different application domains
– HEP/Astrophysics/Other
– Data analysis/Online Monitoring/GRID analysis
– Experiment/User specific modules
Modules can be updated independently of shell
– Possible to release bug fixes fast
 Includes support for programming in many languages
Scripting: Pnuts, Python, ….
– Command prompt
Java (compiled)
 Analysis (histograms, tuples, fitting) based on AIDA standard
 Not technically backwards compatible with JAS2
 But migration is straightforward.
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Lightning Tour
 Tour designed to
give you an
overview of the
capabilities of JAS3,
you can try them
out for yourself:
http://jas.freehep.org/jas3
Welcome Page, gives initial
info and links to example
scripts and programs
JAS3 – Current Status and Prospects, V. Serbo
Memory
monitor
ACAT03
1-5 December 2003
Opening Files
Use file
menu
Drag from
explorer
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Plot Manipulation and Printing
Or
copy/paste
into Word,
PowerPoint
etc.
Can send
individual
plots or full
page
direct to
printer
Or save as PS,
EPS, PDF, SWF,
SVG, PNG, GIF…
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Java Editor, Compiler and Loader
Tree
shows
loaded
programs
Built-in Java
compiler
JAS3 – Current Status and Prospects, V. Serbo
Built-in
editor for
writing
analysis
code
JAS3 allows any Java program to be
loaded. This example “main routine” is
taken
directly from the AIDA manual
ACAT03 1-5 December 2003
Miscellaneous Features
Save/Restore
configuration
User
Preferences
Plugin
Manager
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Spreadsheet
 Simple spreadsheet plugin
 for
 Displaying results
 Calculations
 Simple Plots
 Supports reading/writing
 .csv files
 Excel files
 Cut/Paste with Excel etc
 Coming Soon…
 Scripting interface
 GUI for building plots
 User defined functions
– Java, scripting
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Scripting
 JAS3 has multi-language OO scripting support
 Command line, Console, Editor
 Major components (e.g. AIDA) have scripting interfaces
 Currently have plugins to support
 Pnuts – syntax almost identical to Java, fast, well documented
and feature complete
 Python
Use Jython (the Java implementation of Python)
Limitation: can not call Python libraries written in C
 More scripting languages can be added
not restricted to Java implementations (e.g. could use C-Python, JPE)
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Pnuts Language
 Currently support Pnuts scripting language
 Complete and well documented
http://javacenter.sun.co.jp/pnuts/doc/guide.html
 Fast (although not as fast as compiled Java)
 Syntax very similar to Java; can compile script into Java
bytecode or convert it to Java code
 Can easily call compiled Java classes from scripts – best of both
worlds
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Scripting II
Can also
write and run
scripts
Console allows
direct interaction
with scripting
language
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
AIDA - Abstract Interfaces for Data Analysis
 AIDA is a standard set of interfaces for creating and manipulating
histograms, n-tuples and related data analysis objects, like Plotter,
Fitter, etc.
 AIDA defines storage XML format.
 It has been created cooperatively by a group of developers working
on high-energy physics data analysis tools. The goal of the AIDA
project is to provide the user with a powerful set of interfaces which
can be used regardless of which analysis tool they are using.
 Only define “protocol” for analysis objects, no internal details
 AIDA defines behavior, analysis tool provides implementation
 C++ and Java versions of interfaces are generated from the AID
files.
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
AIDA Examples
 IHistogram1D
-
IAxis axis()
void fill(double x)
void fill(double x, double w)
double mean()
double rms()
double binEntries(int index)
double binError(int index)
….
 IAxis
-
int bins()
double lowerEdge()
double upperEdge()
double binLowerEdge(int index)
double binUpperEdge(int index)
double binWidth(int index)
int coordinateToIndex(double x)
….
 For more details on AIDA,
see: http://aida.freehep.org
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAIDA
 JAIDA = Java implementation of AIDA
 Complete implementation of AIDA interfaces
With a few exceptions
– Support for plot styles is somewhat limited
– Some missing features in ITree, like cp, mv
 Uses Java “service provider mechanism” for dynamic loading of
services at Runtime
Can be used to add support for new fitters, stores etc.
Have defined “developer level” interfaces for IOptimizer, IStore etc
 Uses Java Expression Library (JEL) for runtime compilation of
expressions (for tuple evaluators and functions)
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAIDA – Features continued…
 Fitting




support for multiple fitting engines (optimizers)


Support for multiple fitting methods

Stores







LeastSquares, Chi2, BinnedMaximumLikelihood and UnbinnedMaximumLikelihood
Read/Write support for AIDA XML format (with built-in gzip)
Read support for HBOOK/PAW files (needs hbookAdapter native library)
Read support for Root files (write support hopefully coming soon)
Future plans to add:

Plotting

UNCMIN (pure Java)
Minuit (needs minuitAdapter native library)
SQL interface, binary store format, text file support …
Two implementations of IAnalysisFactory


BatchAnalysisFactory – dummy plotter, ignore calls to IPlotter
AnalysisFactory – Full plotter functionality
Multi-Threaded – plots update in real-time as they are filled

No need to call interact() or refresh()
Support for embedding plotter


In other Java GUI’s
In Java servlets for web based applications.
Graphics export in many formats including:

PDF, EPS, SVG, SWF, PNG, GIF, JPG
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
AIDA-JNI
 C++ interface uses AIDA-JNI
AIDA- JNI
AIDA
AIDA
C++ program
JAIDA (Java)
 AIDA-JNI partially generated by AID, partly by hand
 Does support full AIDA functionality
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Using JAIDA with Java
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Using JAIDA with Python
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Using JAIDA with C++
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Future Plans for JAIDA
 Complete implementation of missing features
 Continue to track improvements to AIDA standard
 Add support for function arithmetic
 Web Interface
 Support for “Remote AIDA”
 Support for more storage formats
 Need good binary IO format .. either AIDA standard or roll our own
 Future AIDA developers’ meetings
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 and AIDA
 JAS3 uses JAIDA - supports all AIDA functionality, including
 Histograms (includes arithmetic, projections, etc.)
 Clouds (unbinned histograms, scatterplots)
 Plotter
 Tuples
 Fitting – AIDA interfaces allow for multiple fitters
 Uncmin -- pure java minimizer
 Minuit -- Fortran called by Java Native Interface (JNI)
 IO
 AIDA XML, PAW, Root
 JAS3 supports user interaction with AIDA in three ways
 Scripting (Pnuts, Python etc)
 Compiled (Java) code
 GUI – Plotting, Fitting, Cuts etc.
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Graphical Interface to AIDA
Histograms,
Clouds, Tuples
all presented in
AIDA tree
.aida files,
.hbook files,
.root files all
presented as
AIDA objects
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
Drag items onto
page, or use
1-5 December 2003
(popup) menus
Tuple Explorer - Plots
Histogram
Works with any tuple,
read from file or
dynamically created
ScatterPlot
XY Data
(More appropriate for
smaller data sets)
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Profile
Tuple Explorer – Define Columns
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Tuple Explorer - Cuts
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Tuple Explorer - Tabulate
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Tuple Explorer – Record Source
To be used with record loop
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Record Sources
Opening record
(or event)
based files
causes the run
control toolbar
to appear
JAS3 – Current Status and Prospects, V. Serbo
Supports random access
and “tagged” data sets
(mainly for event
displays)
ACAT03
1-5 December 2003
JAS3 Plugins
 JAS3 is based on the Freehep Application Framework, which
provides a basic application framework into which extension modules,
or "plugins" can be loaded.
 The application framework provides general purpose functionality,
such as top level GUI (windows, menus etc.), help system, storing
and retrieving user preferences etc.
 All of the data analysis specific functionality of JAS3 is provided by a
set of extension modules.
 Modules communicate with each other via service interfaces and
notification events. JAS3 comes with a standard set of service
interfaces and notification events, but new plugins can also define
their own interfaces and events.
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Plugin II
 Plugin functionality:
 Do some plugin-specific task
 Use services provided by other plugins
 Provide services to other plugins
 Modify JAS3 menus and pop-up menus
 “Core” Plugins – provide basic services
 Simple: File Opener, Memory Status, Status Bar, …
 More Complex: Web Browser, Plotter, Console, XMLIO, …
 “Extensions” – separate packages
 Included with JAS3 distribution
 AIDA, TupleExplorer, JAVA Compiler and Editor, Spreadsheet, …
 Specific to particular Experiment
 BaBar Ambient DB Browser: AmbientPlugin, Geant4 Plugin, etc.
 Partial Tutorial on writing a plugin:
http://jas.freehep.org/jas3/developers/tutorial.jsp
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Plugin Demo I
 AmbientPlugin:
 Access BaBar Ambient DB via CORBA
 Present Time History as an AIDA object (DataPointSet)
CORBA
Server
(C++)
BaBar
Ambient DB
JAS3
(Java)
AmbientPlugin
(Java CORBA
Client)
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Plugin Demo II
 Gean4 Plugin
Geant4
Application
(C++)
Java RMI
Server
JNI Layer
JAS3
(Java)
Geant4 Plugin
(Java RMI
Client)
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
AIDA Web Interface
Goals
 Provide access to Histogram plots without requiring any
software installation
 Putting results on to the web for collaborators etc.
For example for viewing results of “quality control” plots
 Viewing “live” histograms on the web for
Online monitoring
Monitoring progress of batch jobs
 Add as much functionality as possible in a “generic” way
so that it can be used by others (with minimal work)
 Complete “out-of-the-box” web application
Packaged as a .war file for use with any J2EE app server
– (e.g. Tomcat)
 Configurable via a simple .ini file (or better via a web interface)
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Architecture
.root
Application Server
(Tomcat)
Web Application
.paw
JAIDA
AIDA
Servlet
.aida
SQL
…
Remote
AIDA
C++ App
Java App
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Features
 Current Feature set
 Display of “AIDA Tree” for selection of histograms
 Displays plots as .png or .gif images in browser
Allows (limited) control of image properties
 Allows (limited) control over plot properties
 Allows comparison to reference plots (including Kolmagorov test)
 Allows export of plots to PDF, EPS, SVG, SWF, GIF, PNG, etc.
 Planned features
 Extend to support n-tuples as well as histograms
Dynamic definition of new columns and cuts (evaluators, filters)
 Web based fitting (including function definition)
 Support multiple plots per page
Thumbnail browser
 Extend which plot options can be configured through web
 Allow users to retain settings via cookies
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Current Interface
Can try from the web:
http://glast02.slac.stanford.edu:8080/
aida/welcome.html
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
Status
 Currently released JAS3 version 0.7.3 (0.7.4 next week)
 AIDA functionality is quite solid
 Compiler, Loader, Record Loop all quite recently added,
Certainly still some rough edges
 Documentation limited but available
 Built-in example scripts and programs
 Tutorial on web
 If you used JAS2 you will find some functionality not yet
ported to JAS3
 Remote (client/server) access to data.
 3D Lego/Surface plots
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Conclusion
 JAS3 is a powerful and flexible Analysis Tool
 JAS3 uses Plugin Architecture
 Easy to add new functionality
 Easy to fix problems with existing modules – do not have to wait
for new release of all libraries
 JAS3 release 0.7.3 is out – try it!
 It is still a beta version, so bugs happen
 We need your input to make JAS3 better!
Report bugs, problems, suggestions at http://bugs.freehep.org
Do not have to be on JAS team to be JAS developer – ideas/code for
general-purpose plugins are always welcome
 Try writing JAS3 plugins for your own needs: for Experiment and
Home 
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003
JAS3 Links, More Info
 Report Issues – http://bugs.freehep.org/
 JAS – Java Analysis Studio - http://jas.freehep.org
 JAS3 – http://jas.freehep.org/jas3
 JAIDA – http://java.freehep.org/jaida/
 AIDA – http://aida.freehep.org
 FreeHEP - http://www.freehep.org
 FreeHEP Java Libraries - http://java.freehep.org
 WIRED – http://wired.freehep.org
JAS3 – Current Status and Prospects, V. Serbo
ACAT03
1-5 December 2003