Transcript Java Root IO

Java Root IO
Part of the FreeHEP Java
Library
Tony Johnson ([email protected])
Mark Dönszelmann ([email protected])
Victor Serbo ([email protected])
Max Turri ([email protected])
Presented at Computing in High Energy Physics
Interlaken, Switzerland, September 2004
Contents


Goals
Java IO library for Root
–
–
–
–
–
–

Implementation
Test Suite
Benchmarks
Networked Root IO (rootd/xrootd)
Demo tools
Root Plugin for AIDA
Examples of Use
– AIDATLD Web Interface
– JAS3
– WIRED4


Future plans
Links for more information
Java Root IO: Goals

Goals
– Pure Java package for reading Root Files
 Will
be extended to writing later
– High Performance
 We
want to do data analysis so we need very good
performance
– Should not need to know about objects ahead
of time
(no need for dll’s, .so files etc.)
 But
should provide full support for reading userdefined objects
– Easy to use
– Provide access to Root data, not C++ code
Implementation



Use “TStreamerInfo” objects inside Root files
(since Root 3.0) to decode contents of file
Dynamically generate Java proxies for each Root
object read
Each proxy has custom “streamer”
– Build as Java bytecode using BCEL
(http://jakarta.apache.org/bcel)
– Converted at runtime to machine code by HotSpot VM

Uses java.nio package (new since Java 1.4)
– Standard Java package for high-performance binary IO
Implementation Details
Java Interface
Accessor method for each data member
Implements
TStreamerInfo
Java Proxy
Custom version based on
how object is stored in file
Interface
Generator
.java file
Example of Use
Test Suite


Most classes are dynamically generated from info
in Root file, so are fairly robust against changes
Need bootstrap process, in particularly need
enough custom Java classes to read the streamer
info, and interpret it
– Relatively few classes, but



This tends to change from one Root release to another
Typically have to make small changes in the bootstrap code
for each new Root release
We have test suite that creates a set of test files
from each new root release, and verifies that our
library can read the files.
– Currently test 3.00.06, 3.01.06, 3.02.07, 3.03.07,
3.05.07, 3.10.02, 4.00.08

In addition we have a set of Root files collected
from users, and the test suite verifies that we can
(continue) to read these files with each release.
Java Root IO - Benchmark





Task: Loop over all events, loop over all tracks, sum
px for all tracks. 5000 event files created with “event”
test program distributed with Root (200-400 MByte).
1.4GHz AMD Athlon, Windows
XP, 512MByte DDR, ATA100
disk..
S= Split level = 0,1,2
C= Compression = 0,1
Each test run 5 times in
succession, lines show range in
results
– in later runs file is typically
cached in memory == faster if
task IO limited
C
S ms/Event
0
0 12.3 - 4.23
0
1 10.7 -- 2.75
0
2 1.70 -- 0.20
1
0 10.5 -- 5.12
1
1 9.6 -- 4.77
1
2 2.7 -- 0.25
Network Root IO (x)rootd

Recently added ability for Java library to
talk directly to rootd or xrootd server
– Defined “root:” protocol handler for URL class.
– Defined Authenticator for authenticating to
rootd
– Extended RootFileReader to accept (any) URL
argument to constructor
 If
URL corresponds to file: read the file directly
 If URL corresponds to root: protocol use xrootd/rootd
(random access)
 If URL corresponds to http: and web server supports
http 1.1 “ranges”, use extended http
 Otherwise buffer URL contents into memory and read
from there (suitable for small files only).
Authentification Issues
Authentification can be set programmatically, or via
GUI Authenticator
Rootd supports many authentification schemes


1.
2.
3.
4.
5.
6.
7.
•
•
•
Anonymous (c.f. anonymous ftp)
User/Password (clear text)
User/Password (RSA encrypted)
SCP
SSH like
GSI certificate
UID/GID
Currently FreeHep library supports only 1 and 2
Work underway on 3 (but some compatibility problems
between Root and Java RSA implementations)
Java libraries exist for all other methods, so can be
supported in future.
Network Root IO: Example of Use
Demo Apps: Root Object Browser
Demo Apps: Root Histogram
Browser
Root Plug-in for JAIDA


JAIDA = Java implementation of AIDA
AIDA = Abstract Interfaces for Data Analysis =
Histograms + Tuples + Fitting + Plotting + IO data
analysis package
– (See Monday’s talk by Victor Serbo)

Root plugin for JAIDA makes it possible to read root
files via the AIDA interfaces and treat the objects
contained in the root file as AIDA objects
– Useful if:




You
You
You
You
like AIDA interfaces better than root interfaces
want to use language not supported by Root
want to analyze data in mixed formats
want to work in J2EE, J2ME, Applet etc.
Root Plug-in for JAIDA: Example
Root Java IO:
Examples of Use
AIDATLD Web Interface
JAS3
WIRED4
AIDATLD + Root

AIDATLD = AIDA Tag
Library for use in JSP
pages
– Trivial to put “live”
histograms on web
– Directly supports Root
files
AIDATLD Example: GLAST System
Tests Web Interface
JAS3 + Root
JAS3 = Modular, Extensible, Data Format
Independent data analysis system.
 Among many other formats JAS3 supports
Root

– Contents of Root files can be accessed directly,
or via Root->AIDA interface
 Full
access to AIDA functionality for fitting,
projections, plotting etc.
– Can operate on Root data
 using
GUI
 using scripting (Python, Pnuts)
 using compiled Java code
JAS3 Access to Root Files
JAS3 + Root: Scripting
Root + WIRED4

Build yourself a super cool event display for your custom
Root file in under 1 hour!
Future Plans

Continue to track new Root releases
– Root network protocol
 More
work on authentification
– Root file writing
 Most
of work already done by Petr Vokac/Julius
Hrivnac
 Easier than reading because only need to support one
root protocol version

Support Root XML format
– Will be much slower, but perhaps more robust

Use LCGDICT system to allow Java access
to C++ methods as well as data?
– Probably also slower, but more flexible
More Info

Freehep Java Library
– http://java.freehep.org/

Root IO component of library
– http://java.freehep.org/lib/freehep/doc/root/

AIDATLD
– http://aidatld.freehep.org/

JAIDA
– http://java.freehep.org/jaida/index.html

JAS3
– http://jas.freehep.org/jas3