Transcript in Java Electronic Document Management systems

CHEP 2000 Session F:
Data Analysis:Technology and Presentation
Convenors: Francois Etienne - IN2P3, Atlas,
Ruth Pordes, FNAL
Thank You to all the Speakers for Providing me
with their Transparencies to use.
29 Oral presentations; 1 Poster
Outline
• Quick Review of Topics/Titles
• More Detail for Each Category
• Summary
Numerical Library Software - 2 papers
• Development of Numerical Library Software in Java –
KEK – Initial Work
• Scientific Toolkit for Java – Phenix – Really an Analysis
Framework
• (poster paper not in this session on COLT - Cern/IT)
• All in Java
Electronic Document Management systems
- 4 papers from 2 systems
• Java-based User Interfaces for CERN's Engineering
Data Management System
• Integrated Information Management for TESLA
• Asset Management for Cavities and Computers at
DESY
• Experience with Document Management at DESY
Singletons:
• Bringing Mathematical Formatting to the Web - An
Overview of the Technology and SLAC's Experience
• Multi-threaded, Discrete Event, Simulation of
Distributed Computing Systems – CMS
• CORBA Scripting in HEP and Beyond.
• ATLAS DAQ back-end Software




Integrated toolset, mainly Open Source
Second use of this will be an interesting test.
Defining deliverables of “pre-design” phase.
How to move to an Open Source development
environment for critical piece of Experiment
architecture?
Event Displays - 3 talks
• Atlantis an Event Display for ATLAS: When
displaying 6000 hits recorded by only 7 layers,
can we recognize tracks?
• WIRED - World-Wide Web Interactive Remote
Event Display
• HepRep: a Generic Interface Definition for
HEP Event Display Representables
General Analysis Packages/Frameworks
• Java Analysis Studio
• ROOT at RUN II
• Open Scientist
• A Comparison of Data Analysis Packages (Root
and IDL)
Experiment Analysis Packages
• A Graphics Sampler for the D0 Experiment
• The CMS IGUANA Project
• Object Oriented Data Analysis in the DELPHI Experiment
• Distributed BELLE Analysis Framework
• New Data Analysis Environment in H1
General Analysis Components
• Abstract Interfaces for Data Analysis – AIDA –
Cern/IT
• A Histogramming package in Java - KEK
• Lightweight Histogram Interface Layer – from
NERSC/LBL
Software Process and Configuration Management
- 5 talks all from Europe..
• An Evaluation of Tools for Static Checking of C++
Code
• Impact of Software Review and Inspection
• C++ Code Analysis: an Open Architecture for the
Verification of Coding Rules
• CMT: a Software Configuration Management Tool
• Use of a Configuration Management Tool in LHCb
Software
What there was Not
• Software Development Methodologies and Case
•
Tools.
3-D and Virtual Reality.
Overall Impression
• Continuation of directions and initiatives from
last Chep.
F19 MathML Interface for Abstracts in
SLAC Spires
• Situation today (finally):


method for defining a mathematical expression in
terms of its presentation and semantics
MathML is a standardized instance of XML
• Used “IBM TechExplorer Hypermedia Browser”



A Web browser plug-in
Dynamically formats and displays documents coded
with TEX/LATEX and/or MathML
Beta version applied to existing Spires TeX files
works.
• Try it and feed back comments ...
F19 - Tool for Simulation of Complex
Distributed Data Handling Systems • Technology – in house developed JAVA toolkit.
Process driven not event driven architecture
 Full presentation and easy editing of parameters
 encourage many different models and users
 Immediate publication of results on the web
 Enable physicists to read, analyse and feedback.
• Alternative architectures and designs can be
commented with quantitative and analytic input
• Technology is ready to be tested for “Simulator
Friendliness” …
 Volunteers to model the existing Babar and Run
II systems would indeed be useful.

F359, F65, F66, - DESY Integrated
Information Management
• Reminded us of the importance of need for and time
required to
 Peoples Use and Access Patterns change after they
have experience with a working system
 Current and Future Business Processes significantly
affect design
 Users expect latest Technology (currently video,
mobile computing etc)
• Demonstrated Success of collaborative development of
vendor tool for HEP needs
Current Tuovi Status & Statistic
• 20 projects with:
- 6 GB data vault
- 2400 project nodes
- 18600 files
- 3300 registrated user
Tuovi 1.2 Up- and Download Statistic in 1999
10000
1000
upload
download
100
10
Ap
ril
M
ay
Ju
ne
Ju
l
Au y
g
us
Se
t
pt
em
b
O er
ct
o
No be
ve r
m
De be
r
ce
m
be
r
1
Ja
nu
a
Fe ry
br
ua
ry
M
ar
ch
Number of Actions
100000
Month
Cern EDMS Extensions
• Import/Export Interface - define an exchange data
•
format in XML.
 Conversion of metadata to and from various
formats e.g. Excel, Html, Databases, Text Processors
etc.
in use by several LHC groups:





LHC Baseline and ABS
ATLAS: Import/Export of ABS Structures
CMS: ECAL ABS
ALICE: Documents and Files check-in
Documents Management in the LHC Design office (in
progress)
Histogramming Packages
• Histogrammer in JAVA - Successful interfacing
of Java and C++ objects for in a simple
Histogrammer application
• Lightweight C++ Histogram Interface - Break
between Statistical Data Gathering, from analysis and
visualization tasks. Define Standard API and Support
Object Structures.
 How does it relate to transient Cern/HTL
implementation
Analysis and Event Display Toolkits and
Interfaces
•
•
•
•
•
•
•
•
WIRED
HepRep
AIDA
Java Analysis Studio
Scientific Toolkit for Java
Open Scientist
ROOT
IDL - through comparison with ROOT
Wired
• Rewrite since Last CHEP
• In Use by CHORUS, BaBar, DELPHI etc.
• Lacks a Scripting Language.
• BaBar Use shows up Need to Think before Defining
what Data gets Transported between User Client and
Data Server – don’t want a “Packet per Point”
HepRep
• Experience with Wired at BaBar led to need to layer
interface between Client and Display so as to

reduce granularity in getting remote information

increase ability to for Client/Users to easily add
objects and views
• HepRep defines a general API for serving
Representables, not Physics Objects
• “Serving Representables keeps the detailed
reconstruction code, swimmers and detector models on
the server side where they belong.
• Spatial information is assembled and shipped in an
efficient manner, avoiding the overhead of too many
individual method calls.”
General Comments
• Transport the Physics Analysis Code to the Data - JAS
– No Answer Yet on Whether this will Work Well.
• WIRED and JAS both Talking about Scripting
Languages – no decision yet.
• Once Package in Use and Responding to User Use
Needs, More Core Development More Difficult
• Integration with C++ Analysis Algorithms and Data
Adaptors Not Here Yet.
AIDA
• Part of Next LHC++ Analysis Toolkit – the Start.
• HepVis ’99 established Collaboration for Definition of
Abstract Interface for Data Analysis
• Interface Architected to Support Scripting and GUI
Equivalently.
• Goal of Adaptors/Interfaces to HepTuple, JAS,
Openscientist, Root etc.
JAS
• Extensions since last CHEP:




JEdit Editor Bean
Using Swing JEditorPane JAS now supports HTML display
XML support - JASHist bean directly supports reading/writing
XML
Servlet Support
• Continued use in BaBar Online and LCD Simulations
• Packaged and made Available in OpenSource
Paradigm - Contributions starting e.g. 3D Classes from
Fermilab.
Scientific Toolkit for Java
• Explained was a Separate Project to “Learn
Technology”.
 Use of JavaHelp
 Based onJava 2D (JDK 1.2)
• Generic Swing Console for those Interpretive
Languages not supported by Swing.
• Anticipate Discussing Collaboration with /
Contributions to JAS.
• Starting New Development Allowed use of Latest
Generation Technologies
Open Scientist
• Work Continuing for Experiments – LHCb, Virgo,
Auger etc.
• Concept of “Hub” as Integration Layer. What is
boundary/layering between Frameworks and Utilities Do we need More Concepts Here?
• Worklist includes Integration of CINT and Extensions
to Free Open Inventor Implementation.
ROOT - Run II Perspective
• Progress since Run II Adoption of ROOT with Caveats.
• Priorities from 1999 Workshop been in Large Part
Addressed – Robustness, CINT Extensions, Precompilation of user classes.
• Tutorials and Education Popular with Run II Users.
• To meet Local Needs currently Building 23 Flavors.
• Collaboration Works with Technical Commitment
• Second ROOT Workshop last week at Cern resulted in
New Worklist - now posted – main extension Self
Describing Data format.
Comparison of ROOT and IDL:
• IDL command language




Concatenate Sets of Files of ntuples - chain
command.
Read in and Use Variables from an “ntuple”File Local Extension.
Event Selection Cuts on Several Variables
Plot - Static or “Liveplot”.
• Large ntuple problem addressed by “mapped files”.
• Needs test drive for “large scale analysis”.
Experiment Specific
Event Displays and Analysis
Mature Experiments Retooling Technology
for Analysis – Delphi and H1 reported
• Issues include:


Maintenance – Technical and Resources;
Support of Multiple Generations of Technology.
• OO, ROOT, C++ - Speed and Usability Benefits
• Encourage New Collaborators
• Allow and Provide for Curation of the Data
Atlas Event Display - Atlantis
• Applying Hans Drevermanns Algorithms to
New/Future Experiment
 “V-plot”
 Z finder
 Hit filtering
• Plan Recode from Fortran to Java.
Experiment Event Displays – D0
• Suite of Different Tools based on Legacy and
New Packages.
• New Tools based on Open Inventor. Will
Evolve.
• Finding very useful for algorithm development
and starting for analysis of processed Monte
Carlo data
D Graphics Sampler
Rest of D0 software
D Graphics
HEPVis
Open Inventor
Xi
Graphics
Exceed
3D
Mesa
OpenGL
CMS - IGUANA
• On the way to being a Fully Featured Data
•
•
•
•
Presentation and Manipulation Framework.
Will be used in CMS Simulation and Test Beam
Analysis this Year.
Pragmatic Approach to getting Prototypes and Early
Production Releases.
Based on Open Toolkits and Open Inventor.
“Sharing experience may be more important than
sharing code”- aligned with LHC++ strategy.
IGUANA Prototypes: Objectivity Browsers
and
Persistent Histograms and Tags Plotters
IGUANA
Objectivity
browsing
BELLE – BASF Analysis Framework
• Currently supports Single SMP Parallel Processing of
Events for Users.
• dBASF Extension being Designed to Support Network
Connected Clusters of PCs. Package of Daemons for
 Job Management
 Data I/O
 Resource Management
 Monitoring.
• Resource Managers Dynamically Reconfigures –
Increase, Decrease - Number of Computing Servers
Running Jobs.
HEP Configuration Management Tools
• Report from 2 users of CMT – Build and
Release System.
• Poster papers on SRT Rewrite, NT Extensions
• From Marjorie’s Plenary Talk: “Everyone now
using them, can’t we use the same one…”
Packages & versions
Free action onto the
package
editor
Package areas
browser
jcmt
Configuration monitor
monitor
Requirements editor
F63 - C++ Code Rules and Checking
• C++ Coding Standard – 180 Rules for Naming, Coding
and Style of Varying Severity for Each Experiment.
• Evaluation of Commercial tools for Code Rule
Checking. Tested on Geant4 and Atlas Libraries.
• Out of 5 tools considered 2 Meet the Basic Needs –
about 75% of total: CodeWizard and QA C++.
• Now Traditional Questions on: “What does it take to
Decide?” “Will IT/CD Support It?”
F361 - C++ Code Rules and Checking Alice
• Development of Code Analysis Tools - Parser,
•
•
Formatter, Checker
Limitations in Checking Complete C++ syntax
e.g. Templates, Namespace
Successfully used on ALICE code
F331 - Atlas Software Review and Inspections
• Movement in HEP from Fagan Inspections (Fermilab
SDSS/DART/Run II) to Gilb Inspection Method
• Bodes Well that this is Part of Formal Development of
Proposed OpenSource Project..
F119 - Applications using CORBA Script –
continuation of work reported at Chep ‘98.
• Interpreted Scripting Language which can interact
with CORBA objects through IIOP and has dynamic
access to any CORBA object.
• OMG Standard
• Interface Definitions Stored in Repository
• Applications in Babar Data Handling System
Summary – Technology
Java, XML, CORBA
• JAVA

Mainstream for New Presentation Applications.

Not Mainstream for Data Analysis and
Reconstruction.

Work Not yet Done to Integrate with C++
Algorithm Modules and Frameworks.

Work only just Beginning on HENP Utility
Libraries.
Summary – Physics Analysis Packages
• Not much development using Commercial Analysis
Tools.
• ROOT most Generally Used; Extensions still being
requested to meet Experiment Long Term Needs. Open
Source Collaboration without Open Source licence?
• JAS, OpenScientist, WIRED, progressing steadily.
• Question still Open as to How or Whether any of the
Existing Analysis Technologies will Successfully meet
Needs of >500 Physicists doing Analysis of Babar and
Run II over the next 10 Years.
Summary – Development Environment
Technology
• Experiments looking for more and better tools for
Configuration Management but
 no widespread adoption yet of single solution
 no work yet on significant extensions to the services
provided
 no new work to look at commercial tools
• Attendees very interested in Code Quality and
Checking Tools and Process – but if the number of
talks is an indication of the relative investment..
Summary - misc
• Mature Experiments Taking Time to Upgrade Analyses
to New Technologies to Support Ongoing Analysis and
Provide Tools for Newcomers –

Reminder this will Happen for Current and Future
Experiments.
• (Where did Virtual Reality go?)