Transcript DIRAC Review

DIRAC Review
Application Software Installation
Stuart K. Paterson
DIRAC Review (13th December 2005)
1
Contents
Distribution of Software on the Grid
Software Distribution in DIRAC
User Perspective
Outlook & Improvements
Stuart K. Paterson
DIRAC Review (13th December 2005)
2
Distribution of Software on the Grid
Should be lightweight
Easily maintainable
Easy to deploy
Remote software repository with high connectivity
As platform independent as possible
Python based installation tools
Stuart K. Paterson
DIRAC Review (13th December 2005)
3
Software Distribution in DIRAC
install_project.py developed by F.Ranjard
Standalone
Developed in Python
Maintained by experts at CERN
Uses CMT
Relies on CMT for Application setup and execution
Exports CMT structure based at CERN
Added advantages of this
Use CMT to set up fake package to handle user DLLs
Eventually could ship user CMT packages to WN
Stuart K. Paterson
DIRAC Review (13th December 2005)
4
Pre-Installed Software
Although it is possible for jobs to install all of the LHCb
software, if possible this should be avoided
Significant reduction to Application start time and load on the
repository
Ideally
Install ‘released’ versions of all LHCb software at all Tier 1’s
Automatic distribution of software should be triggered when
new releases are made
Potentially, job arriving at site could trigger the installation
Current Situation
Have DaVinci v12r14 installed at some Tier 1 sites
Stuart K. Paterson
DIRAC Review (13th December 2005)
5
User Perspective
When executing within DIRAC, software installation is
completely transparent for the user
Only the Application and Version need to be specified
job.setApplication('DaVinci','v12r14')
Preinstalled software is checked during installation of
DIRAC on the Grid
Part of dirac-install script
If new versions of packages are required, existing
dependencies will not be reinstalled
Stuart K. Paterson
DIRAC Review (13th December 2005)
6
Virtual Machine Paradigm
Compiler libraries are shipped along with the software
distribution
A self-consistent set of binaries are shipped which do not
require any special environment
Need only prepare binaries for one platform
Use the Linux loader to run applications
This allows the use of libraries shipped with distribution
including the compiler and system libraries
Very successful, can currently run on all Linux systems
64 bit architectures are still to be demonstrated
So far only problems observed are with 64 bit Python
Stuart K. Paterson
DIRAC Review (13th December 2005)
7
Outlook & Improvements
What is in place now is reliable although difficult to debug
if something does go wrong
Currently DIRAC can’t spot if installation fails
Need to agree on status codes between install_project.py and
DIRAC
Being tied to LHCb software distribution makes it difficult
to support some desirable packages e.g. ROOT
Can cause conflicts with Gaudi applications
Stuart K. Paterson
DIRAC Review (13th December 2005)
8
Outlook and Improvements 2
With submission of many jobs, start to suffer from
only having binaries on one server
May need several repositories in the future
Possible move to Grid storage
Mechanism for software removal should be put in
place
Stuart K. Paterson
DIRAC Review (13th December 2005)
9