Transcript Achievement of EUFORIA Project Miguel Cárdenas Montes

JRA3
B. Guillerminet
on behalf of the JRA3 project
22 January 2008
Kick-Off Meeting
21-24 January 2008
Goteborg (Sweden)
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Outline

Participants

Objectives of JRA3

Portal

Jobs launching

Data access

Codes orchestration

Visualization

Service administration

Connections with the others activities

Plans

Road map

Risks assessment
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
WHO


Participants:

PNSC: 24 months (N Meyer, M Owsiak, B Palak, M
Ptocennik)

UEDIN: 8 months (L Smith, J Hein)

Chalmers: 4 months (P Strand, T Svedberg, D Nilsson)

IPP: 8 months (D Coster)

CEA: 16 months (M Airaj, B Guillerminet)

ULP: 8 months (E Sonnendrucker, R David, P Navarro)
Structure

JRA3 leader & coordinator of JRA3/JRA4: B Guillerminet
(CEA)
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Objectives

What
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Objective: Portal



Requirements:

single access point (based on ROAM, Shibboleth, SAFE …?)

access to applications (based on Migrating Desktop):

Data (codes & simulations) mining tool (catalogues + database)

Simulation parameters editor (Java based)

KEPLER: workflow (Java based)

Post-processing tools (SCILAB, VISIT …)

user queries

Accounting, monitoring
Tasks:

Specify & realize the Web portal

Integrate our applications in the portal
Who?

PNSC (24 months)

UEDIN (8 months)

Chalmers (?)

IPP (?)

CEA & ULP participate to the integration & tests of the applications
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Objective: Jobs launching




Requirements:

Complex workflows: loops, various directors, sequential & parallel execution

Long duration simulations: jobs could be launched at each time step (batch) and some must be kept
alive (call + notification: interactive jobs)

Need a Java API for EGEE (gLite based) & HPC (UNICORE 6): temporary certificate, job submission,
get job status, get job output + message oriented middleware (call + notification)

Integration in the workflow engine (Kepler)
Proxy
Status:
Job
transfer
via FTP

EGEE: Globus version (already in Kepler)

gLite: in progress

Migrating Desktop as a “gateway” between Kepler and GRID/HPC infrastructure
Needs:

Access to EGEE (gLite + VO fusion) & HPC (unicore 6) infrastructures

Support + investigation for the non-batch part (interactive jobs?)
Who?

CEA (6 months / 16)

PNSC
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Data
transfer
via FTP
Job
scheduli
ng via
JCL
Objective: Data access

Requirements:

Access to external data (EGEE/HPC): open, get, put … a data structure (allocatable array)

Fast access: in-memory data during the simulation (avoid memory to file + file transfer +
…)

Integration in the UAL (Many languages implementation: C/C++, F90, Java, …)

Must be defined clearly: connection with SA3 for SAFE

Needs:


Access to EGEE (gLite + VO fusion) & HPC (unicore 6) infrastructures: SA1 & SA2

Investigation with the biomedical & astronomy projects (EuroVO_AIDA.pdf @ 4th …)

Support: SA3
Who?

CEA (4 months / 16)

Chalmers (?)

IPP (?)
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Objective: Codes orchestration



Requirements:

Codes launching (EGEE/HPC) and coupling from the workflow engine: use the
previously provided libraries

Distributed workflow: part of the workflow will run on the gateway, on EGEE and
on HPC

Integration of codes: JRA1, JRA2 deliverables

Building a workflow based on ITM & JRA1/2 codes
Needs:

Investigation of various workflow engines: Gridway, … => collaboration or
support

Collaboration with UCSD (Kepler team): designing tool for distributed workflows
Who?

CEA (6 months / 16)

IPP : building the workflow?
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Objective: Visualization

Tasks:




Developments of specific actors

Used in a workflow (during the simulation)

As post-processing tool
Integration of the compression algorithm (deliverable from JRA4)? actor
or as a library
Needs:


Integrate the new visualization tools (based on VTK, Visit) in the code
platform:
Collaboration with JRA4: deliverable, training …
Who?

CEA (<1 months / 16)

ULP (?)
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Objective: Service administration

Requirements:



User administration

Resource management

Accounting

Reporting
Tool based on SAFE (UEDIN)
Integration in the portal
Needs



Task


Deliverable from SA3:
Training on SAFE and support from SA3
Who?

PNSC (24 months)

UEDIN (8 months)
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Connection with the others activities


NA1, NA2, NA3

Management: bi-weekly, f2f (3 months), GA (1
year)

Training on GRID, HPC, SAFE, Migrating Desktop

Dissemination
SA1, SA2, SA3

Login & CPU time to develop & test our software

Help

Implementation of the service administration
facilities
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Connection with the others activities


JRA1, JRA2

JRA1, JRA2 provide several codes

Execution using the API

Needs for orchestration

Data access (ITM data structure)?
JRA4

JRA3 provides Kepler + tools

Development of actors and integration of VTK,
Visit, …

Actor for data compression
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Plans

Road map

Work is mainly in 2008 & 2009
2008
2008
2008
2009
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
2010
Plans

Deliverables
2008
2008
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Plans

Risks assessment

Jobs launching:


only the batch mode is available => simulations will be less
efficient
Data access


2008
2008
No external data => could use file transfer (GridFTP …) =>
hidden to the user
Orchestration

No way to run a part of Kepler on the GRID/HPC => use
Gridway and build a tool to convert the Kepler workflow to
Gridway
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804
Work methods





Define precisely the work plan (JRA3 parallel
session)
Email as a communication tool
Additional collaborative tools (Savannah, Wiki
…)?
Meetings: PSG (2 weeks) + F2F (3 months) =>
rely on NA1 management
Mobility?
EUFORIA FP7-INFRASTRUCTURES-2007-1, Grant 211804