Transcript EU-IndiaGrid Interoperability

International Grid Interoperability and
Interoperation Workshop (IGIIW07)
Bangalore, 10 December 2007
EU-IndiaGrid Interoperability:
status and plans
Marco Verlato
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
1
Outline
• EU-IndiaGrid project overview
– Objectives
– Activities
– 1st year achievements
• Interoperability between EGEE and GARUDA
– Analysis
– Strategy
– Some scenarios
• Conclusions
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
2
An EGEE related project
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
3
Indian Grid Projects
• India has two main grid infrastructures:
– The GARUDA National Grid Initiative
– The Department of Atomic Energy (DAE) Grid
GARUDA Grid
•Private Network
•Connecting 45 institutes
in 17 cities
•100 and 10 Mbps links
4 Mbps
Link
CAT: archival
storage
BARC: Computing
with shared controls
4 Mbps
Link
VECC: real-time
Data collection
IGCAR: wide-area
Data dissemination
DAE Grid
(gLite based)
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
4
Slide from Natraj C Addekoppa’s talk at Taipei Workshop
GARUDA components
Management & Monitoring
• Paryaveekshanam
Application (PoC)
• Disaster Management
• Bioinformatics
Access Methods
• Access Portal
• Problem Solving
Environments
Resources
• Compute, Data, Storage,
• Scientific Instruments,
• Softwares,..
Resource Mgmt & Scheduling
• Moab from Cluster Resources
• Load Leveler, Torque
• Globus 2.x
Data Management
• Storage Resource
Broker
Development Environment
• DIViA for Grid
Collaborative Environment
• Video Conferencing over IP
• Access Grid
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
5
EU-IndiaGrid goals
• To support the interconnection and interoperability
between EGEE and GARUDA for the benefit of eScience applications
• To build an e-Science Network Community:
– identify and aggregate research, scientific and industrial
communities which may benefit from the use of Grid
• To promote the use of advanced Grid technologies in
scientific communities
– Specific pilot applications (HEP, Biology, Material Science,
Earth and Atmospheric Sciences) and dissemination activities
• Project start: October 2006
• Duration: 24 months
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
6
PARTICIPANTS
EUROPE
• INFN (project coordinator)
• Metaware SpA
• Italian Academic and Research Network (GARR)
• Cambridge University
•
•
•
•
•
•
•
•
INTERNATIONAL
Abdu Salam International Centre for Theoretical Physics
INDIA
Indian Education and Research Network (ERNET)
University of Pune
SAHA Institute of Nuclear Physics, Kolkata
Centre for Development of Advanced Computing (C-DAC)
Bhabha Atomic Research Centre, Mumbai
TATA Institute for Fundamental Research (Mumbai)
National Centre for Biological Sciences (Bangalore)
Not formally, but concretely: VECC, Kolkata and MIT, Chennai
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
7
Activities
Project administrative and
technical management
INFN
+all
WP2
Building an eScience Network
Community
MTW
+GARR,ICTP,NCBS,ERNET, PUNE, CDAC,
BARC, TIFR
WP3
Network Planning & Support
GARR, ERNET
WP4
Pilot grid infrastructure
operational support
INFN
+CAMBRIDGE, ERNET CDAC, BARC, TIFR
Applications
INFN
+ICTP,CAMBRIDGE, NCBS, PUNE,SAHA,
CDAC, TIFR
Dissemination & Networking
Events
MTW
+all
WP1
WP5
WP6
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
8
Main 1st year achievements
• Five Project Workshops + several trainings
– Mumbai, Dec 2006; Bangalore, Apr 2007; Kolkata, Jun
2007; Pune, Sep 2007; Taipei, Nov 2007
• EU-IndiaGrid Project Conference here this week
• Enabling exploitation of high bandwidth EU-India
network connectivity
– 45Mbps in October 2006; 1Gbps November 2007
• Supporting the creation of the National Indian CA
• Operating the euindia VO and infrastructure with
>1200 CPUs , >10 CPU.years consumed by appl.s
• many scientific applications ported on both GARUDA
and EGEE grids see posters at this conference
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
9
Interoperability/1
• The 2nd year will be more focused on Interoperability
• The 1st year was spent in:
– achieving some pre-requisites:
• ASGC CA acting as catch-all CA until the Indian National CA is
established and approved by APGridPMA
• Interconnecting GARUDA’s private network with the
intercontinental link
– carefully analysing the gLite and GARUDA middlewares
– discussing among partners a number of possible
interoperability scenarios
– defining a stepwise strategy to achieve the goal
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
10
Interoperability/2
• It was soon recognized the need of:
– taking advantage from EUChinaGrid experience about
interoperability with CNGrid GOS middleware
– identifying solutions being compliant with consolidated/emerging
standards (GLUE, JSDL, OGSA-BES, gridFTP, SRM)
• Corresponding actions undertaken:
– EU-Asia Interoperability sessions/workshops/round tables
organised at international grid events
• at EGEE’07 of Budapest, October 2007
• at WLCG-Tier2/EU-IndiaGrid Workshop of Taipei, November 2007
• at IGIIW and EU-IndiaGrid Conference of Bangalore, this week
– Participation to Open Grid Forum Interoperability Panel planned
from OGF22, February 2008
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
11
Picture taken at Mumbai
Workshop, December 2006
ANALYSIS
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
12
MW components analysis
GSI+
gridmapfile
MDS
SRB
PORTAL
UI
MOAB
WMS
GLOBUS
JC+LM
GARUDA-CE
(GT2 based)
lcg-CE
(GT2 based)
GSI+
VOMS
BDII
LFC
SRM
WN-1
Linux + AIX
PBS-Like
LRMS
WN-2
WN-n
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
13
The strategy
• To deploy gLite middleware at Indian partners sites
– CE and SE deployed at VECC, Kolkata in September 2007
– First full-fledged gLite testbed deployed at MIT, Chennai in
October 2007
– CDAC-Bangalore and Pune coming soon
• To implement a “basic” job submission interoperation
scenario with the present middleware components
• To implement a more advanced scenario
evaluating/adapting the EUChinaGrid Gateway
• To implement Data Management interoperation
leveraging SRM-SRB interface developed by ASGC
• To design a true interoperability model based on OGF
standards, to be implemented in a future I3 project
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
14
Basic Job submission interoperation
lcg-CE
(Torque)
VOMS
GARUDA
portal
gLite UI
gLite WMS
BDII
lcg-CE
(Torque)
lcg-CE
(Torque)
Mixed WN:
GARUDA + gLite TAR-WN
lcg-CE
In case of AIX sys.
could be evaluated
AIX-ENEA solution
WNs must support
both environments
gLite
Resources
cloud
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
15
ENEA-Grid approach to
provide access to AIX
Can be applied on other contexts, being based on:
1)
2)
availability of a shared file system on the site WN and
on the GRID Proxy WN
use of a remote execution tool
for implementing the wrapper
it works also with
NFS or GPFS
it works also with
rsh or ssh
Invasiveness of the grid middleware and
firewall requirements are minimized !
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
16
Slides from Subrata Chattopadhyay’s talk at EGEE’07
EUChinaGrid Gateway/Adapter Approach
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
17
Data Management
Interoperation
•
•
ASGC is developing a
SRM interface to SRB
DEMO at SC’07
•
Short term plan
SRM
– Implement interface
compatible with SRM
version 2.2
•
Mid-term plan
– Interact and test with
other data management
systems such as Castor,
dCache,…,etc.
•
SRB
Thanks to
Fu-Ming Tsai
Long-term plan
– Interoperate with gLite.
• Hope glite users who
join VO can access files
and resources in SRB
and vice versa.
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
18
Architecture Overview
Thanks to
• The ICTP team developing StoRM
Fu-Ming Tsai
is part of both EU-IndiaGrid and
SRM-WG of OGF
Users/applications
• Possible
cooperation with
ASGC for:
– interacting with
StoRM
– deploying/adapting
the SRM-SRB
interface in EUIndiaGrid
File transfer
(gridftp)
File transfer
(gridftp)
SRM API
Web Service
Core
File catalog
Data server
management
Gridftp/management
API
Gridftp/management API
Cache repository
SRB+DSI
SRB/gridftp
Cache server
(+gridftp server)
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
19
Interoperability with OGF standards
• OMII-Europe EU project is delivering components to enable
interoperability across different middlewares:
– gLite, Globus, UNICORE, OMII-UK and CROWN
• For gLite-Globus we could use:
– CREAM-BES: new gLite CE compliant with BES and JSDL, beta
release in December, stable release by April
– GT4 OGSA-BES: pre-alpha version GT4 CE
– VOMS for the security (compliant with OGF OGSA AuthZ WG )
– GLUE Schema for the Information System
• adapting these components to GARUDA and gLite environment
requires effort for development  JRAs in I3 EU Projects
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
20
Future job submission using
JSDL/BES standards?
(thanks to Moreno Marzolla / OMII-EUROPE)
GARUDA
Portal
CREAM-CLI
BES-CLI
GARUDA Scheduler / MOAB
GLOBUS
GT2
GT4
GARUDA-CE
(GT2 based)
GSI +
VOMS
BDII with
GLUE schema
globus
BES
conn.
BES
GARUDA-CE
(GT4 based)
gLite UI
gLite WMS
gLite
BES
conn.
BES
CREAM-CE
(WS based)
ICE
JC+ LM
lcg-CE
(GT2 based)
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
21
Conclusions
• The main focus of EU-IndiaGrid Project in its 2nd year will be the
interoperability between gLite and GARUDA
• Several discussions held, several approaches figured out during the
1st year, and a stepwise strategy has been proposed
• The Project is actively organizing panel sessions, round tables,
discussions about interoperability with other Projects interested in
this matter (EGEE’07 in Budapest, WLCG-WS in Taiwan, eScience
2007 in Bangalore)
• A contact with OGF was started late September 2007, the
participation at Interoperability Panel at upcoming Open Grid Forum
events is planned
• EU-IndiaGrid is ending in September 2008
• A proposal is ready for EU-IndiaGrid2, describing a more detailed
way on standards adoption/collaboration for job and data
management interoperability
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
22
Thank you
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
23