Transcript GARUDA_-_overview_and_interoperability-egee

GARUDA - The National Grid
Computing Initiative of India
Subrata Chattopadhyay
C-DAC, Bangalore, India
[email protected]
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Outline
• GARUDA Overview
• Architecture & Network
• Resources
• Partners
• Applications for PoC
• Interoperability
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Motivation for GARUDA
• Sharing of high-end computational resources with the
larger scientific and engineering community across
the country
• Emerging High Performance Computing (HPC)
applications require integration of geographically
distributed resources
• Collaborative Framework for solving applications
which are interdisciplinary requiring experts from
multiple domains and distributed locations
• Universal (location-independence, ubiquitous) access
to resources
Computational Grids effectively address the
above application requirements
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Project Overview
•
•
•
•
Precursor to the National Grid Computing Initiative
– Test Bed for grid technology/concepts and applications
leading to the plan for the main grid initiative
Project initiated by the Dept. of IT in November 2004
Major Deliverables
– Technologies, Architectures, Standards & Research
Initiatives
– Nation-wide high-speed communication fabric
– Aggregation of Grid Resources
– Deployment of Select applications of National Importance
– Grid Strategic User Group
High-speed Networking Component in collaboration with
ERNET
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Garuda Component Architecture
Application (PoC)
• Disaster Management
• Bioinformatics
Management & Monitoring
• Paryaveekshanam
Access Methods
• Access Portal
• Problem Solving
Environments
Resources
• Compute, Data, Storage,
• Scientific Instruments,
• Softwares,..
Resource Mgmt & Scheduling
• Moab from Cluster Resources
• Load Leveler, Torque
• Globus 2.x
Data Management
• Storage Resource
Broker
Development Environment
• DIViA for Grid
Collaborative Environment
• Video Conferencing over IP
• Access Grid
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
GARUDA Component Deployment
Scenario`
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Cyber Infrastructure – Network Fabric
•
•
•
The Proof of Concept network has
been established in collaboration with
ERNET
The MPLS Virtual Private Network
(VPN) connects 22 institutions at 100
Mbps and 23 institutions at 10 Mbps
across 17 Indian cities with SLA
agreements
Collaborative environment enabled
through Video Conferencing over IP at
the following centres of C-DAC :
Bangalore, Pune, Chennai, Hyderabad,
Mumbai and Trivandrum
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Cyber Infrastructure – Resources
•
•
•
In addition to PARAM Padma,
Linux Clusters have been
setup at Pune, Hyderabad &
Chennai
Grid Labs have been setup at
Bangalore, Pune & Hyderabad
Fourteen of the partner
institutions are contributing
resources including satellite
terminals (compute
aggregating to more than 300
CPUs)
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Resource Details
•
•
•
•
HPC Clusters from C-DAC
Bangalore
:
AIX Cluster
Pune, Chennai, Hyderabad :
Linux Clusters
Linux Clusters from
• IISc & RRI, Bangalore
• IMSc & MIT, Chennai
• IIT, Guwahati
• IIT, Kharagpur
AIX SMP from University of Hyderabad
SAC, Ahmedabad is contributing
GSAT3 terminals (GSAT4 when
launched) and Satellite bandwidth
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Job submission flow
Bangalore Users
Garuda Access Portal
Pune Users
Other Users
End Users access the grid through the Garuda Access Portal
Bangalore
Pune
Resource
Manager for Grids
Resource
Manager for Grids
High
Availability
High
Availability
High
Availability
Shared User Space
Shared Data Space
AIX
Cluster
Chennai
Hyderabad
Resource
Manager for Grids
Resource
Manager for Grids
Linux
Cluster
High
Availability
Shared User Space
Shared Data Space
Solaris
Cluster
Linux
Cluster
Linux
Cluster
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Grid Management & Monitoring Centre
•
•
•
An integrated Resource
Management &
Monitoring Framework
Network Traffic Analysis
and Congestion
Management
Help desk for Grid Users
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
•
•
GARUDA Partners
Motivation
– To Collaborate on
Research and
Engineering of
Technologies,
Architectures, Standards
and Applications in Grid
Computing
– To Contribute to the
aggregation of resources
in the Grid
Current Participation
– 36 research & academic
institutions in the 17
cities
– ERNET-HQ in Delhi
– 8 centres of C-DAC
– Total of 45 institutions
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
GARUDA Partners (contd..)
C-DAC Centres (8):
• Pune (2)
• Bangalore(2)
• Hyderabad
• Mumbai
• Chennai
• Thiruvananthapuram
Government Collaborators
• ERNET India
Academia
• Motilal Nehru National Institute of Technology, Allahabad
• Indian Institute of Science, Bangalore
• Punjab Engineering College, Chandigarh
• Madras Institute of Technology, Chennai
• Jawaharlal Nehru University, Delhi
• Guwahati University, Guwahati
• University of Hyderabad, Hyderabad
• Indian Institute of Technology at :
• Chennai
• Delhi
• Guwahati
• Kanpur
• Kharagpur
• Mumbai
• Roorkee
• University of Pune, Pune
• Institute of Technology, Banaras Hindu University, Varanasi
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
GARUDA Partners (contd..)
Research Labs & Institutions
• Institute for Plasma Research, Ahmedabad
• Physical Research Laboratory, Ahmedabad
• Space Application Centre, Ahmedabad
• Harish Chandra Research Institute, Allahabad
• Indian Institute of Astrophysics, Bangalore
• National Centre for Biological Sciences, Bangalore
• Raman Research Institute, Bangalore
• Institute of Mathematical Sciences, Chennai
• Institute of Microbial Technology, Chandigarh
• Institute of Genomics and Integrative Biology, Delhi
• Centre for DNA fingerprinting and Diagnostics, Hyderabad
• Saha Institute of Nuclear Physics, Kolkatta
• Sanjay Gandhi Post Graduate Institute of Medical Sciences, Lucknow
• Bhabha Atomic Research Centre, Mumbai
• Tata Institute of Fundamental Research, Mumbai
• Inter-University Centre for Astronomy and Astrophysics, Pune
• National Chemical Laboratory, Pune
• National Centre for Radio Astrophysics, Pune
• Regional Cancer Centre, Thiruvananthapuram
• Vikram Sarabhai Space Centre, Thiruvananthapuram
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Illustrative Applications
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Objectives and Deliverables
•
•
Objectives
– Enable applications
of national
importance requiring
aggregation of
geographically
distributed resources
Deliverables
– Grid enablement of
illustrative
applications and
some
demonstrations such
as
• Bioinformatics
• Disaster
Management
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Bioinformatics
•
Bioinformatics Resources
& Applications Facility
(BRAF) on PARAM Padma
•
Supports highly optimized
Bioinformatics codes on
the PARAM Padma
•
Web computing portal
providing all
computational facility to
solve related problems
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Disaster Management
Grid Partner
Resource
Flight data
transmission from
nearby Airport
User
Agencies
GRID
Communication
Fabric
User
Agencies
High Speed
Communication
PARAM Padma
EU-IndiaGrid (RI-031834) is funded
by the European Commission
under the Research Infrastructure Programme
at Pune
at Bangalore
www.euindiagrid.eu
Virtual User Community
•
•
•
•
•
•
•
Astrophysics
High Energy Physics & Astronomy
Grid Technology
Disaster Management
Earth Science
Bioinformatics (Genome)
Network Technology
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Garuda – EGEE Interoperability
•
•
•
•
Garuda components
EGEE components
Options for interoperability
Discussion
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Job Submission Flow Through Grid Portal
Login to Portal
Develop
Application
Write
Application
Get & Match
Requirement and
identify resources
Compile
Yes
Use
Debugger
The files get
transferred to the
execution
machine(s)
The job
executes
The job
completes
execution
The files get
transferred to the
Grid Head Node
If
Error
No
Use resource
browser and identify
the platform
Submit Job with
its requirements
PSE
Upload Input
datasets
Get & Match
Requirement and
identify resources
The files get
transferred to the Grid
Head Node
View O/P files
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Genome Grid
Garuda Portal
User
Dynamic Resource
Scheduling
User
Client
GARUDA
Globus middleware
PUNE
Bangalore
100 Mbps
Chennai
100 Mbps
10 Mbps
10 Mbps
8 Node Dual CPU Hyderabad
8 Node Dual CPU
Xeon Cluster
Xeon Cluster
8 Node Dual CPU
8 Node Dual(RI-031834)
CPU
EU-IndiaGrid
is funded by the European Commission under the Research Infrastructure
Programme
Xeon Cluster
Xeon Cluster
www.euindiagrid.eu
EGEE Services
CLI
API
Access
Information &
Monitoring
Authorization
Auditing
Authentication
Metadata
Catalog
Storage
Element
Security
Services
Service
Discovering
Network
Monitoring
File & Replica
Catalog
Information &
Monitoring Services
Job
Provenance
Package
Manager
Computing
Element
Workload
Management
Accounting
Data
Movement
Data Management
Job Mgmt. Services
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Basic Interop. Job submission
GARUDA
Submission
(Portal)
gLite
Submission
(WMS)
PBS-Like
Master Node
gLite WN & UI TAR
available on SLC3/4
gLite (TAR)UI or subset(WMProxy APIs)
GARUDA
Portal Submission
GARUDA
Linux
Resources
WN-1
lcg-CE
(PBS submission)
gLite WMS
WN-2
WN-n
In case of AIX sys.
could be evaluated
AIX-ENEA solution
JDL creation &
Job submission
Mixed WN:
GARUDA + gLite TAR-WN
glIte
Resources
cloud
WNs must support
GARUDA ENV.
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Job Submission Flow for EGEE Grid User
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Job Submission Flow for GARUDA Grid User
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Challenges !
•
•
•
•
•
Multiple components – evolving
User authentication mechanisms
Complexity and stability
Open source and commercial license
Accounting
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu
Thank you!
www.garudaindia.in
EU-IndiaGrid (RI-031834) is funded by the European Commission under the Research Infrastructure Programme
www.euindiagrid.eu