Transcript UNICORE Plus

GRIP: Interoperability between
UNICORE and Globus
D. Erwin, M. Rambadt, Ph. Wieder
Zentralinstitut für Angewandte Mathematik
Forschungszentrum Jülich
Terena 2002, Limerick, Ireland
June 5, 2002 - Session 7A
Contents
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGA
June 5, 2002
Terena 2002
2
GRIP: GRid Interoperability Project
Funded in part by EU grant IST 2001-32257
Duration: January 2002 – December 2003
Budget: 1.9 Mio €
URL: www.interoperability-project.org
June 5, 2002
Terena 2002
3
Partners
• Forschungszentrum Jülich, D (Coordinator)
• Pallas GmbH, D
• University of Manchester, UK
• University of Warsaw, PL
• Deutscher Wetterdienst, D
• University Southampton, UK
• Fecit, UK
• Argonne National Laboratory, US
Foils created by partners of the GRIP project
June 5, 2002
Terena 2002
4
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGA
June 5, 2002
Terena 2002
5
UNICORE Goals
UNICORE:
UNiform Interface to COmputing Resources
• conceived prior to the ‘invention’ of the GRID
• UNICORE has both a modest and an ambitious
goal:
• Create
– Seamless
– Secure
– Intuitive
access to distributed German HPC resources
• Create a system that can be used in production
at the German HPC centers
June 5, 2002
Terena 2002
6
PC²
LRZ: SR 8000, VPP
RUS: Hitachi, SX-5
FZJ: 2xT3E, T90, SV1
FZJ
ZIB: T3E, SGI
DWD: T3E, IBM SP
RUKA: IBM SP
PC²: hpcLine
Pallas
TUD: SGI
RUKa
Pallas: Sun
fecit: VPP, Sun
June 5, 2002
Terena 2002
DWD
ZIB
TUD
RUS
LRZ
7
UNICORE Functions
• Creation of system-independent jobs
• Execution at different sites
• Creation of complex interdependent multi-system
and multi-site jobs
• Monitoring and control of jobs
• Dependencies
• Workflow
June 5, 2002
Terena 2002
8
UNICORE Functions
• Transfer of data to and from the workstation
• Automatic data transfer between systems and
sites
• Access to data in Unix file systems and archives
• Application support
• No changes to third party applications
• Support for legacy job scripts
• Support for Metacomputing
• Performance analysis using VAMPIR
June 5, 2002
Terena 2002
9
UNICORE Functions
• Single sign-on
• No changes to local user management and
naming conventions
• Secure communication over the Internet
• Support for all platforms, operating systems, and
batch systems at partner sites
• Retaining of administrative autonomy and
security policies of participating sites
June 5, 2002
Terena 2002
10
UNICORE Functions
• Continued development and support after the
end of the project
• Basis for additional national and international
projects
• Production quality prototype
Objects will be achieved by year end 2002
URL: www.unicore.de
June 5, 2002
Terena 2002
11
EUROGRID
• European Grid Testbed
• Based on UNICORE technology
• Development of additional UNICORE functions
• Funded in part by EU grant IST-1999-29247
• Duration: November 2000 - October 2003
URL: www.eurogrid.org
June 5, 2002
Terena 2002
12
Geography
June 5, 2002
Terena 2002
13
EUROGRID Partner
–
–
–
–
–
–
–
–
–
–
–
Forschungszentrum Jülich, D
Pallas GmbH, D (Project Coordinator)
Parallab - University of Bergen, N
CNRS - IDRIS, F
Warsaw University - ICM, PL
Victoria University of Manchester, UK
Deutscher Wetterdienst, D
GIE EADS CCR, F
ETH Zürich (CSCS Manno), CH
Fecit, UK
T-Systems, D
June 5, 2002
Terena 2002
14
Globus
• Development by Argonne National Laboratory
and other US partners
• Defines Protocols and APIs for GRIDs
• Toolbox to enable development of GRID aware
applications
• Basis for many international Grid projects
June 5, 2002
Terena 2002
15
Grid Architecture
Application
Layer
Applications, Portals, Environments
Collective
Layer
Higher Services, APIs, Protocols
Resource
Layer
APIs, Protocols, SDKs
Fabric
Layer
June 5, 2002
Computers, Networks, Devices
Terena 2002
16
Grid Architecture
Application
Layer
Applications, Portals, Environments
Collective
Layer
Higher Services, APIs, Protocols
Resource
Layer
APIs, Protocols, SDKs
Fabric
Layer
June 5, 2002
Globus
Computers, Networks, Devices
Terena 2002
17
Grid Architecture
Legion
Application
Layer
Applications, Portals, Environments
Collective
Layer
Higher Services, APIs, Protocols
Resource
Layer
APIs, Protocols, SDKs
Fabric
Layer
June 5, 2002
Computers, Networks, Devices
Terena 2002
18
Grid Architecture
UNICORE
Application
Layer
Applications, Portals, Environments
Collective
Layer
Higher Services, APIs, Protocols
Resource
Layer
APIs, Protocols, SDKs
Fabric
Layer
June 5, 2002
Computers, Networks, Devices
Terena 2002
19
Grid Architecture
UNICORE
Application
Layer
Applications, Portals, Environments
Collective
Layer
Higher Services, APIs, Protocols
Resource
Layer
APIs, Protocols, SDKs
Fabric
Layer
June 5, 2002
Globus
Computers, Networks, Devices
Terena 2002
20
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGA
June 5, 2002
Terena 2002
21
GRIP Objectives
• Develop software to facilitate interoperation
between UNICORE and Globus combining the
unique strength of each system
(Pallas, fecit, UoM, FZJ, ANL)
• Build and demonstrate biomolecular (MD & QC
codes) and meteorological (RLM) inter-grid
applications (ICM, DWD, SOTON)
• Create wrappers to support commercial
applications in a Grid (no code changes)
• Contribute to and influence international Grid
standards through the Global Grid Forum
June 5, 2002
Terena 2002
22
GRIP Objectives
• Run a UNICORE job on Globus resources
• Run Jobs using Globus and UNICORE
resources together
• Prototype proposed standards in future versions
of UNICORE
June 5, 2002
Terena 2002
23
Key UNICORE Technologies
•
•
•
•
Seamless Computing Model
– Job abstraction
– Incarnation
– File staging and transfer support
Security Model
– X509 certificate based security model
– Consigner plus endorser model
– Several CA policies
– Fully flexible firewall support
Generic Client
– No changes to application needed
– Handy GUI environment
Portable Server
– Java + Perl
June 5, 2002
Terena 2002
24
Parallels and Contrasts
•
Resource Description
– UNICORE: Same model for discovery and request
– Globus:
Different models for discovery and request
•
Jobs vs Applications
– UNICORE: Workflow environment
– Globus:
Application APIs and toolkit
•
Security
– UNICORE: End-to-End security model
– Globus:
Requires transitive trust
•
Incarnation and Grounding
– UNICORE: Incarnation of abstractions at server
– Globus:
Client side grounding (substitution to ‘normal form’)?
•
Protocol Complexities
– UNICORE: Polling (request/response)
– Globus:
Call back
June 5, 2002
Terena 2002
25
Missing Technologies
• Missing Models in UNICORE
–
–
–
–
–
Distributed directory services
Multi-language Implementations
Application level message passing
Event management
Synchronized job start
• Implementation to be completed in UNICORE
– Resource reservation
– Interactive processing
– Dynamic support for new Vsites on the fly
June 5, 2002
Terena 2002
26
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGA
June 5, 2002
Terena 2002
27
UNICORE Architecture
User Workstation
UNICORE GUI
HTTP
Site List
SSL
UNICORE Server
UNICORE Server
Gateway
Gateway
NetworkJob
JobSupervisor
Supervisor
Network
Network
Job Supervisor
SSL
TCP/IP
NetworkJob
JobSupervisor
Supervisor
Network
Network
Job Supervisor
TCP/IP
Batchsystem
Target Batchsystem
System
Interface
Batchsystem
Target Batchsystem
System
Interface
Batch SubSystem
Batch SubSystem
UNICORE Site 1
June 5, 2002
UNICORE Site n
Terena 2002
28
UNICORE Architecture
User@Workstation
UNICORE Client
X.509
User Certificate
Job Preparation Agent
(JPA)
Abstract jobs,
status requests
Job Monitor Controller
(JMC)
Authentication
UNICORE Login
optional firewall
Site-specific
authentication
Server@Usite
Gateway
User validation
optional firewall
Network
Job Supervisor
(NJS)
(Incarnation/Scheduling)
Network
Job Supervisor
(NJS)
(Incarnation/Scheduling)
a
Local
Local
useruser
DBDB
Incarnation
Incarnation
DBDB
batch jobs, status requests, data
system1
TS Interface
hosts
in a Vsite
batch subsystema
June 5, 2002
systemn
TS Interface
batch subsystemx
Terena 2002
29
Proposed Architecture
Client
grid-proxy-init
plugin
Gateway
(Logical) Globus Usite
Generic
Globus IDB
Vsite A
NJS
IDB
Host A
Globus TSI
Vsite B
NJS
IDB
Host B
Globus TSI
IDB
Maker
Globus JM
Host A
June 5, 2002
GRIS
Terena 2002
Globus
GIIS
GRIS
Globus JM
Host B
30
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGA
June 5, 2002
Terena 2002
31
GRAM job request
Batch Job
Callback
Data
Data
Request
Creation
Enhanced Target System Interface
June 5, 2002
Terena 2002
32
Experience
Simplified Architecture implemented by M.
Rambadt as part of his Master thesis:
• No changes to UNICORE or Globus
• Fixed Globus resource as a Visite
• Extended TSI interfaces UNICORE and Globus:
– To create proxy certificates
– To map AJO to RSL
– To acts as Globus client
– To return results and status from Globus to NJS and
Client
 Proof of concept
June 5, 2002
Terena 2002
33
GRIP Partners
UNICORE and Globus
GRIP Objectives
The GRIP Architecture
Experience with an early prototype
GRIP and OSGA
June 5, 2002
Terena 2002
34
Outlook
• First GRIP deliverable (end 2002) will interface
UNICORE and Globus 2.0
• Project partners work with GGF on future
developments
• OGSA is an important topic
• OSGA will be considered by GRIP project for
2003 (requires agreement by EU)
June 5, 2002
Terena 2002
35
Outlook
• Architecturally OGSA and UNICORE are very
compatible
• WDSL service types for AJO classes
– Semi-automatic translation for Web Services
– More work for OGSA extension
– Demonstrated by Dave Snelling in Manchester
• Authorization and Security
– Not yet fully defined in OGSA
– UNICORE expects X.509 to Unix login mapping
June 5, 2002
Terena 2002
36
Outlook
• UNICORE will continue to contribute to Grid
developments
• Grid development will benefit from experience
gained at UNICORE Test and Production sites
• Accepted standards with proven
implementations are essential
June 5, 2002
Terena 2002
37
Web addresses:
• http://www.unicore.de
Information about the project
• http://www.fz-juelich.de/unicore-test
Free UNICORE Client Software
• http://www.unicore.org
Download for Software + Sources
• http://www.eurogrid.org
Information about EUROGRID
• http://www.grid-interoperability.org
Information about GRIP
June 5, 2002
Terena 2002
38
http://www.unicore.de
http://www.fz-juelich.de/unicore-test
?
http://www.eurogrid.org
http://www.grid-interoperability.org
June 5, 2002
Terena 2002
39