Transcript ppt
Globus – status & perspective Olle Mulmo <[email protected]> PDC Enabling Science What this talk will cover • Background: Why we are Where we are today • Where we are today • Where we will go tomorrow [Insert proper acknowledgement to everyone I have stolen slides from here] PDC Enabling Science Metacomputing... • A concept conceived of in the (late?) ’80s • 1990’s: Dramatic increase in network bandwitdh • • • • • Opens up for entire new problem areas Solving ”larger-than-a-supercomputer” problems Remote visualization Remote instrumentation Collaboration (VR) • 1995: • I-WAY, proof-of-concept demo/project • 1996: • The Globus Project sees the day PDC Enabling Science The Globus approach • Middleware approach • A set of orthogonal APIs that applications can use • Command-line tools that use the APIs (for scripting) • IP hourglass model Applications Diverse global services Core services • Portability requirements • No perl on the Cray at the time • Poor C++ support • -> Implementation using bourne shell (sh) and ANSI C only Local OS PDC Enabling Science Hot technologies half a decade ago • Network communication • • • • • Raw ATM vs TCP/IP? Use Nexus to avoid technology lock-in Already ported to several UNIX flavors Automatic data conversion Optimization for local communication (AAL5,TCP,MPI,SHMEM) • Information Discovery • • • • Queryable, Extensible (”give me a computer with 128M RAM”) Use LDAP (Lightweight Directory Access Protocol) Add TTL on all published data Add heart beats to know the test bed status PDC Enabling Science Hot technologies half a decade ago (cont.) • Security • • • • Hot potato: Local control a must -> mapfiles (what users running on what accounts) PKI evolving, big push from SSL -> let’s use it • Resource Allocation • No standard: myriads of different local schedulers • -> RSL (Resource Specification Language) + wrappers PDC Enabling Science First round: the GUSTO testbed • 1997: The Globus Metacomputing Toolkit • MDS (Metacomputing Directory Service) • GRAM (Globus Resource Allocation Manager) • DUROC (multi-site coordinated startup) • No security (plain-text passwords) • No file transfers PDC Enabling Science The GUSTO testbed (cont.) • 1998: Globus release 1.0 • Security added: GSI (Globus Security Infrastructure) • • • • PKI (or Kerberos) below GSS-API SSL handshakes Proxy certificate delegation No encryption due to export laws • File transfer added: • GASS (fetch files over https:// or file://) PDC Enabling Science The GUSTO testbed (cont.) • ... not really that heavily used, though... PDC Enabling Science Lessons learned from Globus v 1.x • Severe performance problems with MDS • A centralized LDAP server didn’t scale very well • -> MDS-2 framework (GRIS and GIIS) • Complaints on configuration and build times • You got everything and all the ”flavors” • debug,nodebug, nothreads,pthreads,native threads, with MPI, without MPI, ... • -> Packaging technology • One of the multi-site jobs often got stuck in a queue / poor network performance • -> GARA (Advanced reservation) PDC Enabling Science Lessons learned from Globus v 1.x • Users don’t want to hack their favorite applications • Easy access • They have NT on their desktops • Mass storage accessibility is important • -> Data and replica management work begins PDC Enabling Science 1999: The year of the Grid • The Concept of a Grid evolves • Resource sharing & coordinated problem solving in dynamic, multi-institutional virtual organizations PDC Enabling Science 1999-now: The Grid boom... • The Grid book covers a lot of ground in the community • Lots of grid projects spring to life • Increase in funding • Commercial interests (Microsoft,Sun,IBM,...) • Grid Forum is founded • ”The place to standardize Grid-related stuff” • Many of the protocols and approaches tried out by Globus are adopted as a first shot PDC Enabling Science Some renaming is required... • GSI -> Grid Security Infrastructure • GRAM -> Grid Resource Allocation Management • MDS -> Monitoring and Discovery Service • Never say ”Metacomputing” again... PDC Enabling Science Back to the drawing board: Globus v 2.0 • • • • • New packaging technology (not RPM) Precompiled binaries GridFTP MDS-2: decentralized LDAP framework Security on all communications channels • Repackaged with add-ons by several DataGrid projects • GriPhyN, EDG, NMI, ... PDC Enabling Science Where we are today • The Globus Toolkit™ centers around four key protocols • Connectivity layer: • Security: Grid Security Infrastructure (GSI) • Resource layer: • Resource Management: Grid Resource Allocation Management (GRAM) • Information Services: Grid Resource Information Protocol (GRIP) and Index Information Protocol (GIIP) • Data Transfer: Grid File Transfer Protocol (GridFTP) PDC Enabling Science A second look • Many different protocols and methods are used • LDAP, HTTP, FTP, ... • Missing or lack of support for • Databases • Notification systems • Error propagation • The security part needs more work • Dynamic, Multi-VO Authorization schemes • Audit • Proxy delegation (IETF draft not likely to become an RFC) PDC Enabling Science Let’s go shopping: Hot technologies of today • XML & Web services • Generic and extensible • Many tools available • Backed up by industry • Attribute Certificates • Give a certificate holder certain privilegies (access to files, resources,...) • NMI is pursuing this path • -> Globus ships CAS (similar concept) PDC Enabling Science Web services means generic encapsulation • ”Corba rewritten in XML” (and then some) • WSDL: Web Services Definition Language • Defines a service and what methods that can be invoked • Stub compilers exist for several programming languages • -> Easy to wrap your old application • SOAP: Simple Object Access Protocol • Invocation protocol, operating on an instance of a WSDL target • WS-Inspection and UDDI • Conventions (naming etc.) for querying and locating WSDL documents with certain properties PDC Enabling Science Open Grid Services Architecture (OGSA) • Web Services for the Grid • WSDLs for some key components • Factory, Registry, Grid Service • Dynamic properties (lifetime, version compatibility) are added • Open-source implementation in Java, .NET & C • Guidelines, conventions and best practices • • • • Global naming Lifetime managment Notification framework ... PDC Enabling Science OGSA • Still a draft (continously revised) • Alpha release of Core implementation (Java) • The Factory and generic Grid Service part • Base implementation next • Current Globus functionality refactored as Grid Services • Alpha at the end of this summer • Globus 3.0 (Q1 ’03?) • Old APIs will still work (or so they say) • ...but the wire protocols are new GT3 Data Services Other Grid Services GT3 Base Services GT3 Core PDC Enabling Science Next steps: your guess is as good as mine... • Experiences learned with v2 and v3 • Access to tertiary storage and databases • How much can be gotten ”for free” from other WS initiatives? • Other forms of security infrastructures? • Passport / Liberty Alliance technologies • ”MyProxy” effort • Continued work on authorization & delegation PDC Enabling Science