Transcript Grid Computing

Grid Computing
Yoab Gorfu
Abe Guerra
Kay Odeyemi
Renel Smith
Presentation Outline




Introduction
Architecture
Large Deployment Example - National
Fusion Grid
Grid Toolkits
–
–
Globus Toolkit
Stateful Web Services
Introduction


“A computational grid is a hardware and software
infrastructure that provides dependable, consistent,
pervasive, and inexpensive access to high-end
computational capabilities.”
Criteria for a Grid
•
Coordinates resources that are not subject to
centralized control.
•
Uses standard, open, general-purpose protocols
and interfaces.
•
Delivers nontrivial qualities of service.
Introduction

‘Grid Problem’ - ‘coordinated resource
sharing and problem solving in dynamic,
multi-institutional virtual organizations’ [1]

Virtual Organizations (VOs)
•
•
Vary dramatically
Core set of requirements
Introduction

VO requirements
•
•
•
•
Flexibility
Control
Varied resources
Usage modes
Introduction
Introduction

Grid Computing Benefits:
•
•
•
•
Exploit underutilized resources
Resource balancing
Virtualize resources across an enterprise
Enable collaboration for virtual organizations
Companies involved in Grid
Computing













Avaki
Axceleon
CapCal
Centrata
DataSynapse
Distributed Science
Elepar
Entropia.com
Grid Frastructure
GridSystems
Groove Networks
IBM
Intel











Jivalti
Mithral
Mind Electric
Mojo Nation
NewsToYou.com
NICE, Italy
Noemix, Inc.
Oracle
Parabon
Platform Computing
Popular Power
Source: http://www.gridcomputing.com/










Powerllel
ProcessTree
Sharman Networks Kazza
Sun Gridware
Sysnet Solutions
Tsunami Research
Ubero
United Devices
Veritas
Xcomp
Computation Grid Projects

Particle Physics
–global sharing of data and computation

Astronomy
–‘Virtual Observatory' for multi-wavelength astrophysics

Chemistry
–remote control of equipment and electronic logbooks

Engineering
–industrial healthcare and virtual organizations

Bioinformatics
–data integration, knowledge discovery and workflow

Healthcare
–sharing normalized mammograms

Environment
–Ocean, weather, climate modeling, sensor networks
Grid Architecture


Protocol architecture
Standards-based open architecture offers:
•
•
•
Interoperability
Services
API flexibility
Grid Architecture
Grid Architecture

Fabric Layer – ‘provides the resources to
which shared access is mediated by Grid
protocols’
•
•
Resource-specific operations
Functionality vs. simplicity
Grid Architecture

Fabric layer should provide:
•
•
Enquiry mechanisms
Resource management mechanisms
Grid Architecture

Connectivity Layer – ‘defines core
communication and authentication protocols
required for Grid-specific network
transactions’
•
•
Data exchange
Verification
Grid Architecture

Connectivity layer should provide:
•
•
•
•
Single sign on
Delegation
Integration with various local security solutions
User-based trust relationships
Grid Architecture

Resource Layer – ‘defines protocols for the
secure negotiation, initiation, monitoring,
control, accounting, and payment of sharing
operations on individual resources’
•
•
Use Fabric Layer functions
Information vs. Management protocols
Grid Architecture

Resource layer should provide:
•
•
•
Fabric layer functionality
‘exactly once’ semantics
Error reporting
Grid Architecture

Collective Layer – ‘contains protocols and
services which capture interactions across
collections of resources’
•
General vs. specific purpose
Grid Architecture

Collective layer could provide:
•
•
•
Software discovery services
Community accounting and payment services
Collaboratory services
Grid Architecture

Applications Layer – ‘comprises the user
applications that operate within a VO
environment.’
National Fusion Grid

A Collaboratory Pilot
project that is creating
and deploying
collaborative software
tools throughout the
magnetic fusion
research community
National Fusion Grid
Simple Goals




To advance scientific understanding and innovation
in magnetic fusion research by enabling more
efficient use of existing experimental facilities and
more effective integration of experiment, theory, and
modelling.
To advance scientific understanding and innovation
in fusion research
Making widespread use of Grid technologies
http://www.fusiongrid.org/
National Fusion Grid
VISION FOR THE FUSION GRID

Data, Codes, Analysis Routines, Visualization Tools should be thought
of as network accessible services

Shared security infrastructure

Collaborative nature of research requires shared visualization
applications and widely deployed collaboration technologies
— Integrate geographically diverse groups

Not focused on CPU cycle scavenging or “distributed”
supercomputing (typical Grid justifications)
— Optimize the most expensive resource - people’s time
National Fusion Grid

The problems of data sharing and rapid data
analysis the National Fusion Collaboratory
community adopted:
–
–
a common data acquisition and management
system
common relational database run-management
schema
National Fusion Grid
Geographically Diverse Community

3 Large Experimental Facilities
— Alcator, C-Mod, DIII-D
— NSTX ~$1B replacement cost

40 U.S. fusion research sites
— Over 1000 scientists in 37 state

Efficient collaboration is a requirement!
— Integrate geographically diverse groups

One future worldwide machine
— Not based in US
— US needs collaboration tools to benefit
National Fusion Grid

National Magnetic Fusion Research Community
FUSION COMMUNITY HAS 40 US SITES IN 37 STATES
National Fusion Grid
Design and Implementation of Access Grid
 Produced of both design and architecture
documents for review by public (beginning
introduction into GGF document process)
 Demonstrated full-featured prototypes in Nov
2002 at SC2002 of new venue architecture,
venue client, workspace docking complete
with application sharing
National Fusion Grid
Building the Fusion Grid (Progressive testbeds)
 Deployment Phrase
 Use Policies and Issues of Trust
 Moving to Real-Time
 Wrapping it up
Globus & the Globus Toolkit

Globus
–

Open source community focused on Grid computing
Globus Toolkit
–
–
–
Started in the late 1990’s to address common Grid
application problems
… found at www.globus.org
Includes




A set of services focused on infrastructure management
Tools for building new Web services, in Java, C, and Python
Standards-based security infrastructure
Client APIs and command line programs
Globus Toolkit & Web Services
Grid GT1
Started
far apart
in apps
& tech
Web
GT2
OGS
I
Have been
converging
HTTP
,
WSDL
WS-*
WSRF
2,
L
D
S
W
WSDM
Modeling Stateful Resources with Web
Services





Web Services Background
What is a Web Service?
Web Service Environments
A Brief Taxonomy of State and Services
Stateless Implementations, Stateful
Interfaces
Modeling Stateful Resources with Web Services
What is a Web Service?

Machine to Machine over a network via
exchange of SOAP messages

Conveyance via HTTP

Key facility in distributed environment known
as SOA
Modeling Stateful Resources with Web Services
Why Web Service in Grid Discussion?

Convergence in Grid and SOA

Many grid implementations use Web
Services
Modeling Stateful Resources with Web Services
Web Services are usually Stateless




All information needed by the service is
contained in the input message
All results are return via the output message
The service does not ‘remember’ what it just
did on completion
Not that useful for Grid
Modeling Stateful Resources with Web Services
State and Web Services

Most applications are not stateless

Grid application need their components to
keep state

Web services can be components of Grid
applications
Modeling Stateful Resources with Web Services
State and Web Services

Two general ways for representing state

The service keep track of it’s state

The service has other systems keep track of
state for it

Ideally, Option 2 preferred
Modeling Stateful Resources with Web Services
WS-Resource

Protocol for modeling stateful resources

Standards for read, update and querying of
state values.
Modeling Stateful Resources with Web Services
WS-Resource Lifecycle
Creation
Assignment & Use
Destruction
Modeling Stateful Resources with Web Services
WS-Resource Example
Modeling Stateful Resources with Web Services
WS Resource – ACID properties

Atomicity

Consistency

Isolation

Durability
References





Foster, Ian; “Globus Toolkit Version 4: Software for Service-Oriented
Systems:, IFIP International Conference on Network and Parallel
Computing, Springer-Verlag LNCS 3779, pp 2-13, 2005
Foster, Ian; “WS-Resource Framework: Globus Alliance Perspectives”,
GlobusWORLD, January 20, 2004
Foster, I., C. Kesselman, and S. Tuecke, The Anatomy of the Grid:
Enabling Scalable Virtual Organizations. International Journal of High
Performance Computing Applications, 2001. 15(3): p. 200-222.
Foster, I., Frey, J., Graham, S., Tuecke, S., Czajkowski, K., Ferguson,
D., Leymann, F., Nally, M., Storey, T. and Weerawaranna, S. Modeling
Stateful Resources with Web Services. Globus Alliance, 2004.
Keahey, K, Fredian, T., Peng, D.P. Schissel, M. Thompson, I. Foster,
M. Greenwald, D. McCune, Computational Grids in Action: The
National Fusion Collaboratory, submitted to Future Generation
Computer System, October 2002. 18(8): p. 1005-1015.