Transcript Overview of Grid Computing - Department of Computer Science

Future
Scientific Infrastructure
Ian Foster
Mathematics and Computer Science Division
Argonne National Laboratory
and
Department of Computer Science
The University of Chicago
http://www.mcs.anl.gov/~foster
Keynote Talk, QUESTnet 2002 Conference, Gold Coast, July 4, 2002
2
Evolution of Infrastructure

1890: Local power generation
– AC transmission => power Grid => economies
of scale & revolutionary new devices

2002: Primarily local computing & storage
– Internet & optical technologies => ???
[email protected]
ARGONNE  CHICAGO
3
A Computing Grid

On-demand, ubiquitous access to computing,
data, and services
“We will perhaps see the spread of ‘computer utilities’, which,
like present electric and telephone utilities, will service
individual homes and offices across the country”
(Len Kleinrock, 1969)

New capabilities constructed dynamically
and transparently from distributed services
“When the network is as fast as the computer's
internal links, the machine disintegrates across
the net into a set of special purpose appliances”
(George Gilder, 2001)
[email protected]
ARGONNE  CHICAGO
Distributed Computing+Visualization
Remote Center
Generates Tb+ datasets from
simulation code
WAN Transfer
FLASH data transferred
to ANL for visualization
GridFTP parallelism
utilizes high bandwidth
(Capable of utilizing
>Gb/s WAN links)
Chiba City
4
Visualization
code constructs
and stores
high-resolution
visualization
frames for
display on
many devices
Job Submission
Simulation code submitted to
remote center for execution
on 1000s of nodes
ActiveMural Display
Displays very high resolution
large-screen dataset animations
LAN/WAN Transfer
User-friendly striped GridFTP
application tiles the frames and
stages tiles onto display nodes
FUTURE (1-5 yrs)
• 10s Gb/s LANs, WANs
• End-to-end QoS
• Automated replica
management
• Server-side data
reduction & analysis
• Interactive portals
[email protected]
ARGONNE  CHICAGO
eScience Application:
Sloan Digital Sky Survey Analysis
[email protected]
5
ARGONNE  CHICAGO
Cluster-finding Data Pipeline
6
catalog
5
cluster
4
core
core
3
3
brg
brg
brg
brg
2
2
2
2
field
field
field
1
1
1
field
1
tsObj
[email protected]
tsObj
tsObj
tsObj
ARGONNE  CHICAGO
Chimera Application:
Sloan Digital Sky Survey Analysis
7
Size distribution of
galaxy clusters?
Galaxy cluster
size distribution
100000
Chimera Virtual Data System
+ iVDGL Data Grid (many CPUs)
10000
1000
100
10
[email protected]
1
1
10
Number of Galaxies
100
ARGONNE  CHICAGO
Grids at NASA: Aviation Safety
8
Wing Models
•Lift Capabilities
•Drag Capabilities
•Responsiveness
Airframe Models
Stabilizer Models
•Deflection capabilities
•Responsiveness
Crew Capabilities
- accuracy
- perception
- stamina
- re-action times
- SOPs
Engine Models
Human Models
•Braking performance
•Steering capabilities
•Traction
•Dampening capabilities
Landing Gear Models
[email protected]
•Thrust performance
•Reverse Thrust performance
•Responsiveness
•Fuel Consumption
ARGONNE  CHICAGO
9
Life Sciences: Telemicroscopy
DATA ACQUISITION
PROCESSING,
ANALYSIS
ADVANCED
VISUALIZATION
NETWORK
IMAGING
INSTRUMENTS
COMPUTATIONAL
RESOURCES
LARGE DATABASES
[email protected]
ARGONNE  CHICAGO
10
Business Opportunities

On-demand computing, storage, services
– Significant savings due to reduced build-out,
economies of scale, reduced admin costs
– Greater flexibility => greater productivity

Entirely new applications and services
– Based on high-speed resource integration

Solution to enterprise computing crisis
– Render distributed infrastructures manageable
[email protected]
ARGONNE  CHICAGO
11
Grid Evolution
[email protected]
ARGONNE  CHICAGO
12
Grids and Industry: Early Examples
Entropia: Distributed computing
(BMS, Novartis, …)
Butterfly.net: Grid for
multi-player games
[email protected]
ARGONNE  CHICAGO
13
Grid Infrastructure

Resources
A
– Computing, storage, data

Services
– Authentication, discovery, …

Connectivity
– Reduce tyranny of distance

A
A
Technologies
– Build applications, services

Communities
– Operational procedures, …
[email protected]
ARGONNE  CHICAGO
14
Example Grid Infrastructure Projects

I-WAY (1995): 17 U.S. sites for one week

GUSTO (1998): 80 sites worldwide, experim.

NASA Information Power Grid (since 1999)
– Production Grid linking NASA laboratories

INFN Grid, EU DataGrid, iVDGL, … (2001+)
– Grids for data-intensive science

TeraGrid, DOE Science Grid (2002+)
– Production Grids link supercomputer centers

U.S. GRIDS Center
– Software packaging, deployment, support
[email protected]
ARGONNE  CHICAGO
15
Topics in Grid Infrastructure

Regional, national, intl optical infrastructure
– I-WIRE, StarLight, APAN

TeraGrid: Deep infrastructure
– High-end support for U.S. community

iVDGL: Wide infrastructure
– Building a (international) community

Open Grid Services Architecture
– Future service & technology infrastructure
[email protected]
ARGONNE  CHICAGO
16
Topics in Grid Infrastructure

Regional, national, intl optical infrastructure
– I-WIRE, StarLight

TeraGrid: Deep infrastructure
– High-end support for U.S. community

iVDGL: Wide infrastructure
– Building a (international) community

Open Grid Services Architecture
– Future service & technology infrastructure
[email protected]
ARGONNE  CHICAGO
Targeted StarLight
Optical Network Connections
AsiaPacific
CERN
SURFnet
CA*net4
Vancouver
17
Seattle
NTON
Portland
U Wisconsin
San Francisco
Chicago
NTON
PSC
IU
NYC
NCSA
AsiaPacific
DTF 40Gb
Los Angeles
San Diego
(SDSC)
NW Univ (Chicago) StarLight Hub
I-WIRE
ANL
St Louis
GigaPoP
UICAtlanta
Atlanta
Chicago Cross connect
Ill Inst of Tech
Univ of Chicago
Indianapolis
AMPATH
AMPATH
(Abilene NOC)
NCSA/UIUC
[email protected]
www.startap.net
ARGONNE  CHICAGO
19
I-Wire Fiber Topology
Starlight
(NU-Chicago)
Argonne
18
4
Qwest
455 N. Cityfront
UC Gleacher Ctr
450 N. Cityfront
4
UIC
10
4
12
12
• Fiber Providers: Qwest,
Level(3), McLeodUSA,
360Networks
• 10 segments
• 190 route miles; 816 fiber miles
•Longest segment: 140
miles
• 4 strands minimum to each site
[email protected]
4
McLeodUSA
UIUC/NCSA
151/155 N. Michigan
Doral Plaza
Level(3)
111 N. Canal
State/City Complex
2
2
James R. Thompson Ctr
City Hall
State of IL Bldg
FNAL
(est 4q2002)
2
IIT
UChicago
Numbers indicate fiber count (strands)
ARGONNE  CHICAGO
20
I-Wire Transport
TeraGrid Linear
3x OC192
1x OC48
First light: 6/02
Starlight
(NU-Chicago)
Argonne
Starlight Linear
4x OC192
4x OC48 (8x GbE)
Operational
UC Gleacher Ctr
450 N. Cityfront
Qwest
Metro Ring
1x OC48 per site
First light: 8/02
455 N. Cityfront
UIC
McLeodUSA
UIUC/NCSA
151/155 N. Michigan
Doral Plaza
State/City Complex
James R. Thompson Ctr
City Hall
State of IL Bldg
IIT
UChicago
• Each of these three ONI DWDM systems have capacity of up to 66 channels, up to 10 Gb/s per channel
• Protection available in Metro Ring on a per-site basis
[email protected]
ARGONNE  CHICAGO
Illinois Distributed Optical Testbed
21
Northwestern
Univ-Chicago
“Starlight”
I-290
DAS-2
UI-Chicago
I-294
I-55
Illinois Inst.
Tech
Argonne Nat’l Lab
Dan Ryan Expwy
(I-90/94)
(approx 25 miles SW)
[email protected]
U of Chicago
UIUC/NCSA
Urbana (approx 140 miles South)
ARGONNE  CHICAGO
22
Topics in Grid Infrastructure

Regional, national, intl optical infrastructure
– I-WIRE, StarLight

TeraGrid: Deep infrastructure
– High-end support for U.S. community

iVDGL: Wide infrastructure
– Building a (international) community

Open Grid Services Architecture
– Future service & technology infrastructure
[email protected]
ARGONNE  CHICAGO
TeraGrid: Deep Infrastructure
www.teragrid.org
[email protected]
23
ARGONNE  CHICAGO
24
TeraGrid Objectives

Create unprecedented capability
– Integrated with extant PACI capabilities
– Supporting a new class of scientific research

Deploy a balanced, distributed system
– Not a “distributed computer” but rather …
– a distributed “system” using Grid technologies
> Computing and data management
> Visualization and scientific application analysis

Define an open and extensible infrastructure
– Enabling infrastructure for scientific research
– Extensible beyond the original four sites
> NCSA, SDSC, ANL, and Caltech
[email protected]
ARGONNE  CHICAGO
26
TeraGrid Timelines
Proposal
Submitted
To NSF
TeraGrid
Operational
Jan ‘01
McKinley
systems
Jan ‘02
Initial apps
On McKinley
TeraGrid
Prototypes
Early access
To McKinley
At Intel
TeraGrid prototype
At SC2001, 60 Itanium
Nodes, 10Gbs network
Grid Services on
Current Systems
Networking
Basic Grid svcs
Linux clusters
SDSC SP
NCSA O2K
Jan ‘03
Early McKinleys
TeraGrid clusters
at TG sites for
Testing/benchmarking
“TeraGrid Lite”
Systems and Grids
testbed
Core Grid services deployment
10Gigabit Enet testing
Operations
Advanced Grid services testing
TeraGrid Networking Deployment
TeraGrid Operations Center
Prototype
Day Ops
Production
Applications
[email protected]
ARGONNE  CHICAGO
27
Terascale Cluster Architecture
(a) Terascale Architecture Overview
(b) Example 320-node Clos Network
Myrinet
System
Interconnect
Spine
Switches
Clos mesh Interconnect
Each line = 8 x 2Gb/s links
128-port
Clos
Switches
64 hosts
64 hosts
64 hosts
64 hosts
64 hosts
Add’l Clusters,
External Networks
64 inter-switch links
64 inter-switch links
64 inter-switch links
= 4 links
64 TB
RAID
Local Display
Networks for
Remote Display
100Mb/s Switched Ethernet
Management Network
Rendered Image files
(c) I/O - Storage
[email protected]
(d) Visualization
•FCS Storage Network
•GbE for external traffic
(e) Compute
ARGONNE  CHICAGO
28
Initial TeraGrid Design
384 McKinley Processors (1.5 Teraflops, 96 nodes)
125 TB RAID storage
384 McKinley Processors (1.5 Teraflops, 96 nodes)
125 TB RAID storage
Caltech
ANL
DWDM
Optical
Mesh
SDSC
768 McKinley Processors (3 Teraflops, 192 nodes)
250 TB RAID storage
[email protected]
NCSA
2024 McKinley Processors (8 Teraflops, 512 nodes)
250 TB RAID storage
ARGONNE  CHICAGO
29
NSF TeraGrid: 14 TFLOPS, 750 TB
574p IA-32
Chiba City
256p HP
X-Class
128p HP
V2500
92p IA-32
Caltech: Data
collection analysis
HR Display &
VR Facilities
ANL: Visualization
HPSS
HPSS
WAN Architecture Options:
WAN Bandwidth Options:
•
•
•
•
•
•
•
•
Myrinet-to-GbE; Myrinet as a WAN
Layer2 design
Wavelength Mesh
Traditional IP Backbone
Abilene (2.5 Gb/s, 10Gb/s late 2002)
State and regional fiber initiatives plus CANARIE CA*Net
Leased OC48
Dark Fiber, Dim Fiber, Wavelengths
UniTree
HPSS
1024p IA-32
320p IA-64
1176p IBM SP
Blue Horizon
Myrinet
Myrinet
1500p Origin
Sun E10K
SDSC: Data-Intensive
[email protected]
NCSA: Compute-Intensive
ARGONNE  CHICAGO
34
NSF TeraGrid: 14 TFLOPS, 750 TB
574p IA-32
Chiba City
256p HP
X-Class
128p Origin
128p HP
V2500
HR Display &
VR Facilities
Caltech: Data
collection analysis
92p IA-32
HPSS
HPSS
ANL: Visualization
SDSC: Data-Intensive
UniTree
HPSS
1024p IA-32
320p IA-64
1176p IBM SP
Blue Horizon
Myrinet
Myrinet
1500p Origin
Sun E10K
NCSA: Compute-Intensive
[email protected]
ARGONNE  CHICAGO
Defining Standard Services
35
Finite set of TeraGrid servicesapplications see standard
services rather than particular
implementations…
Grid
Applications
…but sites also provide additional services
that can be discovered and exploited.
[email protected]
IA-64 Linux Cluster Runtime
Interactive Collection-Analysis Service
File-based Data Service
Volume-Render Service
IA-64 Linux Cluster Interactive Development
Collection-based Data Service
ARGONNE  CHICAGO
36
Standards  Cyberinfrastructure
• TeraGrid: focus on a finite set of service
specifications applicable to TeraGrid resources.
• If done well, other IA-64 cluster sites would adopt
TeraGrid service specifications, increasing users’
leverage in writing to the specification, and others
would adopt the framework for developing similar
services (for Alpha, IA-32, etc.)
• Note the specification should attempt to offer
improvement over general Globus runtime
environment without bogging down attempting to do
everything (for which a user is better off running
interactively!)
Certificate
Authority
Certificate
Authority
Certificate
Authority
Certificate
Authority
TeraGrid
Certificate
Authority
Grid
Applications
Grid
Info
Svces
Visualization Services
Alpha Clusters
IA-64 Linux Clusters
IA-32 Linux Clusters
Data/Information
File-based Data Service
Collection-based Data Service
Compute
Analysis
Interactive Development
Interactive Collection-Analysis Service
Runtime
Visualization Services
Relational dBase Data Service
[email protected]
ARGONNE  CHICAGO
37
Strategy: Define Standard Services

Finite number of TeraGrid Services
– Defined as specifications, protocols, APIs
– Separate from implementation

Example: File-based Data Service
– API/Protocol: Supports FTP and GridFTP,
GSI authentication
– SLA: All TeraGrid users have access to N TB
storage, available 24/7 with M% availability,
>= R Gb/s read, >= W Gb/s write, etc.
[email protected]
ARGONNE  CHICAGO
General TeraGrid Services

38
Authentication
– GSI: Requires TeraGrid CA policy and services

Resource Discovery and Monitoring
– Define TeraGrid services/attributes to be
published in Globus MDS-2 directory services
– Require standard account information
exchange to map use to allocation/individual
– For many services, publish query interface
> Scheduler: queue status
> Compute, Visualization, etc. services: attribute details
> Network Weather Service
> Allocations/Accounting Database: for allocation status
[email protected]
ARGONNE  CHICAGO
39
General TeraGrid Services

Advanced Reservation
– On-demand services
– Staging data: coordination of storage+compute

Communication and Data Movement
– All services assume any TeraGrid cluster node
can talk to any TeraGrid cluster node
– All resources support GridFTP

“Hosting environment”
– Standard software environment
– More sophisticated dynamic provisioning issues
not yet addressed
[email protected]
ARGONNE  CHICAGO
41
Topics in Grid Infrastructure

Regional, national, intl optical infrastructure
– I-WIRE, StarLight

TeraGrid: Deep infrastructure
– High-end support for U.S. community

iVDGL: Wide infrastructure
– Building a (international) community

Open Grid Services Architecture
– Future service & technology infrastructure
[email protected]
ARGONNE  CHICAGO
42
iVDGL: A Global Grid Laboratory
“We propose to create, operate and evaluate, over a
sustained period of time, an international research
laboratory for data-intensive science.”
From NSF proposal, 2001

International Virtual-Data Grid Laboratory
– A global Grid laboratory (US, Europe, Asia, South
America, …)
– A place to conduct Data Grid tests “at scale”
– A mechanism to create common Grid infrastructure
– A laboratory for other disciplines to perform Data Grid
tests
– A focus of outreach efforts to small institutions

U.S. part funded by NSF (2001-2006)
– $13.7M (NSF) + $2M (matching)
[email protected]
ARGONNE  CHICAGO
43
Initial US-iVDGL Data Grid
SKC
BU
Wisconsin
PSU
BNL
Fermilab
Indiana
Caltech
JHU
UCSD
Other sites to be
added in 2002
[email protected]
Hampton
Florida
Brownsville
Tier1 (FNAL)
Proto-Tier2
Tier3 university
ARGONNE  CHICAGO
iVDGL:
International Virtual Data Grid Laboratory
44
Tier0/1 facility
Tier2 facility
Tier3 facility
10 Gbps link
2.5 Gbps link
622 Mbps link
Other link
[email protected]
 CHICAGO
U.S. PIs: Avery, Foster, Gardner, Newman, SzalayARGONNE
www.ivdgl.org
45
iVDGL Architecture
(from proposal)
[email protected]
ARGONNE  CHICAGO
46
US iVDGL Interoperability

US-iVDGL-1 Milestone (August 02)
iGOC
US-iVDGL1
Aug 2002
ATLAS
SDSS/NVO
CMS
LIGO
1
1
2
2
1
2
[email protected]
1
2
ARGONNE  CHICAGO
47
Transatlantic Interoperability

iVDGL-2 Milestone (November 02)
iGOC
Outreach
iVDGL-2
DataTAG
Nov 2002
ATLAS
SDSS/NVO
CMS
ANL
HU
IU
LBL
UC
CS Research
BNL
BU
LIGO
FNAL
CIT
CIT
FNAL
CERN
PSU
INFN
ANL
UTB
UK PPARC
UC
UWM
U of A
UCSD
UF
JHU
UM
UCB
OU
IU
UTA
ISI
NU
UW
[email protected]
ARGONNE  CHICAGO
48
Topics in Grid Infrastructure

Regional, national, intl optical infrastructure
– I-WIRE, StarLight

TeraGrid: Deep infrastructure
– High-end support for U.S. community

iVDGL: Wide infrastructure
– Building a (international) community

Open Grid Services Architecture
– Future service & technology infrastructure
[email protected]
ARGONNE  CHICAGO
“Standard” Software Infrastructure:
Globus ToolkitTM

49
Small, standards-based set of protocols for
distributed system management
– Authentication, delegation; resource
discovery; reliable invocation; etc.

Information-centric design
– Data models; publication, discovery protocols

Open source implementation
– Large international user community

Successful enabler of higher-level services
and applications
[email protected]
ARGONNE  CHICAGO
50
Example Grid Projects in eScience
[email protected]
ARGONNE  CHICAGO
51
The Globus Toolkit in One Slide

Grid protocols (GSI, GRAM, …) enable resource
sharing within virtual orgs; toolkit provides reference
implementation ( = Globus Toolkit services)
MDS-2
(Monitor./Discov. Svc.)
Reliable
remote
GSI User
invocation Gatekeeper Reporter
(Grid
(registry +
Authenticate &
(factory)
Security create proxy
discovery)
Create process Register
Infrastruc- credential
ture)
User
process #1
Proxy
User
process #2
Proxy #2
GRAM
(Grid Resource Allocation & Management)

Soft state
registration;
enquiry
Other GSIauthenticated
remote service
requests
GIIS: Grid
Information
Index Server
(discovery)
Other service
(e.g. GridFTP)
Protocols (and APIs) enable other tools and services
for membership, discovery, data mgmt, workflow, …
[email protected]
ARGONNE  CHICAGO
52
Globus Toolkit: Evaluation (+)

Good technical solutions for key problems, e.g.
– Authentication and authorization
– Resource discovery and monitoring
– Reliable remote service invocation
– High-performance remote data access

This & good engineering is enabling progress
– Good quality reference implementation, multilanguage support, interfaces to many systems,
large user base, industrial support
– Growing community code base built on tools
[email protected]
ARGONNE  CHICAGO
53
Globus Toolkit: Evaluation (-)

Protocol deficiencies, e.g.
– Heterogeneous basis: HTTP, LDAP, FTP
– No standard means of invocation, notification,
error propagation, authorization, termination, …

Significant missing functionality, e.g.
– Databases, sensors, instruments, workflow, …
– Virtualization of end systems (hosting envs.)

Little work on total system properties, e.g.
– Dependability, end-to-end QoS, …
– Reasoning about system properties
[email protected]
ARGONNE  CHICAGO
54
Globus Toolkit Structure
Service naming
Soft state
management
Reliable invocation
GRAM
Notification
MDS
GSI
GridFTP
MDS
???
GSI
GSI
Data
Resource
Other Service
or Application
Job
manager
Job
manager
Compute
Resource
Lots of good mechanisms, but (with the exception of GSI) not that easily
incorporated into other systems
[email protected]
ARGONNE  CHICAGO
Grid Evolution:
Open Grid Services Architecture




55
Refactor Globus protocol suite to enable
common base and expose key capabilities
Service orientation to virtualize resources
and unify resources/services/information
Embrace key Web services technologies for
standard IDL, leverage commercial efforts
Result: standard interfaces & behaviors for
distributed system management: the Grid
service
[email protected]
ARGONNE  CHICAGO
Open Grid Services Architecture:
Transient Service Instances

56
“Web services” address discovery & invocation
of persistent services
– Interface to persistent state of entire enterprise

In Grids, must also support transient service
instances, created/destroyed dynamically
– Interfaces to the states of distributed activities
– E.g. workflow, video conf., dist. data analysis

Significant implications for how services are
managed, named, discovered, and used
– In fact, much of OGSA (and Grid) is concerned
with the management of service instances
[email protected]
ARGONNE  CHICAGO
57
Open Grid Services Architecture

Defines fundamental (WSDL) interfaces and
behaviors that define a Grid Service
– Required + optional interfaces = WS “profile”
– A unifying framework for interoperability &
establishment of total system properties

Defines WSDL extensibility elements
– E.g., serviceType (a group of portTypes)

Delivery via open source Globus Toolkit 3.0
– Leverage GT experience, code, community

And commercial implementations
[email protected]
ARGONNE  CHICAGO
The Grid Service =
Interfaces/Behaviors + Service Data
Service data access
Explicit destruction
Soft-state lifetime
Binding properties:
- Reliable invocation
- Authentication
GridService
(required)
Service
data
element
… other interfaces …
(optional)
Service
data
element
58
Standard:
- Notification
- Authorization
- Service creation
- Service registry
- Manageability
- Concurrency
Service
data
element
Implementation
+ applicationspecific interfaces
Hosting environment/runtime
(“C”, J2EE, .NET, …)
[email protected]
ARGONNE  CHICAGO
Grid Service Example:
Database Service

A DBaccess Grid service will support at
least two portTypes
Grid
– GridService
Service
– DBaccess

59
Each has service data
DBaccess
Name, lifetime, etc.
DB info
– GridService: basic introspection
information, lifetime, …
– DBaccess: database type, query languages
supported, current load, …, …

Maybe other portTypes as well
– E.g., NotificationSource (SDE = subscribers)
[email protected]
ARGONNE  CHICAGO
60
Example:
Data Mining for Bioinformatics
Community
Registry
Mining
Factory
Database
Service
BioDB 1
User
Application
“I want to create
a personal database
containing data on
e.coli metabolism”
Compute Service Provider
.
.
.
.
.
.
Database
Service
Database
Factory
BioDB n
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
61
Example:
Data Mining for Bioinformatics
“Find me a data Community
mining service, and Registry
somewhere to store
data”
Mining
Factory
Database
Service
BioDB 1
User
Application
Compute Service Provider
.
.
.
.
.
.
Database
Service
Database
Factory
BioDB n
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
62
Example:
Data Mining for Bioinformatics
GSHs for Mining
and Database
factories
User
Application
Community
Registry
Mining
Factory
Database
Service
BioDB 1
Compute Service Provider
.
.
.
.
.
.
Database
Service
Database
Factory
BioDB n
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
63
Example:
Data Mining for Bioinformatics
Community
Registry
“Create a data mining
service with initial
lifetime 10”
User
Application
“Create a
database with initial
lifetime 1000”
Mining
Factory
Database
Service
BioDB 1
Compute Service Provider
.
.
.
.
.
.
Database
Service
Database
Factory
BioDB n
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
64
Example:
Data Mining for Bioinformatics
Community
Registry
“Create a data mining
service with initial
lifetime 10”
User
Application
“Create a
database with initial
lifetime 1000”
Mining
Factory
Database
Service
Miner
BioDB 1
Compute Service Provider
.
.
.
.
.
.
Database
Service
Database
Factory
BioDB n
Database
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
65
Example:
Data Mining for Bioinformatics
Community
Registry
Mining
Factory
Query
Miner
User
Application
Database
Service
BioDB 1
Compute Service Provider
.
.
Query
.
.
.
.
Database
Service
Database
Factory
BioDB n
Database
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
66
Example:
Data Mining for Bioinformatics
Community
Registry
Keepalive
Mining
Factory
Query
Miner
BioDB 1
Compute Service Provider
.
.
Query
.
User
Application
Keepalive
Database
Service
.
.
.
Database
Service
Database
Factory
BioDB n
Database
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
67
Example:
Data Mining for Bioinformatics
Community
Registry
Keepalive
User
Application
Keepalive
Mining
Factory
Database
Service
Miner
BioDB 1
Compute Service Provider
.
.
.
.
.
.
Results
Database
Service
Database
Factory
Results
BioDB n
Database
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
68
Example:
Data Mining for Bioinformatics
Community
Registry
User
Application
Keepalive
Mining
Factory
Database
Service
Miner
BioDB 1
Compute Service Provider
.
.
.
.
.
.
Database
Service
Database
Factory
BioDB n
Database
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
69
Example:
Data Mining for Bioinformatics
Community
Registry
Mining
Factory
Database
Service
BioDB 1
Compute Service Provider
.
.
.
User
Application
Keepalive
.
.
.
Database
Service
Database
Factory
BioDB n
Database
Storage Service Provider
[email protected]
ARGONNE  CHICAGO
GT3: An Open Source OGSACompliant Globus Toolkit

70
GT3 Core
– Implements Grid service
interfaces & behaviors
– Reference impln of
evolving standard
– Multiple hosting envs:
Java/J2EE, C, C#/.NET?

GT3 Base Services
GT3
Data
Services
Other Grid
Services
GT3 Base Services
GT3 Core
– Evolution of current
Globus Toolkit capabilities

Many other Grid services
[email protected]
ARGONNE  CHICAGO
OGSA Definition and Delivery
(Very Approximate!!)
71
Other Systems
T
I
M
E
GGF
OGSI
WG
Grid Service
Specification
GGF
WGs
Other core specs:
• Security
• Res. Mgmt.
• Etc.
GGF
WGs
Other specs:
• Databases
• Etc.
• Etc.
[email protected]
Prototype
Feedback
Globus OGSI
Reference Impln
Globus Toolkit
Version 3
Other OGSAbased software
ARGONNE  CHICAGO
Q2: What Higher-Level Services?
[email protected]
72
ARGONNE  CHICAGO
Summary:
Grid Infrastructure

73
Grid applications demand new infrastructure
beyond traditional computers and networks
– Network-accessible resources of all types
– High-speed networks
– Services and operational procedures
– Software technology for building services
(which must also be treated as infrastructure)

TeraGrid, iVDGL, StarLight, DOT
– Connections to international sites?
[email protected]
ARGONNE  CHICAGO
Summary:
Open Grid Services Architecture





74
Open Grid Services Architecture represents
(we hope!) next step in Grid evolution
Service orientation enables unified treatment
of resources, data, and services
Standard interfaces and behaviors (the Grid
service) for managing distributed state
Deeply integrated information model for
representing and disseminating service data
Open source Globus Toolkit implementation
(and commercial value adds)
[email protected]
ARGONNE  CHICAGO
For More Information

75
Survey + research articles
– www.mcs.anl.gov/~foster




I-WIRE: www.iwire.org
TeraGrid: www.teragrid.org
iVDGL: www.ivDGL.org
The Globus Project™
– www.globus.org

GriPhyN project
– www.griphyn.org

Global Grid Forum
– www.gridforum.org
– www.gridforum.org/ogsi-wg
– Edinburgh, July 22-24
– Chicago, Oct 15-17
[email protected]
ARGONNE  CHICAGO