Transcript SERVOGrid_ACESJuly-04

SERVO Grid:
Solid Earth Research Virtual Observatory
Grid/Web Services and Portals
Supporting Earthquake Science
July 13 2004
Fourth ACES APEC Cooperation for
Earthquake Simulation Workshop
Beijing China
Geoffrey Fox, Marlon Pierce
Community Grids Lab,
Pervasive Technologies Laboratories
Indiana University
What is a Grid?
• You won’t find a clear description of what is Grid and how does differ
from a collection of programs on the Internet
• Grids were once defined as “Internet Scale Distributed Computing”
but this isn’t very good as Grids depend as much if not more on data
as well as simulations
• So I will use: Grids are “Internet Scale Distributed Services” and
represent a way of collecting services together in same way that a
program (package) collects methods and objects together on a
sequential machine.
• In this view, Grids are naturally and critically tied to Web Services and
are built on top of Web service standards
• The high performance computing and e-Science origin of Grids does
give some special challenges
– Streams, State, Dataflow, High Performance data transfer, link to
parallel computers, cross administrative domain access
• Grids are built with Web Services and so a Grid Service is a Web
Service and the differences are quite small and in particular irrelevant
for current version of SERVO
Programs
Computational resources
service logic
BPEL, Java, .NET
Databases
SOAP and WSDL
• Use “Pure” Web Services
which can be augmented
when debate between
Microsoft/IBM on
WSRF/WS-GAF
• Note Web Services still
evolving
• >60 proposed Web Service
specifications competing to
be “standards”
– Only 4.5 out of 60 have
made it to Industry WS-I
interoperability suite
• But essential idea of services
and their interconnection
with messages will not
change!
Humans
<env:Envelope>
<env:Header>
...
</env:header>
<env:Body>
...
</env:Body>
</env:Envelope>
SOAP messages
message processing
SERVO Strategy
resources
Devices
Some Grid Controversies
• 1) There are several proposals for the Web Service
extensions needed for Grids – why do we ignore?
–
–
–
–
OGSI (GT3)
WSRF (GT4)
WS-GAF (Newcastle)
WS-I+ (Pure Web Services)
• We use WS-I+ approach – can later add extensions when
consensus clear
– This approach adopted by next phase of UK e-Science Program
• 2) Web Services are too slow as use HTTP with clumsy
ASCII XML data (SOAP)?
– Negotiate with “slow clumsy” but totally interoperable messages
– Agree on very fast data stream protocol and encoding
– i.e. separate control channel from data channel
SERVOGrid Requirements
• Seamless Access to Data repositories and large scale
computers
• Integration of multiple data sources including sensors,
databases, file systems with analysis system
– Including filtered OGSA-DAI (Grid database access)
• Rich meta-data generation and access with SERVOGrid
specific Schema extending openGIS (Geography as a Web
service) standards and using Semantic Grid
• Portals with component model for user interfaces and web
control of all capabilities
• Collaboration to support world-wide work
• Basic Grid tools: workflow and notification
• Not metacomputing
iSERVO Strategy
• Agree on what (type of) resources and capabilities need to put on the
ISERVO Grid
– Computers, instruments, databases, visualization, maps, job
submittal ….
• Agree on interfaces to resources from OGSA-DAI (databases) to
particular data structures (GML/OpenGIS) – specify in XML
• Implement Resources and Capabilities as Services
– User Interface should be a portlet that can be integrated by the
portal into web interface
• Make certain overarching Grid capabilities such as workflow,
federation and metadata are sufficient
• SERVO Grid is a prototype of this strategy using several US sites
rather than several countries
– Can be naturally extended to iSERVO, education, emergency
response by extending resources
• Web Service Architecture ensures continued interoperability and
extensibility
(i)SERVO Web (Grid) Services
• Programs: All applications wrapped as Services using proxy strategy
• Job Submission: supports remote batch and shell invocations
– Used to execute simulation codes (VC suite, GeoFEST, etc.), mesh generation
(Akira/Apollo) and visualization packages (RIVA, GMT).
• File management:
– Uploading, downloading, backend crossloading (i.e. move files between remote
servers)
– Remote copies, renames, etc.
• Job monitoring
• Workflow: Apache Ant-based remote service orchestration (NCSA)
– Move towards a BPEL framework (can still implement with ANT)
• Database services: support SQL queries
– Expect Simpler version of OGSA-DAI (“Web Service-DAI”) Grid Database
• Data services: support interactions with XML-based fault and surface
observation data.
– For simulation generated faults (i.e. from Simplex)
– XML data model being adopted for common formats with translation services to
“legacy” formats.
– Migrating to Geography Markup Language (GML) descriptions.
SERVOGrid Application Descriptions
• Codes range from simple “rough estimate” codes to parallel, high
performance applications.
– Disloc: handles multiple arbitrarily dipping dislocations (faults) in an elastic
half-space.
– Simplex: inverts surface geodetic displacements for fault parameters using
simulated annealing downhill residual minimization.
– GeoFEST: Three-dimensional viscoelastic finite element model for
calculating nodal displacements and tractions. Allows for realistic fault
geometry and characteristics, material properties, and body forces.
– Virtual California: Program to simulate interactions between vertical strikeslip faults using an elastic layer over a viscoelastic half-space
– RDAHMM: Time series analysis program based on Hidden Markov Modeling.
Produces feature vectors and probabilities for transitioning from one class to
another.
– PARK: Boundary element program to calculate fault slip velocity history
based on fault frictional properties.a model for unstable slip on a single
earthquake fault.
• Preprocessors, mesh generators
• Visualization tools: RIVA, GMT
SERVOGrid Codes, Relationships
Elastic Dislocation Inversion
Viscoelastic FEM
Viscoelastic Layered BEM
Elastic Dislocation
Pattern Recognizers
Fault Model BEM
This linkage called Workflow in Grid/Web Service parlance
Role of Workflow
Service-1
Service-3
Service-2





Programming the Grid: Workflow describes linkage
between services
As distributed, linkage must be by messages
Linkage is two-way and has both control and data
Apply to multi-scale (complexity) linkage, multiprogram linkage, link visualization to simulation,
GIS to simulations and viz filters to each other
Microsoft-IBM specification BPEL is current
preferred Web Service XML specification of
workflow
Security: Authentication and Authorization
• Authentication describes who the user is
• Authorization describes what a given user can do
– What data and computers can be accessed
– Basically a database
• Current portal uses password accounts and provides services for free for
demonstration.
– iSERVO should decide on “charging for” services
• We have (through Community portal effort OGCE) support for GSI and
Kerberos authentication services.
– These just plug in and replace the default login service.
• Authorization is currently simple: you can only reach your files.
– iSERVO should develop an authorization policy
• Simultaneous Cross Administrative Domain access is a very hard Grid
problem and no consensus as to good solution
• Systematic use of Services helps security/privacy/IP issues as “danger
of misuse” is lower for services (which have limited privileges) than for
direct computer access
Each Service
has its own
portlet
Individual
portlet for the
Proxy Manager
Use tabs or
choose
different
portlets to
navigate
through
interfaces to
different
services
2 Other Portlets
OGCE
Consortium
Important Lessons/Principles







Use OGCE Portal Architecture and portal services
Can expect GGF activities like OGSA to define/refine interfaces and
projects around the world to produce more powerful services
• Obsolescing of implementations is a consequence of interoperability
Use Grids of Grids of Services Architecture
• Interoperable Component Grids Built from interoperable services
• Collaboration, Compute, Database, GIS, Sensor, Visualization Grids
Build a GIS (Geographical Information Systems) Grid spanning
simulation/crisis management and different fields with openGIS
compliance
• openGIS has defined Web Service Interfaces
• Visualization should build on these
Geoscience Education Grid by transformations on research grid
Emergency Response and Planning Grids by adding real-time
control/collaboration and GIS tools
• These additions common to all crises
Collaboration between Beihang University and Indiana University to
produce Web Service based audio/video conferencing
Earthquake CIGrid
…
Electricity
CIGrid
…
Flood Services
and Filters
Earthquake
Services
Collaboration Grid
Sensor Grid
Registry
Flood CIGrid
Portals
GIS Grid
Data Access/Storage
Visualization Grid
Compute Grid
Metadata
Core Grid Services
Security
Notification
Workflow
Messaging
Physical Network
Critical Infrastructure (CI) Grids built as Grids of Grids of Services
Repositories
Federated Databases
Database
Sensors
Streaming
Data
Field Trip Data
Database
Sensor Grid
Database Grid
Research
SERVO Grid
Compute Grid
Data
Filter
Services Research
Simulations
?
GIS
Discovery Grid
Services
Analysis and
Visualization
Portal
Earthquake Research and Education Grids
Education
Customization
Services
From
Research
to Education
Education
Grid
Computer
Farm
Further iSERVO Challenges
• Make everything a Service
• Think about Data Curation
– Set up policies for observational data and criteria for inclusion
in iSERVO data repositories
• Think about Data Provenance
– Generate and maintain metadata describing ownership, origins
and transformations
– Applies to both “experimental data” and results from
simulations (visualizations)
• Curation and Provenance change in research
methodologies and requires funding!
• Education and Emergency Response/Planning interesting
offshoots of iSERVO
QuakeSim Portal Shots
Session Archive Service
A Restored Session
Modify Stored Session Values
Geometry and Mesh Plotting Service
(and the Mesh Generation Service)
Interactive Job Submission
Archive and File Download Services
Non-Interactive Job Submission
Job Monitoring Service
Some IDL Plots of GeoFEST Output
Some GMT Services and Plots
iSERVO Example: Finley
• Finley is a finite element code being developed by
the QUAKES group at the University of
Queensland.
• Compatible with GeoFEST-style geometry models
and mesh generation tools.
– So we can reuse the services we wrapped for GeoFEST.
• The Finley application itself is a separate service
and also has a separate visualization service.
Setting Up Finley Simulation of
Northridge
Selected Fault
Components
Select Fault from
USC database
Run Finley, Retrieve Generate Movie