Transcript WS_Grid_Recommendations_Wilson

Web & Grid Services Track Summary
Coordinators: Brian Wilson, Paul Davis
Contributors: Mike Smorul, Gary Jackson, Rudy Husar,
Heather Weir, Michael Burnett, Yonsook Enloe, and many others
Web & Grid Services Track
Promo

We’ll Answer These Provocative Questions
:
Carbon Cycle
 Is SOAP the universal "glue" for distributed scientific
data processing? The IT world seems to think so.

Will Web and Grid services ever converge (as in
OGSI and WSRF)? Yes.

What exactly can one do with a Data Grid? Plenty.

Can I really use the Globus Toolkit to find and use
compute resources on the Grid? Yes.

How do I do distributed computing and still sleep at
night? (i.e., with security, authentication,
authorization, single sign-on, reliability, traceability,
data provenance, load balancing, etc.)
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
2
Web & Grid Services Track
WS Summary
• SOAP, WS, and Service Collaborations
– Significant interest in and use of Simple Object
Access Protocol (SOAP) for accessing services
and WSDL for describing them.
– Significant work has already been done on
creating a taxonomy of services and a service
registry.
• GCMD SERF documents & services catalog
• ECHO service taxonomy & UDDI registry
– Promote service collaborations (service chains)
within the Federation by developing pilot projects.
– WSRF and WS-* Standards are still evolving.
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
3
Web & Grid Services Track
WS Recommendations
• Use the GCMD SERF catalog for Federation service
registry & discovery.
– As opposed to creating an independent Federation survey of
available services.
– Many ESIP’s have already written SERF documents.
– Modify/add XML tags to the SERF document to support URL
links to WSDL and XML schema docs.
– Promote the GCMD SERF Activity
• Have SERF Fest at the next ESIP Federation Meeting.
• Publish “TOP 20” list of services discovered and called via the
GCMD services catalog.
– Coordinators: Comments/suggestions to Heather Weir at
GCMD and/or Brian Wilson.
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
4
Web & Grid Services Track
WS Recommendations
• Create a “test bed” for registry and discovery
of Federation SOAP/WSDL services.
– ECHO will support a UDDI registry in v7.
– Mature our understanding of shared service
descriptions.
– Promote interoperability and links between SERF
documents, service taxonomies, and UDDI
registries.
– Coordinator: Michael Burnett.
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
5
Web & Grid Services Track
WS Recommendations
• Start a Federation “Service Collaboration” Cluster to create a
Federation Service-Oriented Architecture (SOA) built on
contemporary Web Service technologies (SOAP/WSDL).
– Create service collaborations (chains) within the Federation.
– Develop a taxonomy of services (data discovery, query,
access, etc.) for SOAP purposes (leveraging work by ECHO
and others).
– Use the taxonomy to design a standard set of Federation
SOAP service interfaces.
– Explore the problems of tying together independently
developed services with varying interfaces (semantic
mediation).
– Demonstrate interoperability between OpenDAP, OGC
WMS/WCS and SOAP services.
– Organizer: Brian Wilson.
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
6
Web & Grid Services Track
Grid Services Summary
• Data Grid
– Storage Resource Broker (SRB) is stable and usable.
– A SRB-based Federation Data Grid exists (3 site consortium
led by U. Maryland)
– Globus also provides “Data Grid” services (MCS, RLS, RFT).
• Grid Services
– WS-* Standards are still evolving.
– Globus Alliance Toolkit is evolving rapidly.
• Globus v2.4 in production use, but not WS based.
• Globus v3.2 works (OGSI spec.) but has been superseded.
• Globus v4 (WSRF spec.) is not available yet (beta, Mar 2005).
• Further technology exploration (testbed activity) will
be required.
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
7
Web & Grid Services Track
Grid Services Recommendations
• Expand participation in the SRB-based
Federation Data Grid
– Write a “white paper” describing the benefits of
joining the Fed. Data Grid (WHY-TO doc).
– Create a HOW-TO document for installing SRB
and joining the Fed. Data Grid.
– Review and report back about advances in Globus
services & maturity (v4).
– Presentation, with a demo, at the next ESIP
Federation Meeting.
– Coordinator: Paul Davis.
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
8
Web & Grid Services Track
Grid Services Recommendations
• Investigate WS Choreography / Grid Workflow
technologies.
– One activity within the new Services Collaboration
cluster.
– Join Grid Workflow activities & discussions at
Global Grid Forum (GGF) meetings.
• NASA & Federation input to GGF world.
• Earth Science is a new application domain for
GGF workflow efforts.
– Coordinators: Brian Wilson, Yonsook Enloe.
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
9
Web & Grid Services Track
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
10
Web & Grid Services Track
Data Query/Access Services
 Services Chain goes beyond simple data access
–
–
–
–
–
–
–
–
–
–
–
Time & Geolocation Query  yielding data inventory
Query on Quality Flags
General Metadata Query  catalog satisfying conditions
Data Access - by object or granule ID and variable name
Data Slicing - as in OpenDAP
Data Subsetting – by lat, lon, alt, & time ranges
Parameter Subsetting – select only desired physical variables
Data Reformatting – choose output format as in WMS/WCS
On-Demand Grid Computations – grid diff in GraDS/DODS
Variable Bundles – external metadata from scientist
Return Composite Data Objects
• 4DRectilinearGrid, 4DCurvilinearGrid (swath)
• XML & Native Binary representations
– User Composites – add custom object to data model
– Semantic Support
• Use generic variable names
• Reason about variables to be bundled
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
11
Web & Grid Services Track
SciFlo Data Query/Access (SOAP) Services
• QueryByGeoLocation(startTime, endTime, lat, lon,
timeTolerance, distanceTolerace, variable, metadataGroups)
– Returns granule ID’s and geolocation info. for AIRS L2 swaths
that intersect lat/lon point or are near enough.
• QueryByGeoRegion(startTime, endTime, lat/lon region, . . .)
– Returns granule ID’s and geolocation info. for AIRS L2 swaths
that intersect the lat/lon region or are near enough.
• QueryByMetadata(variable, ListOfConstraints, groups)
– Returns granule ID’s and selected metadata for AIRS granules
that satisfy the metadata constraint expression:
(min1 <= field1 <= max1 and/or min2 <= field2 <= max2 . . .).
• FindDataById(IdList, UrlOptions)
– Given list of unique ID’s, returns list of ftp, http, or DODS URL’s
pointing to the granules (files). Uses cache and redirection
server.
• -- Other semantic interfaces possible
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
12
Web & Grid Services Track
SciFlo Data Query/Access (SOAP) Services
• GetMetadata(type, groups)
– Returns metadata describing scientific domain, dataset info, list
of metadata fields, generic name translation table, location of
XML schema documents, location of related ontologies
(OWL/RDF), etc.
• GetHelp(type, groups)
– A SOAP service that itself returns help documentation describing
the available SOAP services.
• --These Services Combined Can Support:
–
–
–
–
Wilson, 01/04/2005
Data Catalog, General Query, Data Access, Generic Names,
Data Discovery by Domain and Dataset Keywords
Type checking via XML schemas
Hooks to semantic web
ESIP Federation Meeting, Jan. 4-6, 2005
13
Web & Grid Services Track
Data Discovery to Inventory Level
• SOAP Services just described provide:
–
–
–
–
Domain description Metadata
Dataset Info. and Keywords
Generic VariableName Lookup Table
Query by Metadata or just Time & GeoRegion
• Layer more semantics on top of these services
–
–
–
–
Dataset Discovery by keyword search
Bind in additional metadata provided by scientist
Tie ontologies into keywords and genericVariableNames
Semantic Inference: AIRS and MODIS both provide
atmosphericTemperature and cloudProperties. Hmmm.
Compare them.
– XML/SOAP provides substrate; possibilities only limited
by ontology development.
Wilson, 01/04/2005
ESIP Federation Meeting, Jan. 4-6, 2005
14