Transcript Exercising WCS Using Python Libraries

Using python client libraries to
access remote servers
via Web Coverage Services
Dominic Lowe, BADC
Ben Domenico, Unidata/UCAR*
and
the GALEON Team
*Work partially supported by US NSF Atmospheric Science Division
Working Together on
A Mosaic for Atmospheric Data
This presentation describes and draws on the
work* of many collaborating individuals and
institutions
* Unidata’s contribution supported by the U.S. National Science
Foundation
Ostia Antica circa 7 BC
Acronym Glossary
• GALEON (Geo-interface for Air, Land,
Environment, Oceans NetCDF)
• FES (Fluid Earth Systems, mainly the data
systems of the atmospheric and ocean
sciences)
• http://www.unidata.ucar.edu/content/public
ations/acronyms/glossary.html
Outline
• GALEON Background
• OWSLib – OGC Web Services Library
• Simple Exercises for WCS Using
OWSLib Clients
• Participating Institutions and Servers
• Future Directions
• References
Reconciling Disparate Data Models:
Different Ways of Thinking about Data
• To the GIS (solid earth and societal impacts)
community, the world is:
– A collection of static features (e.g., roads, lakes, plots of land) with
geographic footprints on the Earth (surface).
– The features are discrete objects with attributes which can be stored
and manipulated conveniently in a database.
• To the fluids (atmosphere and oceans) communities,
the world is:
– A set of parameters (e.g., pressure, temperature, wind speed) which
vary as continuous functions in 3-dimensional space and time.
– The behavior of the parameters in space and time is governed by a
set of equations.
– Data are simply discrete points in the mathematical function space.
Taking Advantage of Web Services for
Data System Interoperability
FES Client
Applications
GIS Client
Applications
OGC or
proprietary GIS
protocols
OpenGIS Protocols:
WMS, WFS, WCS, CSW
GIS Servers
GIS Server
Hydrologic,
demographic,
GIS Server
infrastructure, societal
impacts, … datasets
OGC or THREDDS,
OPeNDAP, ADDE. FTP…
protocols
FES Servers
THREDDS Server
THREDDS
Server
Satellite,
radar,
forecast model output, …
datasets
Traditional GIS view
Attributes
in DBMS
tables
Features
as points,
lines,
polygons
Typical NetCDF Visualization
OGC GALEON
• Phase 1:
– Many successful experiments with WCS 1.0
– Gridded data
• Phase 2:
– WCS 1.1 (and later) plus CS-W for catalogs
– Non-gridded data collections via other Services
(WFS, SOS)
– Dearth of client implementations slows progress
• OGC Network Rather than Interoperability
Experiment
Apply GIS Tools To
Atmospheric Science Data
OWSLib –
OGC Web Service utility library
• is a lightweight package for working with
OGC map, feature, and coverage services
using Python
• provides a common API for accessing
service metadata and wrappers for
GetCapabilities, GetMap, GetCoverage
and GetFeature requests
• facilitates implementation of simple WCS
clients as python scripts
OWSLib –
Design Goals
• Instantiate an OGC web service proxy (ala
xmlrpclib)
• Make a GetCapabilities request and marshal the
response into a Python structure that models
OWS Common metadata
• Proxy GetMap/GetFeature/GetCoverage/Get*
requests to the service with Pythonic wrappers
for those operations
• Note that OWS common is an initiative to
separate out common elements of OGC web
services -- likely important as GALEON attempts
to serve non-gridded data
OWSLib History
• Originally developed by Sean Gillies as part of the
Python Cartographic Library.
• Version 0.2 contained support for accessing WMS
version 1.1.1 and WFS 1.0.0 servers.
• Version 0.3 developed by Sean Gillies & Dominic Lowe
introduced support for WCS versions 1.0.0 and 1.1.0.
• Version 0.3 also introduced a harmonized OWS
Common-like API to service metadata that is common
across WCS, WMS and WFS. This API is a 'pythonic'
interpretation of OWS-Common which provides easy
access to the OWS-common metadata.
OWSLib Design Decisions
• Lightweight
• Easy to use
• Modelled on OWS-common but not verbatim;
“pythonic beats pedantic as long as significant
meaning is not lost”
• Use python idioms:
e.g. convenient dictionary like access to
coverage metadata: wcs['temperature']
• Harmonize metadata between services where
not harmonized (and where possible) in line with
intentions of OWS common.
Rudimentary WCS Exercises
via OWSLib Python Scripts
• Use getCapabilities for list of coverages
• Extract name of first coverage in list
• If multiple times are available, use the
initial time
• Use getCoverage to request first coverage
• Use defaults wherever possible
• Focus on whether protocol works
• Worry about “correctness” of the returned
coverages later
WCS Sites
•
•
•
•
•
•
•
•
•
•
•
US Pacific Fisheries Environmental Lab
US Unidata THREDDS Data Server
US USGS Coast Environmental
US NCDC NOMADS
Netherlands KNMI Geoservices
Netherlands ARGOSS
UK BADC
US NSIDC
US George Mason U.
Italy U. of Florence CNR
US NNEW Weather.aero
General Results
• 3 servers returned a coverage on the first try
• 2 required minor changes to the default
parameter list
• The rest required interactions between Dominic,
Ben, and the WCS site administrator
• 2 cases of missing required parameters
• 2 cases uncovered bugs in python client library
which were subsequently fixed
• All cases working in the end
Future Enhancements
• Include more servers in the exercise
• Augment exercise in the direction of a true
test
• Retrieve representative coverages
• Retrieve representative times
• Use bounding box other than default
• Examine returned coverage
• Track evolution of WCS specification
More Inforamtion
• OWSLib wiki:
http://trac.gispython.org/lab/wiki/OwsLib
• Subversion Repository: svn co
http://svn.gispython.org/svn/gispy/OWSLib/trunk
• Python Cheeseshop:
http://pypi.python.org/pypi/OWSLib/0.3
• GALEON Wiki page:
https://sites.google.com/site/galeonteam/Home/
GALEON%20WCS%20"Tests"