Transcript ODIP 3 prototype progress presentation

Helen Glaves (NERC- BGS), Dick Schaap (MARIS),
Robert Arko (LDEO) and Roger Proctor (IMOS)
•
•
•
Paradigm shift from
traditional discipline
based marine research
Multidisciplinary
ecosystem level
approach: promoted in
Europe by Marine
Strategy Framework
directive MSFD (2008)
and Marine Knowledge
2020
Requires large
amounts of good
quality, interoperable
data from a range of
disciplines and
organisations
Barriers to re-use of
marine data
 Use of different
 Formats
 Standards
 Best practice
 Co-ordinate systems
 Technologies
 National and organisational
data access policies
3
E-infrastructures
 A number of regional initiatives
have made significant progress
in addressing these barriers by
developing marine data
management infrastructures
Australia
Europe
 Development of these
infrastructures is being
promoted and supported by
international organizations such
as UNESCO‘s
Intergovernmental
Oceanographic Commission
(IOC)
USA
Ocean Data Interoperability Platform
EU-US-Australia collaborative project
Grant Number: 312492
Call: FP7-INFRASTRUCTURES-2012-1-INFSO
Activity: INFRA-2012-3.2: International co-operation with the USA
on common e-infrastructure for scientific data
Start date: 1 October 2012
Duration: 36 months
Funded in parallel by European Commission, National Science
Foundation (NSF) and Australian Government
ODIP partners
Europe: 10 EU funded partners: 6 countries
NERC-BGS/BODC, MARIS, OGS, IFREMER, HCMR, ENEA, ULG,
CNR, RBINS-MUMM, TNO
USA: NSF funded partners
(supplement to existing R2R project)

San Diego Supercomputer Center (SDSC)

Scripps Institution of Oceanography (SIO)

Woods Hole Oceanographic Institute (WHOI)

Lamont-Doherty Earth Observatory (LDEO)

Florida State University: Center for Ocean-Atmospheric Prediction
Studies (FSU)
Australia
 University of Tasmania (IMOS)
International
 UNESCO IOC-IODE
Associate partners
 Europe
 Alfred Wegener Institute for Polar Research (AWI)
 MARUM
 USA
 NOAA US-IOOS, NOAA US-NODC, NOAA NGDC
 UNIDATA
 Australia
 Australian National Data Service (ANDS)
 Geoscience Australia (GA)
 CSIRO
ODIP: Objectives
 To establish an EU/USA/Australia/IOC-IODE
co-ordination platform to facilitate the interoperability of
ocean and marine data management infrastructures
 To demonstrate this co-ordination through the
development of several joint prototype interoperability
solutions that allows effective sharing of marine and
ocean data across scientific domains, organisations
and national boundaries
 Development of a common approach to marine data
management that can be extended to other regions and
organisations beyond the original project consortium
Overall strategy
Development of a collaboration platform to facilitate organised
dialogue between key organisations in Europe, USA and
Australia involved with marine data management.
Achieved by:
 Creating inventories of existing standards and policies
 Publication of these existing standards and best practice
through ODIP portal and Research Data Alliance
 Regular joint workshops to develop interoperability solutions
and/or agree on common standards
 Development of prototypes for testing and evaluating potential
interoperability solutions
 Dissemination /promotion of ODIP activities to encourage
wider participation and adoption of the outcomes of the project
Progress to date……….
 1st ODIP workshop: Ostende, Belgium (February 2013)
 Addressed 6 discussion topics
 Resulted in an extensive list of actions
 Formulated into three prototype interoperability
solution development tasks
 2nd ODIP workshop: San Diego, USA (December 2013)
 Addressed three discussion topics and implementation
plans for the prototype development tasks
 Initiation of 3 prototype interoperability solution
development tasks
ODIP 1: objective
 Establishing interoperability between the
SeaDataNet, IMOS and NODC data discovery and
access services using brokering services
 Lead by European partners via SeaDataNet
 Initially addressing use of brokers at the metadata
level
 Progress to data access services (possibly including
authentication and authorisation systems)
ODIP 1: the plan
 make use of the (Euro)GEOSS Brokerage harmonise
the 3 regional services to a common level
 SeaDataNet (Europe)
 IMOS (Australia)
 NODC (USA)
 start at metadata level, but progress to data access,
including providing solutions for possible AAA systems
 use the broker to facilitate access to data from the
regional services by the GEOSS and Ocean Data
Portals
ODIP 2: objective
 ODIP 2: Establishing interoperability between cruise
summary reporting systems in Europe, the USA and
Australia
 Lead by Rolling Deck to Repository (R2R) partners
(USA)
 Improvement of delivery and exchange of cruise
summary information through the use of common
formats and vocabularies
 Use of GeoNetWorks for routine harvesting of cruise
data for delivery via the Partnership for Observation of
Global Oceans (POGO) portal
ODIP 2: the plan
 Publish ISO Cruise Summary Reports at regional
nodes:
 Marine National Facility (Australia)
 SeaDataNet (Europe)
 R2R (USA)
 Deploy GeoNetwork catalogues at regional nodes
providing both a GUI (web portal) and API (CSW
service)
 Harvest GeoNetwork nodes into POGO global
catalogue
ODIP 3: objective
 Establishment of a prototype for a Sensor
Observation Service (SOS) for selected sensors
installed on vessels and in real-time monitoring
systems using sensor web enablement (SWE)
 Lead by AODN (Australia)
 regional initiatives progress towards the adoption of
SWE allowing direct standardised access to the data
from operational sensor systems
ODIP 3: the plan
 establish a collaboration tool (Github)
 compile inventory of SOS services and their endpoints
 compile inventory of instrument SensorML records &
O&M structures
 compile inventory of vocab and registry services
 Working groups to assess SOS performance
 propose templates for SensorML/StarFl and O&M profiles
 examine vocabulary services and potential mappings
 Set-up a test bed
ODIP 3: the plan
 Establish a collaboration tool
 Github: https://github.com/aodn/ODIP
 compile inventory of SOS services and their endpoints
 compile inventory of instrument SensorML records & O&M
structures
 compile inventory of vocab and registry services
 Working groups to
 assess SOS performance
 propose templates for SensorML/StarFl and O&M profiles
 examine vocab services and potential mappings
 Set up testbed
StarFL OGC discussion document
*FL Starfish Fungus Language for Sensor Description
http://portal.opengeospatial.org/files/?artifact_id=44438
Testbed - Progress so far… (Scott Bainbridge)
 Set up two public open SOS servers, one running V4.0 of the
52North SOS server, the other running V3.6;
 Both servers are fully open, that is anyone can undertake the full
SOS set of requests including registering sensors and adding data;
 Currently data from the IMOS FAIMMS Sensor Network is going
into the servers, more data can be added to look at load testing if
need be;
 Both servers form an ideal platform for client development and
service chaining experiments.
SOS v4.0
 Base Address: http://115.146.93.169:8080/IMOS-SOS/
 XML Address: http://115.146.93.169:8080/IMOS-SOS/sos/pox/
 So far two Features of Interest:
 IMOS/DAVIES/SF1 = Sensor Float 1 at Davies Reef
 IMOS/HERON/RP8= Relay Pole 8 at Heron Island
 With two Parameters:
 Water temperature (Deg. C.)
 Depth (m)
SOS v3.6
 Base Address: http://130.220.209.177:8080/IMOS-SOS-36/
 XML Address: http://130.220.209.177:8080/IMOS-SOS-36/sos/
 Will be set up with the same Features of Interest and Parameters
as the V4 server;
 Supports both SOS V1 and SOS V2.
Activities underway
 SeaDataNet (Jordi Sorribas)
SensorML_profiles_for_specific_
marine_observations_data-final.pdf
Example:
RONIMAR program station located in
Cadiz with a tide gauge.
SensorML (using the SensorML 2.0 schema)
and O&M files are online:
http://www.utm.csic.es/SensorWeb/Descriptions
Activities underway
 CNR (Alessandro Oggioni, Mauro Bastianini, Cristiano Fugazza)
VMs with 52°North SOS implementations:
a) version 3.2 linked to some data streams from the Venice Acqua Alta Tower
b) version 4.0 linked to some data streams from the LTER ecological data (also for
5 marine stations)
c) version 4.0 linked to the tidal network stations
Examples of SensorML for version 3.x and 4.0:
a) weather stations; b) vessel sensors; c) probe profile (eg CTD)
An inventory of SOS: collected 420 SOS endpoints (site to share this list?)
A SOS client to upload metadata (SensorML format) and Observations,
visualize (also in realtime), and download data from the sensors
‘Starter’ kit
Activities underway
 OGS (Elena Partescano, Alessandra Giorgetti)
Testing a SOS service linked to the meteooceanographic MAMBO1 buoy (in the North Adriatic Sea) and
the observatory site E2M3A (in the South Adriatic Sea)
Installed a OGC 52° North Sensor Observation Service 3.5
(http://nodc.ogs.trieste.it/SOS/sos) and a SOSClient
(http://nodc.ogs.trieste.it/SOSclient)
For testing: MAMBO1 - all CTD and pH sensors – and E2M3A pH sensor - (pH data for both buoys, stored using O&M format).
The SensorML 1.0 are compiled using, where possible, the
SeaDataNet vocabularies and EDIOS catalog.
Activities underway
 AWI (Hans Pfeiffenberger, Ana Macario)
Research vessel SOS
 SensorML and O&M SeaDataNet profiles are being adopted –
lots of work!



AWI hands-on experience will be communicated in next ODIP workshop
AWI is also keen on testing StarFL
SensorML and O&M will be „opened“ for RV Polarstern and RV Heincke
during Q1/2015
Dynamic vessel tracking
 On-going efforts towards integration in EUROFLEETS 2 prototype
Cooperation with 52North (SOS 4.0) is planned
 Optimization of current near real time services
Activities underway
 Ifremer (Thomas Loubrieu, Mickael Treguer)
Made some progress on oceanotron which now implements
SOS protocol. Output is encoded in XML, JSON, and netCDF. It
works for online results or off-line (or asynchronous) results for
big volumes.
Working together with Jordi Sorribas on templates of
sensorML and O&M and shared core rules on them (on
vocabulary's references,unique identifiers, contacts, ...)
Doing some experimentation with 52north.
Activities underway
 US-IOOS (Derrick Snowden)
All IOOS regions now have functioning IOOS milestone 1.0 SOS
services in place.
Strongly urging to adopt the 52North 4.0 branch as the basis for
ODIP.
Inventory of SOS services and endpoints https://www.ngdc.noaa.gov/docucomp/collectionSource/list?record
SetId=&componentId=&serviceType=SOS&serviceStatus=&serviceU
rl=&search=List+Collection+Sources
Inventory of sensorML and O&M structures https://code.google.com/p/ioostech/source/browse/#svn%2Ftrunk
%2Ftemplates%2FMilestone1.0
Vocab and registry services – http://mmisw.org/ont/ioos
Activities underway
Two test instances of 52N and ncSOS (four services - latest stable build
and continuous integration build)
52N: http://ioossos.axiomalaska.com/
ncSOS: https://github.com/asascience-open/ncSOS/wiki#wiki-quicklinks
Test data sets for further development:
For ncsos - https://github.com/asascienceopen/ncSOS/blob/master/resources/datasets.zip
For IOOS 52N - http://ioossos.axiomalaska.com/testdata.html
An early version of a python client for the IOOS SOS profile (and
several other web service interfaces): https://github.com/asascienceopen/pyoos
Activities underway
 BODC (Justin Buck)
Planning to implement a prototype ncSOS system for glider data
at BODC (based on IOOS but need to integrate access control).
What next?
 Use github repository to
 compile inventory of SOS services and their endpoints
 compile inventory of instrument SensorML records & O&M
structures
 compile inventory of vocab and registry services
 Initiate Working groups to
 assess SOS performance
 Justin Buck, Scott Bainbridge
 propose templates for SensorML/StarFl and O&M profiles
 Paolo Diviacco, Irina Bastrakova, Thomas Loubrieu*, Cristiano
Fugazza*, Hans Pfeiffenberger*
 examine vocab services and potential mappings
 Kim Finney / Sebastien Mancini, Cristiano Fugazza
 Add more SOS to testbed, add more datasets
In time for ODIP#3 workshop
 When? 4-8 August 2014
 Where? Australian Institute of Marine Science,
Townsville, Australia
Townsville will be warm and sunny in August, max around 26C., min around 14C
Participation: http://doodle.com/ziazg939u7wbypkw
Accommodation: IBIS TOWNSVILLE HOTEL (“AIMS0814”)
Thank you!
Dissemination of ODIP outcomes
 Project website
 Social media
 International conferences
 Other related initiatives
 Ocean Data Portal (ODP)
 Research Data Alliance
 Belmont Forum
www.odip.eu
Github repository
Users requesting to be registered for Github
Paulo Diviacco - [email protected]
Thomas Loubrieu (tloubrieu) - [email protected]
Mickael Treguer (mtreguer) - [email protected]
Kim Finney - [email protected]
Scott Bainbridge - [email protected]
Alessandro Oggioni - [email protected]
Mauro Bastianini - [email protected]
Cristiano Fugazza - [email protected]
Justin Buck - [email protected]
Ana Macario - [email protected]
Derrick Snowden - [email protected]
Jordi Sorribas - [email protected]
Dominic Lowe - [email protected]
Elena Partescano - [email protected]