Transcript SeaDataNet CDI Search interface

SeaDataNet - Pan-European infrastructure for
ocean and marine data management
Present state, relations to EMODnet, and future developments
By
Dick M.A. Schaap – MARIS (SeaDataNet Technical Coordinator)
Copenhagen – Denmark, May 2011, EEA – WISE-MARINE Meeting
Supported by the European Commission
EU FP6
2006 - 2011
SeaDataNet has set up and is operating a pan-European infrastructure for
managing marine and ocean data by connecting 40 National Oceanographic
Data Centres (NODC’s), national oceanographic focal points, and ocean
satellite data centres, in Europe
Interoperability
Interoperability is the key to distributed data management system success. This
is achieved in SeaDataNet via:
Adopting the ISO 19115 metadata standard for all metadata directories
Providing standard metadata entry tools: MIKADO, and online CMS
Providing XML Validation Services to quality control the metadata
maintenance
Using common and controlled vocabularies, including international content
governance
Using harmonised Data Transport Formats for data sets delivery and standard
tools for format conversion, such as NEMO, ..
Using common quality control protocols and flag scale
Adopting of OGC standards for mapping and viewing services
Using SOAP Web Services in the SeaDataNet architecture
Organising training and instruction workshops for transfer of expertise and
hands-on training
Portal with harmonised services, data
products and tools for users and data centres
http://www.seadatanet.org
Pan-European Directory Services
The SeaDataNet portal provides overviews of marine organisations in
Europe and their engagement in research, scientific cruises,
monitoring and data management for European waters and global
oceans:
EDMO: European Directory of Marine Organisations (>2200 entries)
EDMED: European Directory of Marine Environmental Data sets
(>3000 entries)
EDMERP: European Directory of Marine Environmental Research
Projects (>2500 entries)
CSR: Cruise Summary Reports (>31500 entries)
EDIOS: European Directory of Ocean-observing Systems (>270
programmes for the UK alone and many underway for other
European countries)
These Directories are maintained by NODCs for their country and
published at pan-European level
Pan-European Directory Services
All these Directories have been harmonised and mutually tuned in
format, syntax and semantics (common vocabularies), a common
XML editor (MIKADO), online CMS and online user interfaces.
EDMERP
CSR
EDMO
EDIOS
EDMED
Relations between Directory Services
From the EDMO interface one can oversee and retrieve cross related
entries in each of the Directories.
SeaDataNet from data discovery to unified
access to analysis
Common Data Index (CDI) Data Discovery
and Access Service
CDI is a fine-grained index to individual data measurements. The CDI
Service provides users a highly detailed insight and unified access to
the large volumes of marine and oceanographic data sets
An intelligent middle tier connection is configured between the
SeaDataNet portal and the local data management systems at each of
the data centres.
A shopping basket allows users to submit a shopping request for
multiple data providers in one go and to follow its processing by each
of the providers via an online transaction register.
Common Data Index (CDI) Data Discovery
and Access Service
Distributed Data Management
Data Policy and implementation
Metadata
free and open access, no registration required
each data centre is obliged to provide the meta-data in standardized
format to populate the catalogue services
Data and products services
the general case is free and open access
however users must register once in order to get a personal login password
Web form to provide necessary information
User agrees with “SeaDataNet User Licence”
After processing, login/password sent by email (email check)
Licence is part of the SeaDataNet Data Policy, that is intended to be fully
compatible with the Directive of the European Parliament and of the Council
on public access to environmental information, the INSPIRE Directive, IOC,
ICES, WMO, GCOS, GEOSS and CLIVAR data principles.
SeaDataNet Data Policy is an overarching policy, respecting any local policy
Each user gets a SeaDataNet Role; while each CDI record has a ‘Data access
restriction’ label. The combination of role and access label determines
‘access OK’, ‘access to be discussed’ or ‘access denied’
SeaDataNet CDI – Data retrieval and downloading
Check Status
In RSM
Search
Request
Confirmed
Include in
Basket
Results
Ready at DC x
Shopping list
Submit + Authentication
Download
Data
SDN
format
SeaDataNet CDI Quick Search interface
Drilling down and very intuitive for any user
SeaDataNet CDI Extended Search interface
Searching for nutrients
SeaDataNet CDI Extended Search interface
Details of a selected CDI record
SeaDataNet CDI Search interface
Searching for nutrients – results map zoom in and also with Blue Marble
SeaDataNet CDI Extended Search interface
Shop for nutrient data around Danmark
SeaDataNet CDI Search interface
Submit
Shopping list for nutrient data around Danmark
CDI extra functionalities
Numerous options for searching and drilling down of search results
CDI XML format can handle points, detailed tracks and polygons as
GML
CDI Summary function for search results with statistics and drill-down
function
CDI Mapping supports OGC WMS services with option to add external
WMS layers and vice versa to serve CDI as WMS – WFS layer to other
portals
A query and GIS layer configuration can be stored by users as a
favourite and can be forwarded to other users as a URL
Extra attribute included for linking to (pre)viewing services
SeaDataNet CDI Search interface
Searching for nutrients – results Summary
SeaDataNet CDI Search interface
Searching for nutrients – results Summary – Data centres - clickable
SeaDataNet CDI Search interface
Add OGC WMS layers – EMODNet Hydrography
SeaDataNet CDI Search interface
OGC WMS layers added – EMODNet Hydrography
OGC WMS – WFS support
CDI entries from the CDI portal are available as a series of OGC WMS
and WFS services, e.g. for inclusion in the EMODNet portals.
CDI - INSPIRE compliance
CS-W service on top of the CDI portal to deliver CDI XML entries in
the INSPIRE compliant ISO 19139 format (at present aggregated by
discipline and by data centre)
SeaDataNet Viewing services
Ocean Data View (ODV)
General data analysis and visualization software
>10,000 registered users
ODV software upgraded to ODV4 for:
Extending ODV‘s graphical display capabilities and interactive
controls for automatic and visual quality control and data quality
flagging
Seamless connection to SeaDataNet output: SDN ODV4 data
formats, quality flag scale
Integration of DIVA gridding software
Important tool for the SeaDataNet regional Data Products
DIVA software (Data-Interpolating Variational Analysis) allows to
spatially interpolate observations on a regular grid in an optimal way.
The analysis is performed on a finite element grid allowing for a
spatial variable resolution and a good representation of the coastline
and isobaths.
SeaDataNet Viewing services – ODV + DIVA
Regional Products services
Climatologies and interpolated data products for the Arctic seas, Atlantic
Ocean,Baltic sea, Mediteranean sea and the Black Sea regions. Statistical
output in NetCDF files, that are visible via WMS through Ocean Browser
Cross sector involvement
SeaDataNet cooperates and provides services for many other
European and international groups and projects, contributing and
safeguarding good data management. Examples:
EuroGOOS, maintaining the European Directory of Ocean
observing systems (EDIOS) and quality controlling and archiving
long-term data series. See www.edios.org
GMES Marine Core Services for streamlining the provision of
long-term archives for optimising marine forecast services; there is
an MoU formulating the cooperation between MyOcean and
SeaDataNet
POGO, collecting and providing information on ocean-going
research vessels, and their operators, planned and completed
cruises. See www.pogo-oceancruises.org
Services for the international oil & gas industry for metocean data
(SIMORC). See www.simorc.org
SeaDataNet Expansion
SeaDataNet standards and tools are adopted by other EU-funded projects:
Up-Grade Black Sea SCENE project, (2009 – 2011) involving 6 NODCs
and 35 other data holding institutes from the 6 Black Sea countries, to
provide metadata and data access and to strengthen their national NODC
networks. EU-funding 3.4 Million Euro.
CASPINFO project, (2009 – 2011) involving 12 institutes and private
industry from the Caspian Sea region, to provide metadata and data
access and to build their capacity for data management and user services.
EU-funding 0.8 Million Euro.
Geo-Seas project, (2009-2012) involving 24 geological and geophysical
data centres from 16 European countries (EuroGeoSurveys), to provide
metadata and data access. EU-funding 4.9 Million Euro.
EuroFleets project, (2009-2012) involving 24 research institutes and
data centres from 17 European countries, to optimise the management of
research vessels and to streamline the flow of data from the research
cruises to the data centre infrastructure. EU-funding 7.2 Million Euro.
In each project several SeaDataNet partners are participating for transfer of
knowledge & expertise, to secure interoperability and to achieve
crossfertilisation leading to upgrading of the SeaDataNet standards and tools
as well as enriching of the services.
SeaDataNet and EMODnet
EU has adopted a new Marine Strategy Framework Directive (MSFD)
which includes a requirement for an overarching European Marine
Observation and Data Network (EMODnet)
EMODnet can be considered as a network of existing and developing
European observation systems, linked by a data management
structure covering all European coastal waters, shelf seas and
surrounding ocean basins’.
SeaDataNet and EMODnet
SeaDataNet and EMODnet
SeaDataNet has qualified itself as leading infrastructure for the
EMODnet data management component and is undertaking several
preparatory EMODnet projects:
Chemistry – focus on the groups of chemicals required for
monitoring the MSFD for establishing Good Environmental Status –
cooperation with EEA, ICES and Regional Conventions
Hydrography – to produce and deliver a higher resolution digital
bathymetry for European seas
Physics – together with MyOcean and EuroGOOS Regions to give
wide access to operational metocean data, real-time and archives
The Geology lot is producing harmonised seabed maps (WMS). The
SeaDataNet infrastructure is adopted via the Geo-Seas project to give
overview and access to the underlying geological and geophysical data
sets.
SeaDataNet is data partner in the Biology lot and a closer technical
relation is foreseen in the SeaDataNet II project.
EMODnet geographic areas
Hydrography
North Sea
Celtic Seas, Channel,
Mediterranean sea,
NW Atlantic
Geology
North Sea
Baltic and Celtic Seas
Biology
North Sea
Bay of Biscay and the
Iberian Coast
Chemistry
North Sea
Black Sea + Med spots
Physics
North Sea
All other European seas
Habitats
North Sea
Baltic, Celtic Seas and
the Western
Mediterranean
SeaDataNet and EMODnet
The EMODnet pilots and associated research projects stimulate a
wider uptake of SeaDataNet standards and contribute to enriching the
standards and services
SeaDataNet is a project with a fixed partnership. The EMODnet pilots
are data driven and bring on board many more data providers that
adopt the standards to connect to the infrastructure and to populate
the data resources.
To illustrate this: SeaDataNet comprises 40 Data Centres, but already
60 Data Centres are now connected and we are making good progress
towards ca 90 connected Data Centres
EMODnet is management and policy driven which also provides an
excellent opportunity for SeaDataNet to upgrade its status from
project to sustained operational infrastructure.
CDI Service as common service
www.emodnet-chemistry.eu
www.blackseascene.net
www.geo-seas.eu
SeaDataNet CDI Service
www.emodnet-hydrography.eu
(www.emodnet-physics.eu)
www.caspinfo.net
CDI – Data Coverage
CDI coverage at end May 2011: > 1.000.000 CDI entries from
29 countries and >60 data centres for physics, chemistry,
geology, geophysics, bathymetry and biology; connecting
another 30 data centres and increase in CDIs is well
underway
SeaDataNet II proposal – Recently accepted
There are many challenges and recent innovations which will be taken
up in SeaDataNet II that will run from 1st October 2011 – 30th
September 2015 with a funding budget of 6 Million Euro. A
number of objectives:
Achieving full INSPIRE compliance and contributing to the INSPIRE
process for developing implementing rules for oceanography
Achieving data access and data products services that meet
requirements of end-users and intermediate user communities, such
as GMES Marine Core Service, establishing SeaDataNet as the core
data management component of the EMODNet infrastructure, and
contributing on behalf of Europe to global portal initiatives, such as
the IOC-IODE - Ocean Data Portal (ODP), and GEOSS.
Achieving an improved capability for handling also marine biological
data and interoperability with the emerging biodiversity data
infrastructure in Europe
SeaDataNet II project – selected technical aims
Establishing machine-to-machine interfacing next to present user
portal for technically serving specific user communities, such as
GMES Marine Core Service and WISE-Marine (EEA)
Developing online visualisation (previewing, viewing) services
Extension of the CDI metadata format with Observations &
Measurements module and SensorML module, where applicable for
specific types of data, e.g. for operational oceanography
Formulation of NetCDF core data model including SeaDataNet
metadata header in cooperation with UniData
Tackling the duplicates problem
Conclusions
SeaDataNet is a strong partnership, well established in the Member
States and in the ocean and marine community
SeaDataNet operates and further develops a data management
infrastructure that is adopted as core data management component
in the EMODnet pilots and also adopted by many research projects
and monitoring initiatives
SeaDataNet has established and is further developing marine data
management standards that are adopted on a wide scale
SeaDataNet has established and is further developing interoperability
solutions, such as machine-to-machine interfaces, for serving
specific user communities
Associated projects and EMODnet pilots stimulate expansion of the
number of connected data centres and increase of data population,
which is very relevant for MSFD and WISE-Marine.
EMODnet opportunities
Additional funding (22.5 MEuro) is being arranged for 2012 – 2014 for
more data pilots and extending and operating existing pilots.
Moreover EMODnet has the promise of a substantial and structural
funding from 2014 onwards for the data management component and
the observation systems.
Already the preparatory actions are accelerating the convergence
between different data sectors (geology, hydrography, chemistry, ….).
There is an increasing willingness to harmonise, to become
interoperable and to enable sharing of data in sectors and between
sectors.
These opportunities and the SeaDataNet – EMODnet momentum
should be used by DG Environment and EEA.
MSFD brings EMODnet and WISE-Marine together
Policy relevant data
Country data
Research data
Marine Core Service
data
Reference layers
(maps)
E M O
D
NET
MSFD
uropean
arine
bservation
ata
work
WISE-Marine