Transcript PPT. 9.82 MB

SeaDataNet
A Pan-European Infrastructure
for Ocean and Marine Data
Management
by
Dick M.A. Schaap (MARIS)
Technical coordinator
NATO ARW September 2007
Background
•
•
•
•
•
Data availability of vital importance for marine research
European data set is fragmented
Not always quality controlled
Not always easily accessible
In 35 countries bordering European seas
• more than 1.000 scientific laboratories from government
organizations and private industry collect data
• use various sensors on board of research vessels,
submarines, fixed and drifting platforms, airplanes and
satellites
• measure physical, geophysical, geological, biological and
chemical parameters, biological species, etc.
NATO ARW September 2007
What is SeaDataNet?
•
•
•
•
EU Sixth Framework Programme
Integrated Infrastructure Initiative (I3)
1 April 2006 to 31 March 2011
Objectives
• develop an efficient distributed pan-European marine
data management Infrastructure for managing large
and diverse data sets
• network existing professional data centres of 35
countries, active in data collection
• provide integrated databases of standardised quality
on-line
NATO ARW September 2007
SeaDataNet partners
49 partners including:
• Major oceanographic
institutes of 35 countries
• National Oceanographic
Data Centres (NODCs)
• Divisions of major national
marine research institutes
• 3 satellite data centres
• 3 scientific modelling
centres
• 2 SMEs experts in software
development
• 3 International
organisations (IOC, ICES,
JRC)
NATO ARW September 2007
Building on experience
• Leaders of Data Management Concerted Actions
• MEDAR/MEDATLAS II - Mediterranean Data Archaeology
and Rescue, aiming to make available a comprehensive data
product of temperature, salinity and bio-chemical data in the
Mediterranean and Black Sea through a wide international
cooperation
• Sea-Search – The gateway to Oceanographic and Marine
Data & Information in Europe, to provide an effective
navigation tool to oceanographic and marine data and
information sources for European marine waters and to centres
in Europe with expertise and skills in oceanographic and marine
data & information management.
• Data management of scientific projects:
• MTPII-MATER, CANIGO, OMEX, MERSEA, etc.
• Bringing in new partners – satellite data centres
NATO ARW September 2007
SeaDataNet components
NATO ARW September 2007
Infrastructure, versions & planning
• Version 0
Continuation, maintenance and further population of existing Sea-Search
systems with minor modifications
• Version 1
All data centres involved in upgraded meta-databases (ISO compliant XML
metadata-profiles and web services)
Technical Task Team partners involved in transparent data access, security,
and monitoring of use = trans-national access via downloading of datasets
in standard formats (ODV ASCII; NetCDF; MedAtlas)
Launch by March 2008
• Version 2
All data centres involved in trans-national access and upgrading of data
access to include spatial data infrastructure features (adopting OGC; ISO
and INSPIRE standards) and using XML data profiles
Operational by 2010
NATO ARW September 2007
SeaDataNet Data Policy
• Metadata
• free and open access, no registration required
• each data centre should provide the meta-data in standardized
format to populate the discovery services (metadata directories)
• Data and products services
• in general free and open access, but mandatory registration
• a “SeaDataNet role” (partner, academic, commercial etc.) is
attributed to each user
• Each NODC attributes the roles to the users of its of country
• Out of the partnership, the roles are assigned by SeaDataNet userdesk
• each data centre node delivers data according to the user’s role
and its local regulation
• each data centre should provide freely the data sets necessary
to develop the common products
NATO ARW September 2007
Discovery services
• Marine environmental data
sets (EDMED)
• Marine environmental
research projects (EDMERP)
• Observing systems (EDIOS)
• Research Cruise Summary
Reports (CSR)
• Common Data Index (CDI)
• Directory of marine
organisations (EDMO)
NATO ARW September 2007
Cruise Summary Reports
• Conceived by IOC
• ICES acted as focal point
• Currently over than 37000
cruises
• Documenting data
collected on scientific
research cruises operated
by European vessels
• Cruise track charts
included
• Online entry form
• Online searchable
database
NATO ARW September 2007
European Directory of the Oceanobserving System (EDIOS)
• Initiative of the European
Global Ocean Observing
System (EuroGOOS)
• Regular repeated
measurements
• On-line searchable
directory of observing,
measuring, and
monitoring systems
• Over 10,000 observation
entries
• Links to on-line real-time
and archive data
• Tool to aid design of
observing systems
NATO ARW September 2007
European Directory of Marine
Organisations (EDMO)
• Over 1.000
organisations:
addresses, profiles and
relations in other
directories
• Online maintenance per
country by CMS
• On-line user interface
with combined
alphanumeric –
geographical searching
• Export in .csv files
• Also available as Web
service (SOAP based)
NATO ARW September 2007
Common Data Index
• Index to individual data
sets
• Now >250000 entries
from 20 partners
• Mid Oct 2007 entries
from 35 countries
• ISO19115 metadata
standard has been
adopted
• On-line searchable
directory
• Direct access to on-line
data sets, on-line order
facility or e-mail
request
NATO ARW September 2007
Technical developments
• Streamlining of metadata formats for interrelationships
between the directories
• Establishment and governance of common vocabularies
• ISO 19115 content model as basis for the XML formats
and exchange schemas (XSD) with SeaDataNet XML
approach
• Innovating maintenance modalities:
• XML exchange using local software component (MIKADO) or
own application
XML validation services
Online Content Management System (CMS)
•
•
• Introducing Web services
• Vocabularies and Directory of Marine Organisations (EDMO)
• Coming: other metadata directories
NATO ARW September 2007
Interrelationships between the
directories, using vocabularies
XML
EDMED
…
…
…
CSR
EDMERP
Platf
Param
Country
EDIOS
CDI
EDMO
NATO ARW September 2007
Common Vocabularies
• Controlled vocabularies
containing keys, terms,
abbreviations and
essential (but often
missing) definitions
• Lists for platforms,
countries, sea areas,
etc…
• International content
governance by SeaVox
list server
• Web service
• On-line user client
• Export in .csv files
NATO ARW September 2007
SeaDataNet standards
• Interoperability – key to distributed data
management system success
• Common understanding of terms used via common
vocabularies
• Standard data exchange formats (ODV ASCII,
NetCDF, MedAtlas)
• Common quality control protocols and flag scale
• Standard Web services
• Schemes for handling duplicate data
• Ocean Data View software (ODV) adapted for
SeaDataNet
NATO ARW September 2007
SeaDataNet : Delivery Services V1
Primary function
• Users will be able to
download selected
data sets
CDI User Interface
• Searching and
browsing of
metadata of data
sets
NATO ARW September 2007
SeaDataNet : Delivery Services V1
Shopping basket
• Preparing a user
request of multiple
data sets
Central User Register
• Validation of users
and their roles
Business matrix
• Combining user role
and data access
restriction(s) to
decide on modality
for processing each
data request
A B C D
U
V
W
NATO ARW September 2007
Central User Register
• Single Sign On system required for access to distributed
system
• User’s authentication information based on personal login /
password
• User must register in order to get login
•
•
•
•
•
Web form to provide necessary information
Online user agreement “SeaDataNet General Licence”
Registration processed per country via online CMS
After processing, login/password sent by e-mail (e-mail check)
User gets ‘Role’
• SeaDataNet Data Policy
• Data access based upon combination of User role Access restriction(s) as indicated in CDI record and
business logic
NATO ARW September 2007
SeaDataNet : Delivery Services V1
Request Status
Manager
• Processing of all
requests and data
deliveries
• Enables users to
follow progress
• Enables DC’s to
oversee transactions
Download manager
• Local component for
communication with
portal and preparing
data deliveries
NATO ARW September 2007
SeaDataNet User’s portal schema V1
General request
Project info
Standards
Data request Status of
request
Metadata request
metadata
data
Cross search
Registr.
My transact.
Shopping
basket
Software
EDIOS
EDMED
CDI
Vocabularies
CSR
EDMERP
EDMO
Requests
status
manager
Organisation +
data source id
CSR
User
Data
registration download
EDIOS
AAA
User
Register
CDI
IFREMER Database
EDMO
EDMED
at BSH
at BODC
BODC Database
+ EDMERP
at MARIS
BSH Database
Metadata & Data catalogues
Entry point for access hits
services for metadata
Data downloading services
Visualisation services (WMS) for V2
...
Download managers in data centres
NATO ARW September 2007
Data products
• SeaDataNet will provided added-value products to be
created by regional centres
•
•
•
•
•
Mediterranean Sea
Black Sea
Baltic Sea
Arctic and North Sea
Atlantic and global ocean
• First products => gridded climatologies
• based on already available data-sets (mostly hydrographic data)
• updated during the project life
• To make products comparable, all regional data centers
will use a common tool for spatial mapping
• For this a new version of DIVA (Data Inverse Variational
Analysis) was developed (University of Liege)
NATO ARW September 2007
SeaDataNet wider involvement
• SeaDataNet partners are involved in many other European
and international projects, contributing and safeguarding
good data management. Examples:
• Services for the European marine geological institutes
• EDIOS Services for EuroGOOS
• Partners in the MERSEA project (GMES) for MIM
• Partners in the HUMBOLDT project (GMES) for
developing a framework for a European Spatial Data
Infrastructure
• Partners in SIMORC for oil & gas industry metocean data
• Partners in ECOOP for data provision and management
• Component of the GMES Marine Core Services
NATO ARW September 2007
Marine geology data
Catalogue of >
300.000 seafloor
samples and
>2.5 million line
kilometres of
marine seismic
and sonar survey
from the ocean
basins and
continental
shelves, held at
European
institutions
www.eu-seased.net
NATO ARW September 2007
Industry metocean datasets - SIMORC
• System of
Industry
Metocean data
for the Offshore
and Research
Communities
• Metadatabase
and Database of
datasets from
major oil & gas
companies
www.simorc.org
At present ca 870 data sets (Shell, BP, Total), more
underway
NATO ARW September 2007
SeaDataNet - BlackSeaScene
• BSS is an Integrating Activity of EU-Research Infrastructures
programme and runs from 1 Dec 2005 – 30 Nov 2008
• BSS has 32 partners with 25 from Black Sea region and is
coordinated by MARIS
• BSS is a Black Sea Scientific Network
• analysing the present quality of Black Sea datasets and
applied DQC procedures
• developing a Black Sea regional portal for metadata and
data access
• BSS is adopting the SeaDataNet standards and building
upon the SeaDataNet basic services
• BSS engages 25 Black Sea institutes, strengthening
and expanding the compilation of metadata and access
to data for the Black Sea region
NATO ARW September 2007
SeaDataNet capacity building
The training program has two main objectives:
•to ensure that the data and metadata to be integrated in the
SeaDataNet system will be formatted, checked for quality and
disseminated according to the common protocols developed
•to transfer expertise and to train IT experts of the data centers in the
basics, installation and operation of the SeaDataNet technical
components
NATO ARW September 2007
International Conference IMDIS 2008
• Second international conference “International
Marine Data Information Systems” (IMDIS)
• March 31 - April 2, 2008, Athens, Greece
• Joint organisers: SeaDataNet, Hellenic Centre for
Marine Research, IOC/IODE
• Four sessions:
• Marine environmental data bases: infrastructures and
data access systems
• Data Services in ocean science: standards and
interoperability
• Visualisation tools
• Services for Users and Education
NATO ARW September 2007
www.seadatanet.org
NATO ARW September 2007