Transcript SeaDataNet

SeaDataNet
A Pan-European Infrastructure
for Ocean and Marine Data
Management
SIMORC Workshop March 2007
Background
•
•
•
•
•
Data availability of vital importance for marine research
European data set is fragmented
Not always quality controlled
Not always easily accessible
In 35 countries bordering European seas
• more than 600 scientific laboratories from government
organizations and private industry collect data
• use various sensors on board of research vessels,
submarines, fixed and drifting platforms, airplanes and
satellites
• measure physical, geophysical, geological, biological and
chemical parameters, biological species, etc.
SIMORC Workshop March 2007
What is SeaDataNet?
•
•
•
•
EU Sixth Framework Programme
Integrated Infrastructure Initiative (I3)
1 April 2006 to 31 March 2011
Objectives
• develop an efficient distributed pan-European marine
data management Infrastructure for managing large
and diverse data sets
• network existing professional data centres of 35
countries, active in data collection
• provide integrated databases of standardised quality
on-line
SIMORC Workshop March 2007
SeaDataNet partners
49 partners including:
• Major oceanographic
institutes of 35 countries
• National Oceanographic
Data Centres (NODCs)
• Divisions of major national
marine research institutes
• 3 satellite data centres
• 3 scientific modelling
centres
• 2 SMEs experts in software
development
• 3 International
organisations
SIMORC Workshop March 2007
Building on experience
• Data management of scientific projects, e.g. MTPIIMATER, CANIGO, OMEX, MERSEA, etc.
• Leaders of Data Management Concerted Actions
• MEDAR/MEDATLAS II - Mediterranean Data Archaeology
and Rescue, aiming to make available a comprehensive data
product of temperature, salinity and bio-chemical data in the
Mediterranean and Black Sea through a wide international
cooperation
• SEA-SEARCH – The gateway to Oceanographic and Marine
Data & Information in Europe, to provide an effective
navigation tool to oceanographic and marine data and
information sources for European marine waters and to centres
in Europe with expertise and skills in oceanographic and marine
data & information management.
• Bringing in new partners – satellite data centres
SIMORC Workshop March 2007
SeaDataNet components
SIMORC Workshop March 2007
Infrastructure, versions & planning
• Version 0
Continuation and maintenance of existing Sea-Search systems with
minor modifications
• Version 1
All data centres involved in upgraded meta-databases
Technical Task Team partners involved in transparent data access,
security, monitoring, viewing services = trans-national access
Pre-operational by October 2007 and operational by February 2008
• Version 2
All data centres involved in trans-national access and access
upgraded
Operational by 2010
SIMORC Workshop March 2007
SeaDataNet data policy
• Metadata
• free and open access, no registration required
• each data centre should provide the meta-data in standardized
format to populate the catalogue services
• Data and products services
• the general case if free and open access/mandatory registration
• a “SeaDataNet role” (partner, academic, commercial etc.) is
attributed to each user
• Each NODC attributes the roles to the users of its of country
• Out of the partnership, the roles are assigned by SeaDataNet userdesk
• each data centre node delivers data according to the user’s role
and its local regulation
• each data centre should provide freely the data sets necessary
to develop the common products
SIMORC Workshop March 2007
Discovery services
• Marine environmental data
sets (EDMED)
• Marine environmental
research projects (EDMERP)
• Observing systems (EDIOS)
• Research Cruise Summary
Reports (CSR)
• Common Data Index (CDI)
• Directory of marine
organisations (EDMO)
SIMORC Workshop March 2007
Cruise Summary Reports
• Conceived by IOC
• ICES acted as focal point
• Currently over than 37000
cruises
• Documenting data
collected on scientific
research cruises operated
by European vessels
• Cruise track charts
included
• Online entry form
• Online searchable
database
SIMORC Workshop March 2007
European Directory of the Oceanobserving System (EDIOS)
• Initiative of the European
Global Ocean Observing
System (EuroGOOS)
• Regular repeated
measurements
• On-line searchable
directory of observing,
measuring, and
monitoring systems
• Over 10,000 observation
entries
• Links to on-line real-time
and archive data
• Tool to aid design of
observing systems
SIMORC Workshop March 2007
Common Data Index
• Index to individual data
sets
• Approx. 200000 entries
from 10 partners
• ISO19115 metadata
standard has been
adopted
• On-line searchable
directory
• Direct access to on-line
data sets, catalogues
or e-mail request
SIMORC Workshop March 2007
Technical developments
• Establishment of common vocabularies
• Upgrading formats for interrelationships between these
directories
• ISO 19115 content model is the basis for the XML
formats and exchange schema (XSD)
• Innovating maintenance modalities:
• XML exchange formats upgrading
• XML validation services
• Online Content Management System (CMS)
• Introducing Web services
• Vocabularies and Directory of Marine Organisations (EDMO)
SIMORC Workshop March 2007
SeaDataNet standards
• Interoperability – key to distributed data
management system success
• Includes establishing common understanding of terms
used
• Achieved through use of controlled vocabularies
containing keys, terms, abbreviations and essential
(but often missing) definitions
• Schemes for handling duplicate data
• Common quality control protocols
• Ocean Data View software for visualisation
SIMORC Workshop March 2007
SeaDataNet : Delivery Services V1
Primary function
• Users will be able to download
selected data sets
CDI User Interface
• Searching and browsing of
metadata of data sets
Shopping Basket in combination with AAA services
• Preparing a user request of multiple data sets, handling
validation of users, and routing requests to the data
centres
SIMORC Workshop March 2007
Authentication, Authorization and
Administration (AAA)
• Single Sign On system required for access to distributed
system
• User’s authentication information based on personal login /
password
• User must register in order to get login
• Web form to provide necessary information
• Online user agreement “SeaDataNet General Licence”
• After processing, login/password sent by e-mail (e-mail check)
• Users’ directory management (registration processing)
• Centralised vs Decentralised
• Choice in V1 for CAS system (= centralised)
• SeaDataNet Data Policy
• Authorisation based on “Roles”
SIMORC Workshop March 2007
SeaDataNet : Delivery Services V1
Download manager
• process of requests by each data centre to deliver one
comprehensive data set (one zip file) per data centre
Request status manager
• provide users overview of request status of requests and
download functions
• provide data centres with overview of all requests and
data delivery
Data delivery formats
• CF compliant NetCDF format for gridded and multidimensional data types (e.g. ADCP)
• Modified ODV spreadsheet format for other data
types (e.g. time-series, profiles)
SIMORC Workshop March 2007
SeaDataNet User’s portal schema V1
General request
Project info
Standards
Data request Status of
request
Metadata request
metadata
data
Cross search
Registr.
My transact.
Shopping
basket
Software
EDIOS
EDMED
CDI
Vocabularies
CSR
EDMERP
EDMO
Requests
status
manager
Organisation +
data source id
CSR
User
Data
registration download
EDIOS
AAA
User
Register
CDI
IFREMER Database
EDMO
EDMED
at BSH
at BODC
BODC Database
+ EDMERP
at MARIS
BSH Database
Metadata & Data catalogues
Entry point for access hits
services for metadata
Data downloading services
Visualisation services (WMS) for V2
...
Download managers in data centres
SIMORC Workshop March 2007
Data products
• SeaDataNet will provided added-value products to be
created by regional centres
•
•
•
•
•
Mediterranean Sea
Black Sea
Baltic Sea
Arctic and North Sea
Atlantic and global ocean
• First products => gridded climatologies
• based on already available data-sets (mostly hydrographic data)
• updated during the project life
• To make products comparable, all regional data centers
will use a common tool for spatial mapping
• For this a new version of DIVA (Data Inverse Variational
Analysis) was developed (University of Liege)
SIMORC Workshop March 2007
SeaDataNet capacity building
The training program has two main objectives:
•to ensure that the data and metadata to be integrated in the
SeaDataNet system will be formatted, checked for quality and
disseminated according to the common protocols developed during the
project
•to transfer expertise and to train IT experts of the SeaDataNet data
centers in the basics, installation and operation of the SeaDataNet
technical components
SIMORC Workshop March 2007
SeaDataNet system components
• Discovery services = Metadata directories
• Vocabulary services = Common vocabularies
• Auxiliary tables = Organisations, Research Vessels
• Security services = Authentication, Authorization &
Administration (AAA)
• Monitoring and statistics = usage & system performance
• Delivery services = Data access & downloading of data
sets (ASCII, NetCDF, …, data virtualisation)
• Viewing services = GIS, ODV analysis software, Google
Earth, ..
• Product services = Generic and standard products
SIMORC Workshop March 2007
International Conference IMDIS 2008
1st Announcement
• Second international conference “International
Marine Data Information Systems” (IMDIS)
• March 31 - April 1, 2008, Athens, Greece
• Joint organisers: SeaDataNet, Hellenic Centre for
Marine Research, IOC/IODE
• Four sessions:
• Marine environmental data bases: infrastructures and
data access systems
• Data Services in ocean science: standards and
interoperability
• Visualisation tools
• Services for Users and Education
SIMORC Workshop March 2007
www.seadatanet.org
SIMORC Workshop March 2007