Transcript Swiss Experiment

TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
PERMASENSE
Nicholas Dawes, Ashwin Kumar, Sebastian Michel, Karl Aberer, Michael Lehning
Swiss Experiment
Ecole Polytechnique Fédérale de Lausanne, Switzerland
Swiss Federal Institute for Snow and Avalanche Research (SLF), Davos, Switzerland
Interdisciplinary Environmental Research
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
Outline
•
•
•
•
•
•
•
Introduction to Swiss Experiment (SwissEx)
Problem Statement and Motivation
The SwissEx Wiki
SwissEx Infrastructure (Data)
Semantic Forms
Querying Metadata
Conclusion and Future Work
Swiss Experiment
Interdisciplinary Environmental Research
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
PERMASENSE
Swiss Experiment
Interdisciplinary Environmental Research
Swiss Experiment (SwissEx Aims):
• Provision of a generic infrastructure of:
• web based technologies
• wireless communications
• low cost high density sensors
to serve the environmental science
community, encourage collaboration and
provide a portal for public information on
environmental research
• Climate change forecasts show the greatest impact in high alpine scenarios. These
changes are already being observed and answers to environmental questions need to
be found quickly. The Swiss Experiment infrastructure aims to speed up that process
www.swiss-experiment.ch
Swiss Experiment
Interdisciplinary Environmental Research
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
SwissEx Infrastructure
• SwissEx infrastucture is built to serve many environmental research projects
Black boxes indicate widespread research areas with multiple
experiments within
• Where experimental areas overlap, projects can work more efficiently by
sharing data
• Projects can benefit from external data sources
Swiss Experiment
Interdisciplinary Environmental Research
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
Requirement for Metadata
• Temperatures from Wannengrat, Davos 22/01/07 – 28/01/07
Seem legitimate for 2400m altitude in winter?
Swiss Experiment
Interdisciplinary Environmental Research
HYDROMON
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
PERMASENSE
Requirement for Metadata
• Temperature/Humidity sensor 25/01/07
Sensor opening
covered in reim
The data from this sensor
during the preceding period
is unusable, but
indistinguishable from the
data after this photo was
taken.
…….never trust somebody elses data without the associated metadata
Swiss Experiment
Interdisciplinary Environmental Research
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
The SwissEx Wiki
Information, Data Portal and Metadata Database Interface
• Central to the functioning
of the project
• Incorporates SensorMap
and GSN access:
• If you can access the data,
this is the place to have the
metadata access
• MediaWiki uses RDF format to store semantics
• RDF enables structure to be added to a mass of web based data
Swiss Experiment
Interdisciplinary Environmental Research
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
SwissEx Infrastructure
Swiss Experiment
Interdisciplinary Environmental Research
COGEAR
HYDROMON
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
SwissEx Metadata
Sensor data
• Data provided by the sensor
• Streaming data stored using GSN, manually sampled data storage
tbc
Experiment
centric static
information
• Background information on the experiment, the overall
structure and the phenomena which the sensor network
is there to measure, which must be stored once
Sensor centric
static information
• Background information such as the type and location of
each sensor, which must be stored once
Dynamic sensor
information
• Information on the servicability of the sensor i.e. whether
it is deployed, stored, broken etc.
Dynamic data
quality
information
•
Stored in the wiki
• Stored in the wiki (using semantics)
• Stored in the wiki (using semantics)
• A measure of the quality of the data which requires a
continuous variable
• To be calculated and stored using GSN
Swiss Experiment
Interdisciplinary Environmental Research
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
The Metadata Model
• The metadata model:
Observation with
station no.
Station
Parameter
Parameter
Parameter
Serial No.
Parameter
Parameter
Serial No.
Parameter
Parameter
Serial No.
Parameter
Serial No.
Model
Invalid data registration
Observation
Model
• Can also be viewed as:
Observation with
station no.
Station
Model
Serial No.
Parameter
Parameter
Model
Serial No.
Parameter
Parameter
Parameter
Invalid data registration
Serial No.
Parameter
Parameter
Serial No.
Parameter
Parameter
Observation
Swiss Experiment
Interdisciplinary Environmental Research
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
Semantic Annotations
Label
Model
entities
A
Station
Station name, General information,
Geographical coordinate
B
Action
Station name, Parameter, Observation/Invalid
data, Start data, End date
C
Sensor
Sensor Serial no, Model no
D
Model
Model no, Manufacturer, Measured
quantities, Units
E
Parameter
Semantic Annotations
Parameter name, Station name, Sensor serial
no, Measured quantity, Units, Sampling
frequency, Measurement accuracy
Swiss Experiment
Interdisciplinary Environmental Research
HYDROMON
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
PERMASENSE
Semantic Forms
Metadata Database Interface
• We can use semantic wiki language
to store data in the background as
RDF
• Semantic wiki language is
complicated, not what scientists
need to deal with when they come
back from the field
• Semantic Forms take away this
complexity at the expense of
flexibility - a well designed
system can simplify data entry
Swiss Experiment
Interdisciplinary Environmental Research
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
Use of forms in entering semantics
Swiss Experiment
Interdisciplinary Environmental Research
HYDROMON
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
Station (Model Entity A)
Observations pertianing
to the overall station
General info on the station/vs
List of assigned sensors
Addition of observations to the
data or station/vs and notation of
invalid data periods
Parameters
All possible sensor models
Swiss Experiment
Interdisciplinary Environmental Research
COGEAR
HYDROMON
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
PERMASENSE
Action (Model Entity B)
Observation or invalid
data between 2 dates
Swiss Experiment
Interdisciplinary Environmental Research
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
Sensor (Model Entity C)
Association between parameter
name and model
Associated parameter names
Swiss Experiment
Interdisciplinary Environmental Research
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
Sensor Model (Model Entity D)
Model information
Association sensor instances (serial
numbers)
Swiss Experiment
Interdisciplinary Environmental Research
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
Parameter (Virtual Sensor) (Model Entity E)
Parameter information
Registered invalid data periods and observations
Dependencies
Swiss Experiment
Interdisciplinary Environmental Research
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
PERMASENSE
How do we make the tables?
Semantic Queries
Query languages for RDF databases:
• Standard Wiki
query
{{#ask:[[station name::{{{stationname}}}]] [[Sensor
serialno::+]]|?Sensor serialno|?Action|?Action at|format=table}}
• SPARQL query
PREFIX a:
<http://www.swiss-experiment.ch/index.php/Special:URIResolver/>
SELECT DISTINCT ?smodel
WHERE {?page a:Property:Sensor_model?smodel}
Swiss Experiment
Interdisciplinary Environmental Research
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
PERMASENSE
Semantic Queries
• BUT….
SPARQL can
query multiple
sources
PREFIX a:
<http://www.experiment1.ch/index.php/Special:URIResolver/>
PREFIX b:
<http://www.experiment2.ch/index.php/Special:URIResolver/>
SELECT ?smodel
FROM NAMED <wiki1.rdf>
FROM NAMED <wiki2.rdf>
WHERE {
GRAPH <wiki1.rdf> {
?x a:Property:Sensor_model ?smodel .
}.
GRAPH <wiki2.rdf> {
?y b:Property:Sensor_model ?smodel .
}.
}
• So metadata from A,B,C,D and E can be queried simultaneously
• SPARQL also gives us the possibility to query across metadata from different
experiments……so we have the possibility of querying the data based on its
metadata
Swiss Experiment
Interdisciplinary Environmental Research
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
Metadata Export using SPARQL
SPARQL is already used to query the wiki and
import metadata to SensorMap
Swiss Experiment
Interdisciplinary Environmental Research
HYDROMON
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
HYDROMON
PERMASENSE
Can I use the wiki as a front end for any
database?
• Importing to the Wiki from a relational database:
• Using R2D2 (http://aksw.org) SPARQL queries can be translated to SQL for
querying a relational database. The result is translated back to RDF format
using the same tool.
• The RDF files can be imported to a wiki page.
• Why is this interesting?
• If there are external data sources with metadata, we can query it and give
the results in the same place as the rest of the data
• Exporting to a relational database from the wiki:
• A script should be written to perform the relevant SPARQL query and wrap
the result into a SQL query.
Swiss Experiment
Interdisciplinary Environmental Research
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
MOUNTLAND
COGEAR
Conclusion
•
•
•
•
•
Framework for Metadata capturing
Integrated in a Wiki
Intuitive to use
Provide query interface to applications
Support for import/export to relational DBs
Swiss Experiment
Interdisciplinary Environmental Research
HYDROMON
PERMASENSE
TRAMM
HYDROSYS
SENSORSCOPE
GSN
SENSORMAP
BIGLINK
RECORD
APUNCH
EXTREMES
Questions
Swiss Experiment
Interdisciplinary Environmental Research
MOUNTLAND
COGEAR
HYDROMON
PERMASENSE