Transcript 20160616_QGIS_GDAL_OGR__complex_feature_Grelletx

Supporting GML application
compliant complex features in
QGIS and beyond
BRGM – S.Grellet
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
Overall context
NO DATA
?
NO CLIENT
« another approach to demonstrate
the usefulenessof interoperable standards »
or
« having something to show to those
who consider XML is not sexy »
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
>2
Overall context
> Few applications are able to handle GML flows
compliant to an application schema
•
Appart when hardcoded
> Initial idea :
• A correct xml content points to its xsd(s )
 Can we generate a database structure from what is declared in the
xsd ?
Having a generic solution with no hardcoded configuration ?
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
>3
Overall context
> Proof Of Concept exercise
> Desktop open source application : QGIS
> Use case :
•
•
Ground Water quantity monitoring
Reading data, not writing
> Complex Feature Flows
•
WFS
•
SOS
– Piezometers : Inspire Environmental Monitoring Facility theme
– Aquifer : Inspire Geology-Hydrogeology, GWML2
– WaterML2
> Project started Fall 2015
•
With core QGIS commiters
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
>4
Technological choices
> Input : XML File or URL
> Solution to handle XML types
•
•
Xerces (C++) : supports “Schema API” and esp. “post-schemavalidation infoset” to access schema information (thus types)
Xsd -> Object model conversion
– Code Synthesis (C++) and PyXB (Python)
 PyXB
> XML parsing
•
•
Xsd caching
Well-formed XML = minimum. Degraded mode for ‘non-perfect’ xml
> ‘Content negociation’ on xlink:href
•
•
For codeLists : test on Inspire vocabulary registry, Sandre, …
For Features
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
>5
Technological choices
> Database :
•
PostGIS / SpatialLite
> Simplification / dynamic flatening parameter
> Respecting application schema
•
•
•
Structure
Types
Referential integrity
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
>6
Results – data content context
> Ex : Environmental Monitoring Facility : piezometers
class Env ironmentalMonitoringFacilities
EF-Level
+supersedes
0..*
+narrower
0..*
«featureType»
AbstractMonitoringObject
Hierarchy
http://inspire.ec.europa.eu/schemas/ef/4.0/
genealogy
Observation and ObservingCapability
/* If Observation(s) are attached to an AbstractMonitoringFeature this must have
an ObservingCapability attached to it. The ObservingCapability must reference
the same Domain, Phenomenon and ProcessUsed as the Observation. */
inv: hasObservation->notEmpty() implies observingCapability->notEmpty() and
hasObservation.OM_Observation.featureOfInterest =
observingCapability.featureOfInterest and
hasObservation.OM_Observation.observedProperty =
observingCapability.observedProperty and
hasObservation.OM_Observation.procedure = observingCapability.procedure
+supersededBy
0..*
«featureType»
AbstractMonitoringFeature
+
+
+
+
+
inspireId :Identifier
name :CharacterString [0..*]
additionalDescription :CharacterString [0..1]
mediaMonitored :MediaValue [1..*]
geometry :GM_Object [0..1]
«voidable»
+ legalBackground :LegislationCitation [0..*]
+ responsibleParty :CI_ResponsibleParty [0..*]
+ onlineResource :CI_OnlineResource [0..*]
+ purpose :PurposeOfCollectionValue [0..*]
«featureType»
Env ironmentalMonitoringActiv ity
+uses
0..*
«voidable»
+ reportedTo :ReportToLegalAct [0..*]
hierarchy
+
«voidable»
+ linkingTime :TM_Object
+broader
0..1
«featureType»
Env ironmentalMonitoringProgramme
+triggers
0..*
inspireId :Identifier
Environmental
Monitoring Facilities
GeometryRequired
/* Geometry and
representativePoint can't be
empty at the same time.*/
inv: geometry ->notEmpty() or
representativePoint ->
notEmpty()
+involvedIn
0..*
constraints
{Observation and ObservingCapability}
«featureType»
Env ironmentalMonitoringFacility
+setUpFor
0..*
«voidable»
+ activityTime :TM_Object
+ activityConditions :CharacterString
+ boundingBox :GM_Boundary [0..1]
+ responsibleParty :CI_ResponsibleParty
+ onlineResource :CI_OnlineResource [0..*]
«featureType»
Env ironmentalMonitoringNetw ork
«voidable»
+ representativePoint :GM_Point [0..1]
+ measurementRegime :MeasurementRegimeValue
+ mobile :Boolean
+ resultAcquisitionSource :ResultAcquisitionSourceValue [0..*]
+ specialisedEMFType :SpecialisedEMFTypeValue [0..1]
«voidable»
+ organisationLevel :LegislationLevelValue
+contains
0..*
constraints
{GeometryRequired}
+belongsTo
0..*
+relatedTo
0..*
Netw orkFacility
«voidable»
+ linkingTime :TM_Object
AnyDomainLink
+observingCapability
0..*
«voidable»
+ comment :CharacterString
1..*
«featureType»
Observ ingCapability
+operationalActivityPeriod
«voidable»
«voidable»
+ observingTime :TM_Object
+ processType :ProcessTypeValue
+ resultNature :ResultNatureValue
+ onlineResource :URI [0..1]
«featureType»
OperationalActiv ityPeriod
+
hasObservation:
Link from WFS to SOS
activityTime :TM_Object
+hasObservation
«voidable»
0..*
«FeatureType»
observ ation::OM_Observ ation
+
+
+
+
+
Phenomenon
Domain
ISO FDIS 19156:2011 Observations and Measurements
+propertyValueProvider +featureOfInterest
«FeatureType»
General Feature Instance::GFI_Feature
+observedProperty
0..*
+result
1
«metaclass»
General Feature Model::GF_PropertyType
{root}
+
1 +
+generatedObservation
+procedure
«FeatureType»
observation::OM_Process
+procedure
ProcessUsed
0..*
1
+relatedObservation
0..*
observ ation::
Observ ationContext
+
role :GenericName
GCM Base Types 2
Additional classes from GCM Observations
Base Types 2::DocumentCitation
+
+
name :CharacterString
shortName :CharacterString [0..1]
«voidable»
+ date :CI_Date
+ link :URL [1..*]
DataType
«dataType»
ReportToLegalAct
+
+
+
1
memberName :LocalName
definition :CharacterString
realises
«type»
Records and Class
Metadata::Any
{root}
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
+observedProperty
1
Phenomenon
constraints
{observedProperty shall be a phenomenon associated with
the feature of interest}
{procedure shall be suitable for observedProperty}
{result type shall be suitable for observedProperty}
{a parameter.name shall not appear more than once}
Range
ProcessUsed
+featureOfInterest
ISO 19156:2011
Observations and
Measurements
Domain
0..*
phenomenonTime :TM_Object
resultTime :TM_Instant
validTime :TM_Period [0..1]
resultQuality :DQ_Element [0..*]
parameter :NamedValue [0..*]
0..1
legalAct :LegislationCitation
reportedEnvelope :URI [0..1]
description :CharacterString [0..1]
Base Types 2::LegislationCitation
+
+
+
+
identificationNumber :CharacterString [0..1]
officialDocumentNumber :CharacterString [0..1]
dateEnteredIntoForce :TM_Position [0..1]
dateRepealed :TM_Position [0..1]
«Type»
Observable Properties::
AbstractObservableProperty
«featureType»
Processes::Process
INSPIRE O&M
extensions > 7
+
label :CharacterString [0..*]
«voidable»
+ documentation :DocumentCitation [0..*]
+ inspireld :Identifier
+ name :CharacterString [0..1]
+ processParameter :ProcessParameter [0..*]
+ responsibleParty :CI_ResponsibleParty [1..*]
+ type :CharacterString
«dataType»
Processes::ProcessParameter
Results – plugin modes
> XML: handling the XML file as is with additionnal
intelligence on top
•
•
•
Xpath to geometry(ies) information
Ability to handle hlink:href
Identification of TimeSeries
> Relational database : creating database structure in
SpatialLite
• Maximum table merging depth parameter : dynamic flattening
 adjust the verbosity of the resulting database structure
•
‘Merge unitary sequences’ : for Attributes with max cardinality * in
the xsd but only present once in the xml
 avoid un-necessary additional tables
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
>8
Results – XML Mode
> On WFS flow
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
>9
Results – XML Mode
> On SOS flow
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
> 10
Results – Relational database Mode
> ex : WFS flow
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
> 11
Results – Relational database Mode
> ex : WFS flow
•
Schema representation within QGIS to ease navigation
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
> 12
Results – Relational database Mode
> ex : SOS flow
•
Data structure whithin Database Manager
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
> 13
Results – Relational database Mode
> ex : SOS flow
• Data content is here
 We can deploy domain plugins for our domain colleagues
Ex : analyse trends, correct values, …
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
> 14
Further test results
> Data structure generation is quite fast on
reasonable datasets
•
Matter of seconds
> Tested on others application schema compliant
dataflows
•
Geology : European Geological Data Infrastructure
•
Mineral resources : Minerals4EU
– Inspire Geology
– GeoSciML V4
– Inspire Mineral resources
 Works quite well for a POC
 Both modes have their utility
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
> 15
Next steps
> 1a°/ Publish the plugin
•
•
Name : ‘QGIS GML Application Schema Toolbox’
In the official QGIS repository
> 1b°/ Gather feedbacks
•
•
HydroDWG, GeoSciML, ….
Inspire
 Have people use it…
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
> 16
Next steps
> 2°/ Enhance the work (whishlist)
•
Port the approach to GDAL/OGR
•
Enable application schema compliant writing
•
Improve the UI
•
Add domain useful widget (Hydrogeologists, …)
 Nothing in the code is Inspire specific so this work
could be useful to the overall OGC community.
 Have other structures contribute to its development
•
Ex : Study under COPERNICUS umbrella to enhance INSPIRE data
consumption/use within QGIS
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
> 17
Thank you
[email protected]
OGC Hydro DWG Workshop 2016 – Koblenz – 2016-06-16
> 18