Transcript Cox-ADX

Using the Assay Data Exchange standard with WFS
to build a complete minerals exploration data-transfer chain
Simon Cox
CSIRO Exploration and Mining
5 September 2006
A.Dent, S.Girvan, R.Atkinson
I. Whitehouse, C. Legg
R.M.Woodcock
www.csiro.au
Outline
ADX history
OGC “Observations and Measurements”
ADX3 issues
Interoperability
ADX – IAMG 2006
History
ADX 1
 Aug-Sept 2002
 Sponsors: Metech, Anglo-American, Newmont, Placer-Dome,
WMC
 Contractor: Pindan Software
LIMS-focussed
XML document matched workflow, not information model
Iffy use of XML and XML Schema
ADX – IAMG 2006
History (2)
ADX 2
 May-Oct 2003 (report: Feb 2004)
 Sponsors: Metech, Anglo-American, Newmont, Placer-Dome,
WMC, BHP-Billiton, ALS Chemex
 Contractor: CSIRO
Introduced normalised Procedure/Specimen/Analysis model
Inherited some limitations from ADX1
ADX – IAMG 2006
ADX3 Goals and experiments
1.Harmonize ADX with generic standards (O&M) + GeoSciML
2.Refine Specimen + Procedure model
•
tracking specimen processing history is key to QA/QC procedures
•
tracking chain-of-custody is key to audit trail
3.Testing, feedback, documentation, examples, training

Collaborators: Metech, Newmont, ALS Chemex
ADX – IAMG 2006
Standard transfer format allows multiple data sources
WFS
Server
B
WFS
Client
WFS
Server
WFS
Server
C
ADX – IAMG 2006
Demonstrated: Same format and standards
XMML
User
XMML
GML for assay data:
XMML
ADX – IAMG 2006
XMML and OGC O&M
Completing the loop?
ADX
ADX-request?
Assay-WFS/T
Regulator
Assay Lab
Assay-WFS
“From field to desktop”
ADX – IAMG 2006
Design methodology: model in UML, serialize in XML
<sa:Specimen gml:id="sp1"
…>
<gml:description>Specimen test instance</gml:description>
<gml:name>Specimen test 1</gml:name>
<sa:location>
…
</sa:location>
<sa:sampledOn xlink:href="http://my.big.org/station/st13"/>
<sa:sampleTime ></gml:TimeInstant …
/></sa:sampleTime >
<sa:material codeSpace=“urn:x-seegrid:lexicon:specimenType">chips</sa:material>
<sa:mass uom=“kg">0.35</sa:mass>
GML Application Schema
<sa:reportingSpecimen xlink:href="http://my.big.org/specimen/sp34"/>
<sa:processingStep>
…
</sa:processingStep>
<sa:processingStep xlink:href="http://my.big.org/pst/p98"/>
<sa:relatedObservation xlink:href="http://my.big.org/obs/m45"/>
ADX – IAMG 2006
</sa:Specimen>
Generic “Observation” model
+precedingEvent 0..*
«FeatureType»
Event
«Union»
Procedure
+
+
+followingEvent 0..*
+
+
procedureType: ProcedureSystem
procedureUse: ProcedureEvent
+procedure
eventParameter: TypedValue [0..*]
time: TM_Object
«DataType»
TypedValue
+
+
property: ScopedName
value: Any
1
AnyDefinition
«ObjectType»
Phenomenon
+generatedObservation
0..*
+
+
+
0..*
1
AnyIdentifiableObject
«FeatureType»
Observ ation
quality: DQ_ThematicAccuracy [0..1]
responsible: CI_ResponsibleParty [0..1]
result: Any
+observedProperty
1
{Definition must be of a
phenomenon that is a property
of the featureOfInterest}
+propertyValueProvider
+featureOfInterest
«FeatureType»
AnyIdentifiableFeature
An Observation is an Event whose result is an estimate of the value
of some Property of the Feature-of-interest, obtained using a specified Procedure
- “metadata for a property estimate”
ADX – IAMG 2006
Assay measurements – specialized observation model
+precedingEvent 0..*
«FeatureType»
Event
«Union»
Procedure
+
+
+followingEvent 0..*
+
+
procedureType: ProcedureSystem
procedureUse: ProcedureEvent
+procedure
eventParameter: TypedValue [0..*]
time: TM_Object
«DataType»
TypedValue
+
+
property: ScopedName
value: Any
1
AnyDefinition
+generatedObservation
0..*
«FeatureType»
Observ ation
“Instrument/technique”
+
+
+
0..*
1
AnyIdentifiableObject
quality: DQ_ThematicAccuracy [0..1]
responsible: CI_ResponsibleParty [0..1]
result: Any
«ObjectType»
Phenomenon
+observedProperty
1
{Definition must be of a
phenomenon that is a property
of the featureOfInterest}
+propertyValueProvider
+featureOfInterest
“Concentration[analyte]”
«FeatureType»
AnyIdentifiableFeature
“Rock specimen”
Domain-specialization of observed property, feature of interest, procedure
ADX – IAMG 2006
Assay data issues
Procedure
Event
Phenomenon
+procedure
1
+observedProperty
+generatedObservation
0..*
Observ ation
0..*
+propertyValueProvider
1
+featureOfInterest
AnyIdentifiableFeature
Many measurements per (delivered) specimen:
•
different analytes
•
same analyte QA/QC
- “checks”, “splits”, “duplicates”, “replicates”
1. different instruments/labs
2. same instrument, different time
3. different specimen preparation (“splits”)
ADX – IAMG 2006
How to model measurements on split specimens?
customer viewpoint: one specimen, complex procedure
 compound “procedure” composed of
 preparation phase (as above)
 followed by analytical phase
lab viewpoint: many specimens, standard analytical procedure
 derived from incoming specimen, with different preparation histories
 multi-step preparation procedure
 splitting may occur at various points
 “biased” vs “unbiased” splits
 need to record exact splitting point …
 pulp may then go into several analytical systems
ADX – IAMG 2006
Generic “Specimen” model
«FeatureType»
SamplingFeature
+
+
«Union»
Procedure
property: TypedValue [0..*]
responsible: CI_ResponsibleParty [0..1]
+
0..1 +
procedureType: ProcedureSystem
procedureUse: ProcedureEvent
+preparationDetails
«FeatureType»
Specimen
+
+
+
currentLocation: Location [0..1]
currentSize: Measure [0..1]
materialClass: GenericName
«FeatureType»
LocatedSpecimen
+
+
+
ADX – IAMG 2006
samplingLocation: GM_Object
samplingMethod: ProcedureSystem [0..1]
samplingTime: TM_GeometricPrimitive
«Union»
Location
+
+
geometryLocation: GM_Object
nameLocation: EX_GeographicDescription
Procedure systems, events, histories
«Union»
Procedure
+
+
procedureType: ProcedureSystem
procedureUse: ProcedureEvent
Event
AnyDefinition
«ObjectType»
ProcedureSystem
+
instanceIdentifier: ScopedName [0..1]
+procedure
1
«FeatureType»
ProcedureEv ent
::Event
+ eventParameter: TypedValue [0..*]
+ time: TM_Object
+step
1..* {ordered}
Compose
«FeatureType»
ProcedureHistory
ADX – IAMG 2006
Standard procedure?
preparation and analytical phases

based on a standard design sequence (“path”),

but a specific instance (“history”)
1. has parameters recorded for each step

% passing a given sieve, weight-in/weight-out, instrument
settings, … etc
2. may vary from the prototype
How much information can be “inherited”, how much
should be explicit …?
N.B. This information does matter:

analysis of results

billing!
ADX – IAMG 2006
Different encoding strategies
1. fully explicit “UML→GML” – direct from UML model to an XML document
•
apparently verbose - 40-60 lines of XML per split
•
much duplication between splits
2. manual optimization – inherit parameters from standard procedures
•
less duplication
•
indirect mapping to information model– XML must be maintained separately from model,
processing requires more conditions and loops
Concerned about file-size?
Raw XML
Zipped XML
Explicit
1470 kB
39 kB
Hand-optimised
650 kB
36 kB
ADX – IAMG 2006
Summary
ADX can potentially harmonize with GeoSciML, O&M
challenge in modeling prep/analytical procedures
let Shannon take care of document size issue
standards support interoperability, lead to round-tripping
ADX – IAMG 2006
CSIRO Exploration and Mining
Name
Simon Cox
Title
Research Scientist
Phone
+61 8 6436 8639
Email
[email protected]
Web
www.seegrid.csiro.au
Thank You
Contact CSIRO
Phone
1300 363 400
+61 3 9545 2176
Email
[email protected]
Web
www.csiro.au
www.csiro.au
Conceptual object model: features
Digital objects correspond with
identifiable, typed,
objects in the real world
 mountain, road, specimen, event,
tract, catchment, wetland, farm, bore,
reach, property, license-area, station
Feature-type is characterised by a
specific set of properties
Specimen
 ID (name)
 description
 mass
 processing details
 sampling location
 sampling time
 related observation
 material
 …
ADX – IAMG 2006
Why is a standard methdology useful?
GML-based encoding is supported by
standard web data access protocols
from Open Geospatial Consortium
ADX – IAMG 2006
Is any of this for real?
SEEGrid interoperability demonstrator 2004-5
ADX – IAMG 2006
CLIENT
APPLICATIONS
Reports
Mapping
Common Interface Binding – GML/XMML
DATA
ACCESS
SERVICES
DOIR
Web Feature Translation
Service (WFS)
Geoserver
(Open Source)
DATA
SOURCES
ADX – IAMG 2006
DOIR
Geochemistry
Little or
Feature
Data Source
PostGIS
(Open Source)
GA
to standards here
Web Feature Service (WFS)
PIRSA
Web Feature Service (WFS)
no
PIRSA
change
required
Geochemistry
Feature
Data Source
PostGIS
(Open Source)
here
GA
Geochemistry
Feature
Data Source
Oracle
Completing the loop
ADX
ADX-request?
Assay-WFS/T
Regulator
Assay Lab
Assay-WFS
“From field to desktop”
ADX – IAMG 2006
Assignment of property values
Each property value is either
i. asserted
 name, owner, price, boundary (cadastral feature types)
ii. estimated
 colour, mass, shape (natural feature types)
 i.e. error in the value is of interest
ADX – IAMG 2006
Observations and Features
An estimated value is determined through observation
i.e. by application of an observation procedure
ADX – IAMG 2006
Feature of interest
may be any feature type from any domain-model …
observations provide values for properties whose values are not
asserted
i.e. the application-domain supplies the feature types
ADX – IAMG 2006
Some feature types only exist to support observations
Surv eyProcedure
+surveyDetails
+member 0..*
0..1
SamplingFeature
+
SamplingFeatureCollection
responsible: CI_ResponsibleParty [0..1]
Specimen
+
+
+
constraints
{count(member)>=1}
Station
+
+
Profile
elevation: DirectPosition [0..1]
position: GM_Point
Interv al
+
+
+
SurfaceOfInterest
begin: GM_Point
end: GM_Point
length: RelativeMeasure [0..1]
Trav erse
+
currentLocation: Location [0..1]
mass: Measure
material: CV_Coverage
SolidOfInterest
area: RelativeMeasure [0..1]
Sw ath
+
volume: RelativeMeasure [0..1]
LidarCloud
Sounding
Flightline
Section
+shape
Shape1D
ADX – IAMG 2006
1
+shape
Shape2D
1
+shape
1
Shape3D
Observation model
«FeatureType»
Event
«Union»
ObsProcedure
+
+
+
+
«DataType»
TypedValue
eventParameter: TypedValue [0..*]
time: TM_Object
+
+
property: ScopedName
value: Any
generalProcedure: ObservationProcedure
sensorProcedure: SensorML
+procedure
1
«ObjectType»
Phenomenon
+generatedObservation
0..*
0..*
1
«FeatureType»
Observ ation
+
+
+
+
quality: DQ_DataQuality [0..1]
responsible: CI_ResponsibleParty [0..1]
result: Any
resultDefinition: RecordType [0..1]
+propertyValueProvider
+featureOfInterest
«FeatureType»
AbstractFeature
1
Generic Observation has dynamically typed result
ADX – IAMG 2006
+observedProperty
Observation specializations
Override result type
ADX – IAMG 2006
Observations support property assignment
These must match if the observation is coherent with the feature property
+observedProperty
Mass :
Phenomenon
Observation
Measurement
+
result: RelativeMeasure
+featureOfInterest
SamplingFeature
SamplingFeature
+propertyValueProvider
Specimen
Specimen
currentLocation: Location
Location [0..1]
[0..1]
++ currentLocation:
mass: Measure
Measure
++ mass:
material: CV_Coverage
CV_Coverage
++ material:
+observedProperty
Material :
Phenomenon
+propertyValueProvider
Observation
+featureOfInterest
Cov erageObserv ation
+
result: CV_DiscreteCoverage
N.B. Each “Phenomenon” definition reifies a (Feature) property definition
ADX – IAMG 2006
Observations support property assignment
Mass :
Phenomenon
+observedProperty
Observation
Measurement
+
+procedure
Scales :
Observ ationProcedure
result: RelativeMeasure
+featureOfInterest
SamplingFeature
SamplingFeature
+propertyValueProvider
Specimen
Specimen
++ currentLocation:
currentLocation: Location
Location [0..1]
[0..1]
++ mass:
mass: Measure
Measure
++ material:
material: CV_Coverage
CV_Coverage
+observedProperty
+propertyValueProvider
Observation
+featureOfInterest
Cov erageObserv ation
+
ADX – IAMG 2006
Material :
Phenomenon
result: CV_DiscreteCoverage
+procedure
MicroProbe :
Observ ationProcedure
Sensor service
premises:
O&M is the high-level information model
SOS is the primary information-access interface
SOS can serve:
an Observation (Feature)
 getObservation == “getFeature” (WFS/Obs) operation
a feature of interest (Feature)
 getFeatureOfInterest == getFeature (WFS) operation
or Observation/result (often a time-series == discrete Coverage)
 getResult == “getCoverage” (WCS) operation
or Sensor == Observation/procedure (SensorML document)
 describeSensor == “getFeature” (WFS) or “getRecord” (CSW) operation
ADX – IAMG 2006
optional – probably required for
dynamic sensor use-cases
SOS vs WFS, WCS, CS/W?
getFeature,
type=Observation
WFS/
Obs
getObservation
getCoverage
getResult
describeSensor
getCoverage
(result)
WCS
SOS
getFeatureOfInterest
getRecord
SOS interface is effectively a composition of
(specialised) WFS+WCS+CS/W operations
Sensor
Registry
getFeature
WFS
e.g. SOS::getResult == “convenience” interface for WCS
ADX – IAMG 2006
Non-map use of the service
ADX – IAMG 2006
Tabular Reports by Source
Three servers, one report
ADX – IAMG 2006
One service – many uses
GML-based data can be ….
Rendered into a
queryable map …
… formatted into a
report or ….
Condition: community-standard data model/encoding
… read and used by any
WFS/GML enabled application
ADX – IAMG 2006
Mapping client
http://cgsrv3.arrc.csiro.au/seegrid/savedapps/filter
Three servers, one map
DOIR
ADX – IAMG 2006
PIRSA
GA
Initial Map Interface – HTML & SVG
Click a Map
Feature
Bounding Box
Layers
ADX – IAMG 2006
…and see all Analytes on that sample
ADX – IAMG 2006
Desktop Visualisation (Fractal Technologies)
ADX – IAMG 2006
Now – Different formats and standards
Proprietary
Software
Versions of
Software
Client
Data
Structures
ADX – IAMG 2006