Transcript PPDM Australian Data Management Conference 2004

Australian Government
Geoscience Australia
ACHIEVING PRACTICAL CROSSJURISDICTIONAL INTEROPERABILITY
FOR THE AUSTRALIAN MINING
INDUSTRY
Lesley Wyborn and Stuart Girvan
Geoscience Australia, pmd*CRC
Outline of Presentation
•
This presentation is about the Australian AUSindustry
‘Interoperability for Geospatial Data Project’
•
Our presentation will be based on the OpenGIS®
Reference Model
•
We will discuss:
•
o
what were the drivers for this project
o
which pieces of the standard service orientated architecture
were developed
o
whether other communities and domains can use our results
We will elaborate on what were the most important
lessons learnt
PPDM Australian Data Management Conference 2004
Geoscience Australia
Interoperability for Geospatial Data Project:
Acknowledgements
People
o
o
o
o
Simon Cox, Rob Woodcock, Joan Esterle – CSIRO
Stuart Girvan – Predictive Mineral Discovery CRC
Tim Mackey, Aaron Sedgmen – Geoscience Australia
Rob Atkinson, Peter Barrs – Social Change Online
Funding Organisations (~$250K cash)
o
o
o
o
o
AUSIndustry
pmd*CRC
Minerals Council of Australia
Geoscience Australia
Every State and Territory Geoscience Agency
PPDM Australian Data Management Conference 2004
Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
What
language
are we
speaking?
Information
Viewpoint
Business drivers, policies
Who else is doing it?
What
Computational
components
Viewpoint
& Interfaces?
Engineering Viewpoint
Who does what?
Technology Viewpoint
What technology do
we use?
Source http://www.opengeospatial.org/specs
PPDM Australian Data Management Conference 2004
Geoscience Australia
Key Driver: Minerals Exploration
Action Agenda – June 2004
Industry input highlighted
•
problems in gaining access to pre-competitive geoscience
information.
•
described existing information as commonly incomplete and
fragmented across eight government agencies, each with its own
information management systems and structures.
•
noted that the disparate systems lead to inefficiencies causing
higher costs, reduced effectiveness and increased risk incurred by
the industry and its service providers
PPDM Australian Data Management Conference 2004
Geoscience Australia
Key Driver: Minerals Exploration
Action Agenda – June 2004
Key Initiative
o
Australian Government, State and Territory geoscience
agencies, professional associations and industry to
cooperatively develop and implement nation-wide protocols,
standards and systems that provide internet-based access to,
and effective storage and archiving of, industry and government
exploration-related DATA
Recommended actions
o
Implement web-based services for on-line access
o
Develop and endorse a plan for implementation of an Australian
Earth Science Grid
PPDM Australian Data Management Conference 2004
Geoscience Australia
The Geoscience Portal is an index of websites
NT
Databases
SA
Databases
Queensland
Databases
Victorian
Databases
NSW
databases
GA
Databases
WA
Databases
Tasmanian
databases
Client
has to
down
load
and then
locally
merge
each
file
into
single
data
set
To merge common data types from all surveys either
1. All data sets are downloaded from each source and then combined or
2. All data are combined into a single data set and hosted by one source: currency
of data is a major issue
Performance per Dollar Spent
Global Driver: the exponentially increasing
digital world
0
Doubling Time
(months)
9 12
Therefore:
Optic Fibre
(bits per second)
No long do data,
programs and computer
grunt have to be local
•
We need to switch to a
distributed culture, ie an
interoperable culture
where data, programs
and computer grunt are
‘operable’ regardless of
where they are housed
18
Data Storage
(bits per square inch)
Silicon Computer Chips
(number of transistors)
(ie Moore’s Law)
1
•
2
3
Number of Years
4
5
Moore’s Law vs. storage improvements vs. optical improvements. Graph from Scientific American (Jan-2001)
by Cleo Vilett,
source Vined Khoslan,
Kleiner,
Caufield and Perkins.
PPDM Australian
Data Management
Conference
2004
Geoscience Australia
Evolution of the Internet
Application
People
People
People
Content
Application
PPDM Australian Data Management Conference 2004
Geoscience Australia
Source: http://www.dstc.edu.au/Tech_Transfer/Events/Canberra/web_services_cnb02.pdf
Web Map Service (WMS) can access multiple maps
Borders
Cloud cover
Elevation
Cities
Multiple
overlaid
maps
One GetMap request:
Source: Reed, C., 2004. Data Integration and Interoperability: ISO/OGC Standards for Geo-Information
http://www.directionsmag.com/article.php?article_id=687
PPDM Australian Data Management Conference 2004
Geoscience Australia
WMS can’t “give data away.”
Roma
WMS GetMap returns a server’s
“dumb” JPEG, GIF or PNG
representation of the data on the
server. It does NOT return the actual
data, only a bitmap of the data.
Source: Reed, C., 2004. Data Integration and Interoperability: ISO/OGC Standards for Geo-Information
http://www.directionsmag.com/article.php?article_id=687
PPDM Australian Data Management Conference 2004
Geoscience Australia
Web Feature Service (WFS) returns data.
Web Feature
Server
GetFeature
request:
Feature &
attribute
data
I-295
Source: Reed, C., 2004. Data Integration and Interoperability: ISO/OGC
Standards for Geo-Information
http://www.directionsmag.com/article.php?article_id=687
PPDM Australian Data Management Conference 2004
Geoscience Australia
Web Feature Service (WFS) gets operable
feature data from multiple servers
Elevation
Cities
Borders
Each layer is data, not
merely a view:
Country is:
• Name: Italy
• Population: 57,500,000
• Area: 301,325 sq km. . .
Multiple
thematic
data layers
GetFeature
request:
Source: Reed, C., 2004. Data Integration and Interoperability: ISO/OGC Standards for Geo-Information
http://www.directionsmag.com/article.php?article_id=687
PPDM Australian Data Management Conference 2004
Geoscience Australia
Why seek funding from the AUSIndustry
Innovation Access Program (IAccP)
•
AUSIndustry is part of the Department of
Industry, Tourism and Resources
•
The goal of the IAccP is to promote innovation
and competitiveness by improving Australian
access to global, leading-edge research and
technologies and facilitate their uptake by
Australian firms, particularly by SME’s and
researchers.
•
The IAccP is a technology diffusion program
PPDM Australian Data Management Conference 2004
Geoscience Australia
External links – who else is doing it?
This project provides a mechanism to understand and
share practice with other domains
Australian:
o
o
o
o
o
o
ASIBA Spatial Interoperability Demonstration Project (SIDP)
Australian Spatial Data Infrastructure (ASDI)
Australian Earth and Ocean Network (AEON)
Australia Partnership for Advanced Computing (APAC)
National Oceans Office (NOO)
Australian Antarctic Division (AAD)
International
o
o
o
GEON (Geoscience Network) - US
NERC Datagrid (UK National Environmental Research Council)
IUGS CGI (International Union of Geological Sciences)
PPDM Australian Data Management Conference 2004
Geoscience Australia
Policy Issues for our Project
•
Pricing and Charging
o
o
•
Managing Intellectual Property
o
o
•
For both Commonwealth, State and Territory
Geoscience Agencies, all data are free over the internet
Thus we did not have to build charging into our project
Easy to put data source and request for client to
acknowledge into data set
Acknowledgement of the data source
at the destination is difficult to enforce
Copyright and licensing
o
o
o
o
Source Agency still maintains copyright
They still need to know who is using it
How this is done is a global issue
Geoscience Australia is participating in the OGC project
on Digital Rights Management (DRM)
PPDM Australian Data Management Conference 2004
Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
What
language
are we
speaking?
Information
Viewpoint
Business drivers, policies
Who else is doing it?
What
Computational
components
Viewpoint
& Interfaces?
Engineering Viewpoint
Who does what?
Technology Viewpoint
What technology do
we use?
Source http://www.opengeospatial.org/specs
PPDM Australian Data Management Conference 2004
Geoscience Australia
Computational Viewpoint:
Standard Web Services Architecture
Registry
publish
find
Client
Applications
bind
Processing
Services
Model
Management
Services
Services
Data Access Services
PPDM Australian Data Management Conference 2004
Geoscience Australia
Computational Viewpoint:
Notional Architecture
Ontology
Feature
Type
Catalog
Service
Metadata
Persistent
Bindings
Dataset
Metadata
Registry
find
Client
Applications
publish
bind
Processing
Services
Model
Management
Services
Services
Data Access Services
Features
Coverages
Real-time
data
Models
Computational Viewpoint: our simplified
architecture of components and interfaces
Registry
publish
find
Client
Applications
bind
Model
Management
Services
Processing
Services
Services
Data Access Services
For our demonstrator we are
1. not using registries to find or publish services
2. focusing on delivering data as operable ‘features’
3. hard wiring the applications to the data services
PPDM Australian Data Management Conference 2004
Geoscience Australia
Computational Viewpoint: registries &
catalogues
These people are
developing/thinking
about developing
registries and
catalogues in
Australia
NOO, QLD NRE, WA DLI, SEEGRID
Registry
publish
find
Client
Applications
bind
Services
Data Access Services
PPDM Australian Data Management Conference 2004
Geoscience Australia
Who else is doing what in OZ in the
Computational Viewpoint
Portrayal
Services
Processing
Services
Services
Data Access Services
This
Project/
SEEGrid
WFS
Data
Services
WCS
SCS
Archives
WMS
Coverage
Portrayal
Service
Portrayal Services
ASIBA
CANRI
WA DLI
QLD NRE
GA etc
PPDM Australian Data Management Conference 2004
Geocoding
Coord.
Transf.
Service
Processing
Services
Chaining
Gazetteer
WA DLI
GA
Geoscience Australia
Our Computational Viewpoint of
Components and Interfaces
and on binding these
to client applications
Client
Applications
Services
bind
Data Access Services
This project is focusing on
Data Access Services
PPDM Australian Data Management Conference 2004
Geoscience Australia
Main International Standards used in Project
•
OGC Services
o
o
•
OGC Languages
o
o
•
Geography Markup Language (GML)
eXploration and Mining Markup Language (XMML)
W3C Languages
o
•
Web Feature Service (WFS)
Web Map Service (WMS)
eXtensible Markup Language (XML)
W3C Web services
o
Simple Object Access Protocol (SOAP)
PPDM Australian Data Management Conference 2004
Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
What
language
are we
speaking?
Information
Viewpoint
Business drivers, policies
Who else is doing it?
What
Computational
components
Viewpoint
& Interfaces?
Engineering Viewpoint
Who does what?
Technology Viewpoint
What technology do
we use?
Source http://www.opengeospatial.org/specs
PPDM Australian Data Management Conference 2004
Geoscience Australia
Who is doing what in Australia in language
standards to enable data binding
Client
Applications
Services
bind
Data Access Services
GML
CSIRO
SEEGRID
OGC
XIMA
SLD
Service
SensorML Obs&Meas
Capabilities
Encodings
Image
Metadata
PPDM Australian Data Management Conference 2004
Geoscience Australia
Traditional GIS data models for Geosceince
do not allow the same data to be viewed
seamlessly in 2D, 3D, 4D
•
Data are in points, lines and polygons
Tenement
•
•
•
Geometry-centric abstraction relates to
the implementation, not the business object
One shape per feature does not allow
multiple spatial properties,
scale-dependent versions …
PPDM Australian Data Management Conference 2004
Geoscience Australia
Information Viewpoint in the OZ Geosciences
•
Borehole






Basin 




We use ‘feature’ models which:
o
collar location
shape
collar diameter
Fault
length
operator
logs
related observations
formations
…
shape – time dependent
resource estimate
…
o





are based on geological concepts
allow for multiple geometries in
2D, 3D, and 4D
shape
surface trace
displacement
age
…
Observation







location
subject/specimen/station
property/theme/measurand
method
operator
date/time
result (+ type/reference
Ore-body






commodity
deposit type
host formation
shape (point, polygon, 3D shell)
resource estimate
…
system/scale/classification)
…
PPDM Australian Data ManagementConference
2004
Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
What
language
are we
speaking?
Information
Viewpoint
Business drivers, policies
Who else is doing it?
What
Computational
components
Viewpoint
& Interfaces?
Engineering Viewpoint
Who does what?
Technology Viewpoint
What technology do
we use?
Source http://www.opengeospatial.org/specs
PPDM Australian Data Management Conference 2004
Geoscience Australia
SEEGrid Community Engineering Viewpoint
SEEGrid is helping
to coordinate the
building of an
interconnected
distributed
modelling and
computational
services, whereby
the models as well
as the input data
and modelling
programs are
preserved in a
model library
Our Project’s Focus
PPDM Australian Data Management Conference 2004
Geoscience Australia
Our project’s Engineering Viewpoint
Client
Applications
Services
bind
PPDM Australian Data Management Conference 2004
Data Access Services
Geoscience Australia
Our projects Engineering Viewpoint in Detail
(who engineered what?)
pmd*CRC, GA, CSIRO,
Social Change On-line
Client
Applications
Bind
South Australia
SA
Web Feature
Service (WFS)
SA Geochemistry
Feature
Data Source
Western Australia
WA
Web Feature
Service (WFS)
WA Geochemistry
Feature
Data Source
Gescience Australia
GA
Web Feature
Service (WFS)
PPDM Australian Data Management Conference 2004
GA Geochemistry
Feature
Data Source
Geoscience Australia
OpenGIS® Reference Model
Enterprise Viewpoint
What
language
are we
speaking?
Information
Viewpoint
Business drivers, policies
Who else is doing it?
What
Computational
components
Viewpoint
& Interfaces?
Engineering Viewpoint
Who does what?
Technology Viewpoint
What technology do
we use?
Source http://www.opengeospatial.org/specs
PPDM Australian Data Management Conference 2004
Geoscience Australia
Our Project’s Technology Viewpoint:
what technology do we use?
Client
Applications
Services
bind
PPDM Australian Data Management Conference 2004
Data Access Services
Geoscience Australia
Our projects Technology Viewpoint in Detail
XMML,
GML
(Open Source)
SA
Web Feature
Service (WFS)
(Proprietary)
SA Geochemistry
Feature
Data Source
PostGIS (Open Source)
Western Australia
Bind
WA
Web Feature
Service (WFS)
GA Reporting
Application
GA PLOT-IT
Application
PostGIS (OpenSource)
South Australia
Web Map
Composer
Client
Applications
Geoserver
WA Geochemistry
Feature
Data Source
Oracle (Proprietary)
Geoscience Australia
GA
Web Feature
Service (WFS)
PPDM Australian Data Management Conference 2004
GA Geochemistry
Feature
Data Source
Geoscience Australia
Our Technology for distributing data
•
We used Geoserver – open source
•
Needed modification to be able to
o
o
•
Source data from complex database (many tables of data)
Produce complex application schemas (ie more complex
than simple GML such as XMML)
Other WFS software*:
o
o
Degree (open source), Cadcorp SIS Map Server, CARIS
Spatial Fusion Enterprise, RedSpider Web 3.0, Map
Manager, GenaWare, SclFeatureServer, JCarnacGIS,
GeoMedia WebMap, MapXtreme
(MapInfo), MapServer(UMN) 4.2 + more
We do not know if any of these can do complex mapping
* Source: OGC Registered Products
PPDM Australian Data Management Conference 2004
Geoscience Australia
The Geoscience Portal is an index of websites
NT
Databases
SA
Databases
Queensland
Databases
Victorian
Databases
NSW
databases
GA
Databases
WA
Databases
Tasmania
databases
Client
has to
down
load
and
then
locally
merge
each
file
into a
single
data
set
To merge common data types from all surveys either
1. All data sets are downloaded from each source and then combined or
2. All data are combined into a single data set and hosted by one source: currency
of data is a major issue
PPDM Australian Data Management Conference 2004
Geoscience Australia
Project results: the Geoscience Portal will
now have these wfs components to enable
•
•
access of SA, WA, GA geochem data in real time via common interfaces
each participant to develop their database to suit their business needs
SA
Geochem
Interface
Chem
SA
Databases
WA
Geochem
Interface
Chem
WA
Databases
GA
Geochem
Interface
wfs interfaces still needed for these Surveys
NSW
databases
Victorian
Databases
Queensland
Databases
Tasmania
databases
Northern Territory
Databases
Chem
GA
Databases
Many other
data types
besides
geochem!!!
Information Viewpoint requires the most work
Enterprise Viewpoint
What
language
are we
speaking?
Information
Viewpoint
Business drivers, policies
Who else is doing it?
What
Computational
components
Viewpoint
& Interfaces?
Engineering Viewpoint
Who does what?
Technology Viewpoint
What technology do
we use?
Source http://www.opengeospatial.org/specs
PPDM Australian Data Management Conference 2004
Geoscience Australia
XMML schema’s developed by CSIRO
•
•
•
•
•
•
•
•
•
•
•
•
•
Borehole
Observation (OGC SensorWeb)
Gravity measurement
implemented but only
Geochemistry/Assay result
for WA, SA and GA
Geological material
Geological timescale
Mineral occurrence
Procedure, Project, Station, Specimen, Tenement
Point, Curve, Surface, Solid volume with properties
Structural geology
Time-series
Finite element model (FLAC, FastFlo)
Simulation/model state
PPDM Australian Data Management Conference 2004
Geoscience Australia
Who is the XMML Community?
•
•
•
•
The XMML community led by CSIRO is one of the
most advanced globally – why?
The XMML community has been going since 2000
Data types within each domain have different
communities
For XMML
o
o
•
•
For locations community is global – this led to GML3
For Geochemistry metadata, community is the Australian
Mining industry
This illustrates the complexity of developing
community schema
There is no national or international coordinating
agency
PPDM Australian Data Management Conference 2004
Geoscience Australia
Introducing the
Solid Earth and
Environmental
Grid (SEEGrid)
Community
languages
http:www.seegrid.csiro.au
Community pages
PPDM Australian Data Management Conference 2004
Geoscience Australia
Lesson’s learnt: Business Issues
•
•
Interoperability represents perhaps the most significant
paradigm shift in how data and information are managed
and utilised since the emergence of the Internet
The Interoperable world is a Markedly Different Business
Paradigm: change is from ‘supply’ to ‘demand’
o
o
•
In the supply or ‘push’ paradigm, suppliers push their content to
clients, how they think their clients want to utilise that content
In the demand or user ‘pull’ paradigm clients access and use
what data and information they require, from any data supplier
that can supply authoritative data in the appropriate
standardised formats.
For the Interoperable paradigm, data suppliers must be
able to map their content word for word to international
standard interfaces.
Source: Williams, Neil, 2004. Interoperability – responding to National Drivers.
http://www.osdm.gov.au/osdm/docs/resources/osdm_interoperability_con_03112004/neilwilliams1.pdf
PPDM Australian Data Management Conference 2004
Geoscience Australia
Lesson’s learnt: Impacts on Business
•
Business process
o
o
•
Workflow and versioning control are critical
Archiving dynamically changing data is an issue
As clients get very large data sets, statistically
valid data mining is a reality
o
o
o
Much more detailed metadata will be required for
machine to machine transfer (SensorML in particular)
Errors are more likely to be picked up
Need for, and administration of, organisation-wide
QA/QC will be a major impact
PPDM Australian Data Management Conference 2004
Geoscience Australia
Key Take Home Messages
•
Cross jurisdictional interoperability has been achieved in the
geoscience community (almost) by standards based interfaces
•
The final project release will be in March at the SEEGrid 2 workshop in
Canberra – please come
•
The greatest amount of work still needs to be done on the XML data
models (application schemas) used in any data system within and
between communities
•
The ability of technology and specifications to deal with data models
that realistically reflect the complexities of real world geoscience from
the technical perspective
•
Effective global governance for dividing the complexities of the real
world into manageable pieces for effective data modelling is required
and will help mitigate the above
•
Track our progress (or better still join us) on
http://www.seegrid.csiro.au
http://www.pmdcrc.com.au/
http://geoserver.sourceforge.net
PPDM Australian Data Management Conference 2004
Geoscience Australia