Transcript Data Formats

An Overview of Geospatial Standards
Support within the ArcGIS Platform
Satish Sankaran
Marten Hogeweg
Agenda
•
Interoperability Challenges
•
Standards Organizations and Standards
•
Standards – Data formats and API, Metadata and Web Services
•
ArcGIS – Standards Support
•
Pragmatic Interoperability (de-facto vs de-jure standards)
•
Demonstrations
Interoperability Challenges
Web
Mobile
•
•
•
•
•
•
Create
Manage
Visualize
Analyze
Collaborate
Discover
Cloud
Enterprise
Local
Desktop
-
Multiple Platforms ( Operating Systems, Databases, …)
-
Multiple Architecture ( Local, Enterprise, Web, Cloud,…)
-
Multiple Clients ( Desktop, Web, Mobile, …)
-
Multiple Developer Environments (.NET, Java,…)
-
Multiple Protocols (SOAP, REST, OGC,….)
-
Multiple Encodings ( XML, Raster Formats, GML, JSON, ….)
The ArcGIS Platform
-
An Open & Interoperable Platform
-
-
Supporting a thriving ecosystem of
focused apps and applications.
A Standards Compliant Platform
-
Supporting many relevant standards including
those from ISO TC 211 and OGC.
http://www.esri.com/software/open
Developer
Procurement personnel
System Integrator
User
Standards
Open
ETL
Interoperability
Many Standards Organizations …….
ISO
ANSI/INCITS
IHO
OMA
OGC
WS-I
ACSM
CEN
FGDC
OASIS
GSDI
DGIWG
ASPRS
EPSG
Many Standards ……
WFS
WMS
ISO 19139
SOS
WPS
WCS
ISO 19115
GeoXACML
FGDC - CSDGM
Simple Features
Open LS
GML
OWS Context
CityGML
Geopackage
GIS Standardization – functional areas
Basic Geographic Structures (semantic interoperability)
Geometry, topology, coordinate reference systems
Content Description (semantic interoperability)
Standardized Data Models, Feature Catalog
Modeling languages, Application schemas
Metadata
Data Management (semantic and technical)
Database management
Data Formats (technical interoperability)
Structure, Syntax, language, API
Visualization (semantic and technical)
Symbology, cartography, display
GeoWeb Services (technical interoperability)
Web Catalog, Web Services, SOA
GIS Standardization – functional areas
Basic Geographic Structures
ISO 19107 – Spatial Schema, ISO 19111 – Coordinate Reference Systems
Content Description
FACC, SDSFIE, Standardized Data Models
ISO 19110 – Feature Catalog Methodology , UML, FGDC Metadata, ISO 19115 – Metadata
Data Management
OGC Simple Features
Data Formats
VPF, OGC GML, ISO 19139, 80+ industry and national formats
Visualization
SLD, MIL-STD 2525
Web Services
OGC WMS, WFS, WCS, OpenLS
Registry and catalog services
CAT 1.0, OAI-PMH, CS-W (ebRIM)
Standards
Data Formats
Metadata
Web Services
Data Formats
•
Simple Features
•
CAD formats
•
GML
•
IFC
•
KML
•
NIEM
•
Geopackage
•
…..
•
Gml JPEG 2000
•
OWS Context
•
WaterML, AiXM, …
What are simple features?
•
Simple Feature specification defines:
- Data access model
- Database schema
- 2D Geometry model for points, lines, polygons
- “Well-Known” data formats for geometry and spatial reference
•
ISO and OGC specifications for simple feature access
Feature Tables contain rows (features) sharing common properties (Feature Attributes).
Geometry is a Feature Attribute.
10
area1
yellow
11
area2
green
12
area3
Blue
13
area4
red
Geometry
Feature Attribute
Feature Table
Feature
Geography Markup Language (GML)
•
GML or Geography Markup Language is an XML based encoding standard for geographic information
developed by the Open Geospatial Consortium (OGC).
•
Defines an XML schema for representing geographic features (attributes, geometries, relationships,
Topologies, Observations, etc).
•
GML Simple Features
-
Initially motivated to help WFS use of GML 3
-
GML 3 has many optional elements of schema, which complicates GML parsers
-
GMLSF Profile “fixes” application schemas to only allow the elements prescribed in the profile
-
This allows WFS servers and clients to be simpler and more efficient; the types of data which they need
to anticipate are finite and reasonably structured for faster parsing
GML Support in ArcGIS
•
GML Support in ArcGIS Desktop
- Simple Feature GML support is available to all ArcGIS users
Import/export data
- Use directly as a map layer
- Use in a geoprocessing model
- Consume GML served by any Web feature Service (WFS)
-
•
GML Support in ArcGIS Desktop (ArcMap) using the Data Interoperability Extension
-
Support for specific Application schemas available using the Data Interoperability Extension.
Several application schemas supported
- Custom schemas defined using transformation tools
- Consume GML served by any Web feature Service (WFS)
-
•
GML Support in ArcGIS Server – Web Feature Service (WFS) implementation
- GML is streamed using the Web Feature Service Protocol
Kml Support in ArcGIS
KML is an XML grammar used to visualize geographic data in an earth browser.
KML uses a tag-based structure with nested elements and attributes and is based on the XML standard.
• Serve
– Using ArcGIS Server
• Consume
– ArcGIS Desktop (ArcMap,
PRO)
– Runtime
– ArcGIS Earth
– ArcGIS Online / Portal
• Geoprocessing Tools
– Layer To KML Tool
• The Layer To KML geoprocessing tool allows individual
• layers to be exported directly from ArcMap, ArcGlobe or ArcScene.
– Map To KML Tool
• The Map To KML geoprocessing tool allows multiple layers to be
• exported into a single KML source.
ArcGIS Data Interoperability Extension
• Format Support
– Adds support for 200+ data formats
– Custom Formats
– Direct use in ArcGIS Desktop
• Quick Data Translation
– Data Interoperability Tools Toolbox
– Convert between data formats
• Spatial ETL (Extract, Transform, Load)
– Semantic Data Translation
– Data Restructuring
Standards
Data Formats
Metadata
Web Services
Metadata
Leading the way to semantic interoperability
•
Data about data
•
-
Geographic area covered
-
Producers explain their product
-
Users learn about products
Currency
Rules of use
•
-
A Key interoperability technology:
Non standard products can be understood
Positional accuracy
International Metadata Standards
-
Data dictionary
• FGDC CSDGM
-
Means of encoding
• ISO 19115, 19119 Metadata
-
Datum
-
Map projection
• ISO 19139 Metadata XML Schema
Implementation
• …
ISO Metadata ( Today  Future )
Ted Haberman : hdfcgroup.org
Metadata standards today
Extensions
Content Standard for
Digital Geospatial Metadata
Aka “FGDC Metadata”
Profiles
ISO 19115 Geospatial
Information - Metadata
Aka “ISO Metadata”
<xsd:element name="bounding" type="boundingType"/>
<xsd:complexType name="boundingType">
<xsd:sequence>
<xsd:element ref="westbc"/>
<xsd:element ref="eastbc"/>
<xsd:element ref="northbc"/>
<xsd:element ref="southbc"/>
</xsd:sequence>
</xsd:complexType>
Overview of Metadata Standards
ISO 19115 (-1) Profiles
•
Australia-New Zealand Profile
•
North American Profile of ISO
19115:2003 (NAP)
•
•
•
•
•
•
•
•
•
CAN/CGSB-171.100-2009
•
INCITS 453-2009
INSPIRE Metadata Directive
ISO19115/ISO19119 Application Profile
for CSW 2.0
C3Grid ISO 19115 Metadata Profile
Marine Community Metadata Profile of
ISO 19115
Energy Industry Profile of ISO 19115-1
USGIN ISO 19139 Profile
DISDI Geospatial Metadata Profile
(DGMP)
National System for Geospatial
Intelligence Metadata Foundation (NMF)
•
•
•
•
•
•
•
•
•
•
•
•
•
A metadata profile for precision agriculture
based on ISO 19115 standard.
WMO Core Profile of the ISO 19115
Metadata Standard
Transformation of HDF-EOS metadata from
the ECS model to ISO 19115-based XML
Draft African metadata profile of ISO19115
Samoan profile
NOKIS – an ISO 19115 Based Metadata
System
The UDK and ISO 19115 Standard
Latin American Profile
Núcleo Español de Metadatos (NEM)
UK GEMINI
Biological profile
Coastal zone profile
S100 Metadata profile (IHO)
ArcGIS Metadata Standards Support
ArcGIS
Metadata
ISO 19139
(NAP) XML
ISO 19139
(INSPIRE) XML
ISO 19139 XML
FGDC CSDGM
XML
ArcCatalog
Metadata
ArcGIS Is a Platform
Enabling GIS Everywhere
Desktop
Web
Device
Author / Create
Use
Simple
Integrated
Open
Catalog / Discover
Portal
/ search
Serve / Publish
Server
Online Content
and Services
Available in the Cloud . . .
. . . and On-Premises
ArcGIS Platform and Metadata
Items Carrying Metadata
Indexing / Access / Discovery
•
Portal and Online Items
•
GeoServices REST Spec
•
Mxd’s, Datasets, Services
•
OpenSearch
•
Folders
•
OGC CS-W
•
…
Metadata Formats
Products
•
Item Information
ArcGIS Desktop – ArcMap, PRO
•
ArcGIS Metadata
ArcGIS for Server
•
Formal Metadata – FGDC, ISO, …
ArcGIS Online / Portal for ArcGIS
•
DCAT
•
Encodings – esri JSON, XML, atom,
ArcGIS OpenData App
Geoportal Server
App Widgets
Esri support for metadata
ArcMap
Geoportal
Item Info
ArcGIS Online
Mxd properties
ArcGIS PRO
ArcGIS service
meatadata
Service Metadata
ArcGIS OpenData
WMS metadata
What’s Next for ArcGIS?
•
•
Investigating how to incorporate support for ISO 19115-1/19115-3 into the ArcGIS
platform
-
19115-3 incorporates existing ISO 19115-2 and 19110 XML schemas
-
Evaluating XML schemas available in new ISO repositories
Continue to Improve metadata support in ArcGIS Pro and throughout the platform
-
Desktop -> Server -> Online/Portal
-
Planning next steps
Standards
Data Formats
Metadata
Web Services
OGC Web Services
•
Web Map Service
•
OGC KML
•
Web Map Tiling Specification
•
Sensor observation Service
•
Web Feature Service
•
Web Processing Service
•
Web Coverage Service
•
3D Portrayal service
•
….
DGIWG and OGC
Service Types and
Profiles
Geospaial Interoperability
Reference Architecture doc
Report – Spring 2015
https://www.ise.gov
/sites/default/files/GIRA.pdf
http://ise.gov
from … Geospatial Interoperability Reference Architecture (GIRA)
ArcGIS Platform Components
Portals
(AGOL, Portal)
Providing &
Consuming
Server
Providing
Developer Solutions
(runtime, Javascript
Api)
Consuming
Core Standards
Metadata
(ISO, FGDC, …)
Data
(Geopackage,
geoJSON, gmlJPEG 2000…)
Desktop
(ArcMap & PRO)
Consuming
Portrayal
Standards
Apps
Consuming
Services
(wms,wfs,wcs,
wps,sos,cs-w, …)
JSON clients
OGC clients
Inspire clients
(HTTP REST)
(HTTP SOAP)
(HTTP SOAP)
Geoportal
ArcGIS for Inspire
Inspire CSW
Inspire
ATOM
Inspire WFS
Inspire WMS
WPS
WCS
WFS
WMTS
WMS
KML
GeoJSON
Esri REST
API
ArcGIS for Server
http://www.esri.com/library/whitepapers/pdfs/supported-ogc-iso-standards.pdf
Standards Requirements – Across the platform ….. What we are hearing!
Online / Portal
- Register OGC services
- Consume OGC services
- Create hosted OGC services
- Geopackage support
- Allow analyses on OGC services
- Support Discovery Standards – OGC CSW, OpenSearch, DCAT
- Managing Secured OGC Services
•
Server
- Improve existing OGC support (
compliancy, update to new version, new
profiles)
•
Desktop (ArcMap / Pro)
- Consume OGC services
- Geopackage support
•
Metadata Support across the platform
•
•
Federated searches across portals
Runtime and Apps
- Consume OGC services
- Geopackage support
•
Javascript API
- Consume OGC services
•
Standards Support
•
•
ArcGIS Online
•
ArcGIS PRO
-
Support for formal Metadata (Version 3.8,
July 2015)
-
Improvements to wms wmts and metadata
support in PRO (PRO 1.2)
-
Support for secured OGC services
(Version 3.10, Nov 2015)
-
Improvements to KML support ( Pro 1.3)
-
Major improvements in Online (version 4.2,
June 2016 ) – wms, wfs
ArcGIS for Server
- Improvements to wfs 2.0 support
- Achieved KML Compliancy (Server 10.4)
•
ArcGIS Earth 1.1
-
Supports kml, wms and wmts
Standards Support - Roadmap
Support for Hosted wms, wmts and wfs support
in Online (Q4 2016) and Portal ( Version 10.5)
WCS support in Javascript API (prototype)
Native wfs client support in PRO (Version 1.4)
Web App Builder widget to consume OGC
OWS context document with embedded ogc
services (prototype)
WCS 2.0.1 support in ArcMap and PRO
Support for wms, wmts ( Q4 2016 Quartz
runtime)
Support for geopackage, wfs , s57 in (Q1 2017
Quartz runtime)
Native netCDF Support in ArcMap (10.5) and
PRO (10.4)
Better metadata support across the stack
Improvements to geopackage support
Procurement personnel
http://www.opengeospatial.org/resource/products/compliant#ESRI
@ 10.4 : 16 certificates
Pragmatic Interoperability
SOA Vs REST
Use the
Web Luke !
http://www.programmableweb.com/news/api-anti-patterns-how-to-avoid-common-restmistakes/2010/08/13
http://geekandpoke.typepad.com/.a/
6a00d8341d3df553ef012875f312f9970c-pi
…. Some GIS Standards are still playing catchup with the Web
Pragmatic Interoperability
Standards, Specifications, ….
•
geoJSON
•
geoRSS
•
•
•
•
DCAT /
geoDCAT -AP
•
CSS – Styling
•
RDF
•
No SQL
•
xlsform
Atom
Sqlite
Vector tiles
•
Opensearch +
geoExtension
Pragmatic Interoperability
Open API’s and Specifications
•
Shapefile (July 1998)
• i3S
•
Geodatabase XML (June
2008)
• LERC
(May 2015)
(Dec 2015)
•
GeoServices REST API (Sept
2010)
Open Data Download
Stats for April 2016
•
FGDB API (Jan 2011)
Shapefile: 3,714,978
CSV: 413,037
GeoJSON: 318,052
KML: 43,689
GeoHash: 22,033
The Proof of the Pudding…
@martenhogeweg
Group on Earth Observations
SAEOS
INPE
PANGAEA
GCMD
ArcGIS Online
GEO
SeaDataNet
EEA-SDI
EuroGEOSS
WMO WIS
IOOS
EnviroGRIDS
GSFC-DAAC
GBIF
Connecting GIS and Earth Observations
Geospatial Platform and Data.gov
…
NASA
GSA
FCC
Census
Geoplatform.gov
Data.gov
DOI
DOT
NOAA
DOJ
EPA
DHHS
DHS
Access Geoplatform Content in ArcGIS
•
Connect to
http://geoplatform.maps.arcgis.com
•
Find National Geospatial Data Assets
•
Use in Your GIS Projects
OGC Testbed 12
•
•
•
Viewer
-
ArcGIS, WMS, WCS, KML
-
WMTS, Koop FeatureServer
Discovery
-
OpenSearch
-
OGC CSW 2.0.2, 3.0.0
OGC Context Document
-
GeoJSON encoding
Please take our Survey
Your feedback allows us to help maintain high standards and to help presenters
Find your event in the
Esri Events App
Find the session
you want to review
Scroll down to the
bottom of the session
Answer survey
questions and submit