Transcript solvinggiswarehousedatainteroperabilitychallenges-final

Solving GIS Warehouse Data
Interoperability Challenges
Brian Pont
Software Support Specialist – Safe Software
Validation
Integration
Accessibility
Infrastructure
Transformation
Common Data Challenges
Case Studies
• Puget Sound Energy
• Indiana Department of Homeland Security
• Arkansas GIS Office
Data Loading at Puget Sound Energy
• Gamze Cetken
• Sr. GIS Technical Systems Analyst
Challenges
• County data not normalized
• Not just different fields but different data capturing
techniques as well
A Deeper Look
The Plan
Example 1 - Kitsap County
• D ata : Parcel Shapefile and Address Point Shapefile
The Issue
 One or more address
points per parcel
9
Solution
• Use an ETL Tool to Manipulate the Data
• Overlay address data onto parcels
• Check for duplicates
• Remove offending geometry
Outcome
EXAMPLE 2 – Kittitas County
• Data : Parcel
Shapefile and Assessor Data in MS Access
The Issue
Issue #1:
• House Number is not
stored in a separate
field.
• In GIS House Number
is a mandatory field
02602 N Columbia St Ellensburg
Issue #2:
• In some addresses
house numbers do not
exist
Evergreen Way Easton
ETL Solution
Parsed Addresses
Valid House Numbers
17
Outcome
Clean
Data
Schema
Mapping
Data
Solution
The Long & Winding Road
State uses FME to collect data from counties
in the State of Indiana.
Indiana DHS
Further information at FME Blog and Esri ArcNews
Online
The Challenges
• Acceptance
• Integration
Keys to Success
Non-Invasive
Automated
Accessible
GIS Warehouse
Outcome
Easy
Access
Common
Standard
Data
Solution
Geostor:
State GIS Clearinghouse Cloud Migration
ARKANSAS, USA
Seth LeMaster
Tony Davis
Arkansas GIS Office
Overview
Arkansas GIS Clearinghouse
• GIS open data portal for the state of Arkansas
• 1222 monthly downloads (2,358 items downloaded)
• Users
• 108 registered
• 2,387 non-registered
Overview
The Challenge - GeoStor
Improve the stability and migrate to the cloud.
• 300 vector datasets migrated to PostGIS RDS
• 3TB of raster data on AWS S3
• 4TB of historical raster data to AWS Glacier
Architecture
Outcome
• Stability – Fault tolerant data storage and support for the FME
Cloud architecture.
• Security – Leverage AWS compliance and FME Cloud security
policies.
• Simplicity – Focus on product not the administration
• Price – 3 times cheaper than on-premises
Costs: On Premises vs Cloud
On Premises
DIS Server Space
FME Dekstop
FME Server
Dell Hardware
SQL Server
Windows Server (Software
Assurance)
Windows Server Enterprise
(Software Assurance)
SQL Server (Software Assurance)
SQL Server (Licenses)
MS Server Std Edition (License)
MS Windows Server (Licenses)
Symantec
Tape Backups (No server)
Total Recurring Montly Payment
Total Recurring Yearly Payment
Total One Time Cost (Every 3 yrs)
Total Three Year Cost
True Monthly Cost (/36 months)
Intangiable Costs
Hardware Maintenance Time
DIS Process
Non-Scaleable
Monthly
Yearly
Up-Front
$3,200
Monthly
FME Dekstop
$6,000
$12,000
$100,000
$595
$385
$270
$294
$1,662
$4,490
$680
EC2 Instance (m3.large)
$14,400
$395.81
AWS Storage (EBS 780GB)
$77.11
AWS Storage (S3 2TB)
$61.30
AWS Storage (Glacier 4.2 TB)
$42.41
Total Recurring Montly Payment
$576.63
Total Recurring Yearly Payment
$20,400.00
True Monthly Cost (/36 months)
Yearly Up-Front
$6,000
FME Cloud
Total Three Year Cost
$2,000
$3,200
$26,376
$102,000
$296,328
$8,231.33
Cloud
$81,958.68
$2,276.63
Our rack space costs (real estate on our data center
floor) $3,800 per month. Add to that the hardware
costs, etc and you start to see why moving to the cloud
was a no brainer for us.
Anthony Davis, State Arkansas
Outcome
Offload
Infrastructure
Improved
Efficiency
Data Solution
Summary
Keys
Make it
Available
Make it
the Same
Solutions
Make it
Easy
Make it
Correct
Thank You!
• Questions?
• For more information:
• Brian Pont [email protected]
• Safe Software Inc.
• www.safe.com