Transcript GIS_Overview_22042015_FINALx

GIS
Data Acquisition & Management; Spatial Analysis; Visualisation
Owen Macdonald, GIS Support Officer, GeoSciences, UoE
[email protected]
Outline
• Background/Basics
• The Everyday – What do you need? Desktop Package or Web Map?
• Visualisation – GIS Limitations/Novel Techniques
Background/Basics
Quick Intro to GIS
What is GIS?
• Geographical Information System…GI Science (Poss. rebrand: Spatial Data
Science)
• Conceptualising reality in a computer model
• Underpinned by spatial analysis; more than just maps or manipulation
• Why is GI important?
• Everything happens somewhere
• Ability to attach multitude of information to locations/geography units
• Relationships between phenomena/variables
• spatial coincidence
• relate/transfer info (attributes) between layers where these coincide
GIS Workflow
• Data acquisition/integration
• Data management/database management
• Data analysis/Spatial Analysis
• Cartography/Visualisation
GIS Functionality
• Data acquisition/integration
• (Scan and) digitise
• Tabular Data with Spatial Location (Point or Centroid); or can join to other located
data/geography units
• GPS/Survey/Laser Scanning/LAS/Point-Clouds
• Aerial/Satellite Image
• Data management/database management
•
•
•
•
•
Storage
Editing
Metadata (describe datasets meaningfully and in terms of quality/coverage)
Indexing (speed up data retrieval)
Map Projections – in here as apply to both analysis and visuals…
GIS Functionality
• Data analysis/Spatial Analysis – Map 'Layers' (Layer Stack); Spatial Co-incidence of layers
•
•
•
•
•
•
•
•
Querying; Select by Attributes; Select By Spatial Location;
Proximity Analysis/Buffering
Overlay/Intersect etc.
Interpolation of irregularly spaced data/sample points
Spatial Statistics – point patterns, hotspots/clustering, spatial autocorrelation
Scale/Aggregation/Zonation issues – MAUP, spatial autocorrelation, areal interp, dasymetric map
Spatial is 'a bit' special – but can apply various techniques above to any 2D/3D 'space'
Decision making
• Cartography/Visualisation
•
•
•
•
•
Display new computed variables/statistics
Mostly 2D though long-record of 3D/4D research;
Novel techniques to present complex info in 2D; ESDA; 'new' field of Visual Analytics
3D GIS emerging (traditionally '2.5D')
Decision making; Interpretation
Data Models and Types
• Essentially two common models of space
• Discrete Entity
• Continuous Field
• Discrete Entity (== Vector Types)
• Objects exist with empty space in-between
• Modelled in GIS by Vector data types/structures – points, lines, polygons
• Continuous Field (== Raster Types)
• Continuous grid of raster cells, each has a value for a single variable of interest
• Empty space explicitly represented by NoData (cf. NULL) cells, within grid extent
Data Types: Vector
• Discrete entities with specific locations; empty space between
• Multiple attributes for each feature
Point
Polyline
Polygon
Geographical Information Systems: Principles, Techniques, Management and Applications, 2nd
Edition, Abridged Introduction (P A Longley, M F Goodchild, D J Maguire, and D W Rhind). : 77.
Data Types: Raster
• Continuous surface with changing values; 'empty' NoData cells where
req’d
•
•
•
•
Elevation and derivatives
Satellite imagery
Photographs
Scanned maps
• Attributes
• Each cell has single value
• This can relate to Value Attributes – NB attributes of the value held by the
cell, not the cell itself nor its spatial location!
From ESRI Map Book Volume 24, www.esri.com
Other Model/Data Types
• Firstly, there are some other model frameworks/types:
• Triangular Irregular Networks TINs ('vector surface'!); variable res’n across surface
• Also ESRI (ArcGIS) specific – Terrains, Multipatches; 2D-3D interface
• However:
• Built from points (TINs), and lines, polygons, groups of points, large point-clouds
• Also: Other Data Formats
• Binary Formats – handle large data volumes, compact size
• Multidimensional data – NetCDF
Open GIS – Formats, Apps, Data
• Open Standards / Formats – Open Geospatial Consortium (OGC)
•
•
•
•
GML; CityGML(3D); KML (GE); GeoJSON (JavaScript - NB GeoJSON not maintained by OGC)
Moving Features
NetCDF
WMS; WMTS; WFS
• OpenSource Applications – Open Source GeoSpatial Foundation (OSGeo)
• QGIS/GRASS (use GRASS in QGIS; QGIS plugins; extend with Python)
• gvSIG/uDig (former extensible/mobile; latter Java/Eclipse based)
• RDBMS+Spatial Extensions – e.g. Postgres with PostGIS extensions (also pgRouting)
• OpenData
• OS OpenData (since 2010); Also extra OS/other data avail to HE via EDINA Digimap
• OpenStreetMap (linked to likes of OpenLayers/Leaflet etc.) – VGI (Volunteered GI)
• Also traditionally ‘open’ providers: Natural Earth, USGS
Changes in GIS
• Server GIS
• Dumb-terminal connects to a server (possibly in same building)
• Desktop GIS
• Typical standalone full GIS package on desktop PC (e.g. ArcGIS, Quantum GIS/QGIS)
• Web GIS / Distributed GIS
• Online processing services, Web Map/Feature Services, Remote Servers
• Client-Side Web GIS
• Thin or Thick Client, GIS in your web browser, JavaScript, ArcGIS Online
• Mobile GIS
• GIS on your tablet/phone/PDA – ArcPad, gvSIG
• Ubiquitous GIS
• Sensor Networks, GPS Trackers, Location Based Services, Geo-aware APIs
Everyday GIS
What do you actually need?
Everyday GIS on your PC –
Different Levels of 'GIS'
• Do you really need 'Desktop'? – Ignorance of ArcGIS no longer a crime!
•
•
•
•
•
•
•
Google Maps
Google Earth/ArcGlobe (KML/KML Timestamp)
(Spatial) Database – render on webpage in SVG
Need better SQL querying? – Store in Database: Oracle/Oracle Spatial, Postgres/PostGIS
Connect GIS to DBMS; Also connect R/stats package to GIS/Database
Do all in R?
Python?
• Can access ArcGIS/QGIS codebase directly without GUI performance issues
• Or, mapplotlib basemap, PySAL – ESDA
• Do you need more (specialist) than Desktop? – We’ll look at these later…
• ArcGIS Server/New ArcGIS Pro (integrated 2D3D environment)
• ESDA – GeoDa, CAST, PySAL – again perhaps R?
• Crime Analysis - CrimeStat
GIS in your Browser
• Google Maps
• Even simple click+create maps (but
now may require some degree of
login – MyMaps)
Or can run a web script, e.g.:
• Generates a KML doc and returns:
•
•
•
•
URL to KML
Google Maps URL to KML
User decides which 'viewer' to use
Allows for poor internet connections
GIS in your Browser
• Or can use JavaScript and GM API within a
web page using a map 'object'
• HTML+JavaScript
• Code to produce example image shown,
is included below
• Can add placemarks etc. to JavaScript
(and thus map)
• Mashup (verb.) whole datasets on top of
e.g. Google! Weather, Photos, Locations…
Map Mashups
• JavaScript-powered web pages can also
use HTML Forms to retrieve user input:
• Co-ordinates
• Origin/Destination Postcode
• URLs built using web map sites’ APIs e.g.
• www.streetmap.co.uk/newmap.srf?x=3210
00&y=621000&z&ar=n
• Hit Go! and returned are one, or more,
geo web pages:
• Maps
• Routing (e.g. Multimap, Google)
Map Mashups
• JavaScript-powered web pages can also
use HTML Forms to retrieve user input:
• Co-ordinates
• Origin/Destination Postcode
• URLs built using web map sites’ APIs e.g.
• www.streetmap.co.uk/newmap.srf?x=3210
00&y=621000&z&ar=n
• Hit Go! and returned are one, or more,
geo web pages:
• Maps
• Routing (e.g. Multimap, Google)
Map Mashups
• JavaScript-powered web pages can also
use HTML Forms to retrieve user input:
• Co-ordinates
• Origin/Destination Postcode
• URLs built using web map sites’ APIs e.g.
• www.streetmap.co.uk/newmap.srf?x=3210
00&y=621000&z&ar=n
• Hit Go! and returned are one, or more,
geo web pages:
• Maps
• Routing (e.g. Multimap, Google)
Open Alternatives
• Javascript Map Libraries
•
•
•
•
• MapServer
OpenLayers
Leaflet
MapBox JS (built-on Leaflet)
ArcGIS API for JS (…ArcGIS Online)
• GeoServer
• MapTiler – create hierarchical sets of map
images for rapid display/zoom
• MapBox
•
•
•
•
Hosting – cf. ArcGIS Online
Mapnik – Develop desktop/web apps
TileMill – apps (built-on Mapnik)
Arc2Earth – ArcGIS published to MapBox
• NB Arc2Earth puts Google Maps in ArcGIS!
• Quantum GIS Server
Cloud GIS; GIS Apps
• ArcGIS Online (Cloud GIS)
•
•
•
•
Login
Hosting
ESRI Desktop/Mobile Apps
ArcGIS Marketplace (incl. RS images
'through the cloud'!)
• ArcGIS Apps Including:
• Map and 3D Scene Viewers (Web)
• Collector (Android Jan, iOS Mar)
• Explorer (Android, iOS, Mac!!)
• Where does one begin?
• Other Apps/Marketplaces
• AutoCAD
• Maths/Diagramming Tools
• ESRI Business Apps
• Maps for Office
• Maps for SharePoint
• NB ArcGIS for Developers acc’t – free!!
Visualisation
GIS Limitations / Novel (or newly implemented) Techniques
Cartographic Principles
• Still apply…
•
•
•
•
•
Colour Theory
Map Purpose
Necessary (or unnecessary map marginalia – N arrow, scale bar or text? Etc.)
Colours or shading (changing hue or increasing intensity)
Clarity/avoid Map Clutter (not overloaded, cf. PowerPoint slides - ahem!)
• But also IT concepts to consider…
• Symbol type/style/size – e.g. halo effects to make stand out/legible
• Label placement/collision detection rules
• Image Resolution – end use? (web or print?)
Beyond 2D: 2.5D, Not quite 3D
• Historically used in computer
games
• Appearance of 3D world (e.g.
buildings, extruded shapes) but
movement constrained to XY plane
(2D)
• Use of camera angle/projection
and/or e.g. scaling of height to give
depth perception also parallax
scrolling and texture mapping and
other effects
Image: LinCity / Wikimedia Commons / GPL
Beyond 2D: 2.5D, Not quite 3D
• In GIS to date a 2.5D surface occupies 3D space but
still an XY plane at any point.
• Each XY location in a layer will have one unique zvalue (height)
• Heights obtained from a related DEM raster or
from values stored in either data geometry (zvalues) or attribute table field
• Extrusion and offset of layers possible
• Discontinuities not modelled (functional surfaces
only) – so no use for vertical geological faults (i.e.
dip = 0 degrees)
• May be possible in some open source GIS – e.g.
GRASS r.plane??
Contains OS data © Crown copyright and database right 2015
Beyond 2D: 2.5D, Not quite 3D
• We can however perform 3D analysis:
•
•
•
•
Viewshed/visibility analysis
Visual Impact/Skylines
Building shadow, amount of sunlight
Amount of internal space overlooked
• ESRI also offer multipatches (outline of 3D
shapes and texture mapping) for some 3D
work – close, but?…
Contains OS data © Crown copyright and database right 2015
• Watch this space…
Globe Viewers – 3D?
• ESRI ArcGlobe
• Reprojects all layers to one CRS
• ArcReader (Free basic ‘reader’)
• Explorer for ArcGIS (…Online, basic!)
• ArcGIS Explorer Desktop (Free!)
• Lets you switch quickly 2D<->3D!
Images: ESRI
Globe Viewers – 3D?
• Google Earth
• Subsurface?
• Surface discontinuities?
Image via Google Earth plugin demos
• Thematicmapping.org – B. Sandvik
• KML produced, symbolising datasets
• 1D (bar length), 2D, or 3D symbols
Global Thematic Mapping
• Can use GE or GE plugin
• GE plugin may face extinction
(beware ‘Google pulling the rug’)
• Which might mean the end of
many sites built-on GE
https://youtu.be/t0L9coOwkE8
•
•
•
•
Flight trackers – “Follow in GE”
Ship trackers – “View in GE”
E.g. Google Earth Flight Simulator!
http://www.gefs-online.com/
Map projections / Co-ord Ref Systems (CRS)
• Geographic Systems – curved Earth, angular units vs. Projected Systems – flat Earth, linear units
• Important to understand basics – though not always critical to define/set CRS for all layers if working in one coord system and correct units/extent values seen – be able to spot if wrong!
• However, if you have data based on multiple Earth surface models then you must ensure you specify the correct
datum. GIS offers on-fly transformation between datums but layers in angular units will need projected to new
data layer to match up exactly.
• Get the datum right – what the data are in (e.g. GPS in WGS84 geo), then project as req’d e.g. BNG
• No unique lat, long for every point on Earth – depends on datum/geographic CRS used
• Also becomes more critical when working with e.g. Globe Viewers where all layers are projected to e.g. Cubic
projection! With ArcGlobe all layers must have a CRS defined.
• Also bear in mind peculiarities of different projections.
• Distortion – of familiar shape? area/proportion of map occupied? angular relationships?
• Web Mercator / Web Maps incl. Google – Distortion far from equator, Scale change with latitude!
Interactive Cartography – ESDA
• ESDA – Exploratory Spatial Data Analysis
• GeoDa/CAST/PySAL
• Brushing & Linking
• Explore Feature/Data/Stats Space &
• Explore Real Space (simultaneously)
•
•
•
•
2D Cartogram
3D Box Plot
Parallel Coordinate Plot
Others…
Interactive Cartography – ESDA
• ESDA – Exploratory Spatial Data Analysis
• GeoDa/CAST/PySAL
• Brushing & Linking
• Explore Feature/Data/Stats Space &
• Explore Real Space (simultaneously)
•
•
•
•
2D Cartogram
3D Box Plot
Parallel Coordinate Plot
Others…
Interactive Cartography – ESDA
• ESDA – Exploratory Spatial Data Analysis
• GeoDa/CAST/PySAL
• Brushing & Linking
• Explore Feature/Data/Stats Space &
• Explore Real Space (simultaneously)
•
•
•
•
2D Cartogram
3D Box Plot
Parallel Coordinate Plot
Others…
Interactive Cartography – ESDA
• ESDA – Exploratory Spatial Data Analysis
• GeoDa/CAST/PySAL
• Brushing & Linking
• Explore Feature/Data/Stats Space &
• Explore Real Space (simultaneously)
•
•
•
•
2D Cartogram
3D Box Plot
Parallel Coordinate Plot
Others…
Interactive Cartography – VA
• (GeoSpatial) Visual Analytics
• GeoVis -> GeoVA?
• May well include some existing types of analysis…
• ESRI adopted 'analytics' in place of 'analysis‘!
• Big in London in partic – CASA, giCentre at City
• Plus MapTube, http://spatial.ly/, London data/vis e.g. TFL, Crime
• Use of linked views cf. GeoDa/ESDA
• 'Processing' (noun) prog. language for visuals, often used
Visualising Movement/Flows
• Big Data(sets)
• Twitter Feeds
• APIs/Screenscraping
• Smart Cities
• Crowd-sourcing/VGI
https://www.youtube.com/watch?v=rJC7B-9ZfhE
A Few 100(0) Other Things to Explore…
•
•
•
•
•
•
•
•
•
•
•
Tobler’s 1st Law
Topology
Network / Metro maps…
Database Theory / Best Practice
Modern Databases (NoSQL!)
Spatial Data Infrastructures (SDI)
Spatial Regression/Error Modelling
Metadata (Standards)
Quirks of ESRI File Formats!
ArcGIS Quirks…
Evolution of GIS – why it is the way it is…
•
•
•
•
Human Computer Interaction
User Needs Testing; Accessibilty Issues
User Perception/Psychology
Deeper Information Theory
• User/Software Behaviour
• Display of Info (Bertin)
• Visualisation Luminaries
• Tufte, MacEachren, modern-days
• Automated Carto / Map Generalisation
• Heat Mapping
• Space Time Cubes / Temporal GIS
…But never* be afraid to ask!
Question Time?
*See: Advice to the Aspiring Interactive Cartographer...
http://www.axismaps.com/blog/2012/09/advice-to-the-aspiring-interactive-cartographer/