Transcript Module I Lecture (UP 418)x

Introduction to Big Data and
CyberGIS for urban planning
UP 418
Outline
•
•
•
•
Big Data for cities
Cyberinfrastructure and CyberGIS
CyberGIS for planning
Urban Big Data and CyberGIS
Big Data for Cities
The three Vs of big data
Sagiroglu & Sinanc 2013
Big Data
Big Data attributes
• Volume:
– Correlation, Optimization
– Mapping current check-ins
– Kriging crowd-sourced temperature data
• Velocity
– Real-time monitoring of moving objects
– Real-time map of all smart phones
– Real-time map of tweets related to disasters
• Variety
– Fusion of multiple data sources
– Map of post-disaster situation on the ground
Spatial Big Data examples
•
•
•
•
Point data: Check-ins
Line data: GPS-tracks from smart phones
Raster data: UAV/WAMI Video
Graph: Temporally detailed roadmaps, Waze,
Open Street Map
Traditional Spatial Data vs. Spatial Big Data
Big Data vs. Spatial Big Data
Big Data vs. Spatial Big Data
• Big data questions:
– What are (previously unknown) side-effects of FDAapproved medicines?
• Spatial Big Data questions:
– What are hotspots of spring-break vacationers today?
– What are critical places with many smart phone users
in the last hour?
– Are there any hotspots of recent disaster-related
tweets?
– Are there traffic congestion areas reported by Waze?
Applications of Spatial Big Data
• Climate Change: availability of tremendous
amounts of climate and ecosystem data
• Next-Generation Routing Services: GPS trace
data, engine measurements, and temporallydetailed roadmaps
• Emergency and Disaster Response: leveraging
geo-social media and Volunteered Geographic
Information (VGI)
Big raster datasets:
• Global Climate Models (GCM) data
• Unmanned aerial vehicle (UAV) Data
• LiDAR data
Big Vector Data
• Volunteered geographic information (VGI)
data
• GPS Trace Data
• Spatio-Temporal Engine Measurement Data
• Historical Speed Profiles
Emergence of ‘Smart cities’
Cyberinfrastructure and CyberGIS
What is Cyberinfrastructure?
• “ a coordinated and flexibly-configured
collection of heterogeneous networked
devices (e.g. high performance computers,
sensors, instruments and data repositories),
software and human resources that are
needed to address computational and data
intensive problems in science, engineering
and commerce.” (Armstrong et al. 2011)
Challenges of Cyberinfrastructure
• information integration from multiple sources
• implement seamless interoperation among
heterogeneous datasets
• develop a standard interface to data and
analytical tools
• provide effective provenance to trace the
origin of data and its movement
Cyberinfrastructure Applications
• Social networks: having important effects on
personal interactions, information search and
the diffusion of ideas
• Cyberinfrastructure in geospatial information
collection: individuals acting as volunteer
geospatial data collection agents (Goodchild
2007).
Cyberinfrastructure Applications
• Information delivery services: mobile devices
equipped with GPS able to support the
provision of context-dependent information
services
• Data analysis and visualization: visualization
of data-intensive, large-scale and multi-scale
geospatial problems
Why cyberGIS
• Limited ability of conventional GIS software to
handle very large spatial data and manage
sophisticated spatial analysis/modeling (SAM)
• CyberGIS is a seamless integration of CI, GIS,
and SAM capabilities
CyberGIS software integration
Major aspects of cyberGIS
Four major aspects:
• providing access to data,
• integrating disparate data sources,
• service chaining and
• provenance.
Types of data services
Types of data services for cyberGIS:
• Web Map Services (WMS): returns georeferenced static images
• Web Feature Services (WFS): for exchanging
actual feature data
• Web Processing Service (WPS): facilitate
sharing, discovery and dynamic binding of
geospatial processes
Provenance
Provenance feature enabled by cyberGIS
provides:
• a common understanding about the content
of data and how the data can be used;
• an assessment of datasets to aid in
determining if they meet requirements (of
data quality, accuracy, timeliness, etc.) of
specific application needs; and
• replicability of datasets.
Societal issues of CI and CyberGIS
• Access : Differential use of geospatial
information through CI creates imbalances
among social and economic groups.
• Privacy: Access to high-resolution geospatial
information can also enable individuals to
compromise certain aspects of privacy.
Societal issues of CI and CyberGIS
• Quality: data created by individuals with
unknown skill levels, quality can become
suspect.
• Aggression: intentional errors could be
introduced into geospatial information
Societal issues of CI and CyberGIS
• Piracy: individuals would be motivated to
appropriate and repackage rich data for
commercial or other uses
• Educational shallowness: online access to
geospatial information may limit the pursuit
of richer, more difficult to obtain or use,
resources
CyberGIS benefits
• information now flows bi-directionally.
• offer opportunity for more meaningful
participation than traditional forms of public
participation.
• provide users with increased control over
content and presentation.
CyberGIS for planning
Why planning needs CyberGIS
Increasing complexity in urban planning and
policy-making due to:
• Rapid urbanization process
• asymmetric development in spatial planning
• rise of big urban data
Why planning needs CyberGIS (cont.)
Technical challenges for effective decisionmaking:
• Different types of geospatial resources from
multiple agencies
• Poorly documented data
• Data in local standards resulting in a high
degree of heterogeneity
• Duplicate efforts due to lack of collaboration
Why planning needs CyberGIS (cont..)
• Visualization of big data acquired from different
fields such as telecommunication, public
transportation, social media and crowd
simulation.
• Large amount of ever increasing geo-tagged data
• Find meaningful urban phenomena, reveal spatial
patterns in urban areas, explore interaction
between human beings and environment
CyberGIS as solution to Planning
decision making
• movement away from the stand-alone desktop
paradigm
• virtual web service-based framework
• computation is carried out in the cloud
• improves interoperability of distributed geospatial
resources
• promotes widespread sharing of geospatial data
and analytical functionalities
• empowers data-driven scientific analysis
CyberGIS for transportation and
mobility in cities
• Integration of CyberGIS with Location-Based
Services and Intelligent Transportation Systems
(ITS)
• Capture real-time traffic information by utilizing
anonymous floating car (or any mobile positioning
device) data to update road status for adaptive
routing optimization
• Historical trajectories of vehicles can be used for
spatio-temporal data mining to find interesting
knowledge or statistical patterns to guide practical
driving
CyberGIS-supported urban risk
management
• Use of social media for hazard response and
information sharing
• Utilizing ‘neo-cartographers’ for better
collaboration of professional and participatory
communities
• Multi-hazard risk assessment by stakeholders
based on a navigational interface
CyberGIS-assisted planning decisions
• advanced GIS and interactive media provide
practical and intuitive tools for neogeographers.
• 3D visualization and interactive platform for
public participation
• Noise mapping
• Solar energy
Urban Big Data and CyberGIS
Urban Big Data
• Recent “data-deluge”: generation of large
real-time data sets at fine spatial scales and
over very short time periods
• In the past, we have never had real time data
and most data has not been people-centric
• UK examples: London Datastore and
‘data.gov.uk’
Geodemographics
• Geodemographics are small-area summary
measures of neighborhood conditions.
• Recent ‘big data’ sources for
geodemographics: consumer surveys, smart
travel cards, store loyalty program data, etc.
• New sources of open data (education,
transport, health domains, and social media)
are useful alongside conventional travel-towork statistics
Applications of new data source:
Twitter
• a significant new frontier for data miners and
sociologists alike
• Available in large volume, on a real-time basis and
accessible with considerable ease
• opportunity to explore geographical phenomena
across space and time, without the same sorts of
time delay inherent in many survey methods
• However, the data are only a small and selfselecting subset of the population, Twitter user
unconcerned about locational privacy
The spatial distribution of 6.34 million geo-located tweets across London
from July and August 2012
Geodemographic Uncertainty
Utilization Uncertainty
• Sources of bias
– variation in the intensity of usage
– nature of events and activities that prompt an
individual to tweet
– spatiotemporal variation in Twitter usage
Semantic and Syntactic
Uncertainty
• determining semantic meaning is challenging
without sufficient cultural or personal context
• Language detection process confirms such
uncertainty
Spatial Uncertainty
• uncertainty with respect to the accuracy and
precision of the measurement
• Twitter dataset does not provide any
indication of spatial accuracy, nor information
on the device
• Twitter clients report users’ locations at
different levels of spatial precision
Urban Transport in Real Time
• Increased use of smart cards generate detail data
on travel pattern
• It can be added with timetable information (to
help calculate delays), passenger flow
information, and a wealth of socio-economic
datasets
• Creates billions of rows of data requiring gigabytes
of storage space
• CyberGIS is the potential solution to work with
this big data
A snapshot of the number of passengers passing through nodes in the Tube
network during a typical rush hour
Monitoring transport Infrastructure
• Real-time data services can be used
• Most of the Bus and Train services are
occupied with GPS tracking device.
• APIs can be used for real-time data feeding
and system monitoring
• Any major disruption can be quickly identified
and alternatives measures can be taken
The impact of a bus-driver strike in June 2012. “A” shows the locations of buses at 9am on a
normal day whilst “B” shows the locations of buses at 9am on the strike day. It is clear that east
London was far more affected by the action.
Planning challenges for CyberGIS
• CyberGIS has important implications for the science of
cities and the depth of insights it can provide
• Data that pertains to real time geocoded to the finest
space-time resolution is becoming the new norm and
CyberGIS captures the on-going process of adaption
required to handle such changes
• GIscience is beginning to respond to the real-time
streaming of big data but it needs a new kind of big
science and new infrastructure to really grapple with
the analytics required to make sense of such data.
Readings
•
•
•
•
•
•
•
Armstrong, M. P., Nyerges, T. L., Wang, S., & Wright, D. (2011). Connecting geospatial
information to society through cyberinfrastructure. The SAGE Handbook of GIS and
Society. London, Sage Publications, 109-22.
Cheshire, J., Batty, M., Reades, J., Longley, P., Manley, E. and Milton, R. (2013). CyberGIS
for Analyzing Urban Data, in CyberGIS: Fostering a New Wave of Discovery and
Innovation. Wang, S. and Goodchild, M. (eds) Springer-Verlag (in press)
http://eprints.ncrm.ac.uk/3159/
Evans, M. R., Oliver, D., Yang, K., & Shekhar, S. (2013). Enabling Spatial Big Data
via CyberGIS: Challenges and Opportunities. CyberGIS: Fostering a New Wave of
Geospatial Innovation and Discovery. Springer Book.
Li, W., Li, L., Goodchild, M. F., & Anselin, L. (2013). A geospatial cyberinfrastructure for
urban economic analysis and spatial decision-making. ISPRS International Journal of GeoInformation, 2(2), 413-431.
Sagiroglu, S., & Sinanc, D. (2013, May). Big data: A review. In International Conference
on Collaboration Technologies and Systems 2013 (CTS), (pp. 42-47). IEEE.
Tao, W. (2013) Interdisciplinary urban GIS for smart cities: advancements and
opportunities, Geospatial Information Science, 16:1, 25-34, DOI:
10.1080/10095020.2013.774108
Wang, S., Anselin, L., Bhaduri, B., Crosby, C., Goodchild, M. F., Liu, Y., & Nyerges, T. L.
(2013). CyberGIS software: a synthetic review and integration roadmap. International
Journal of Geographical Information Science, 27(11), 2122-2145.