Transcript Enablers for IoT Analytics in Smart Cities – John Soldatos, VITAL

VITAL
VIRTUALIZED PROGRAMMABLE INTERFACES FOR INNOVATIVE COST-EFFECTIVE
IOT DEPLOYMENTS IN SMART CITIES
IoT Analytics with the VITAL Smart
Cities Platform
Prof. John Soldatos
Athens Information Technology, Greece – VITAL
(Acknowledgement: Prof. Ioannis Christou)
IoT Week, Belgrade, June 2, 2016
© Copyright 2016
VITAL Consortium
Value Potential of IoT (McKinsey, June 2015)
 Interoperability between IoT systems is critically important to
capturing maximum value:
o On average, interoperability is required for 40 percent of potential value
across IoT applications and by nearly 60 percent in some settings
 Most IoT data are not used currently:
o Only 1 % of data from an oil rig with 30,000 sensors is examined
o Data that are used today are mostly for anomaly detection and control,
not optimization and prediction, which provide the greatest value
2
© 2015 VITAL Consortium
VITAL Motivation & Challenge
Integrating Silos & Reducing Fragmentation
Process Integration, Integrated Security, Enhanced Intelligence, City Operations
Optimization
Organizational
Silos
Sustainable Development
Connected Governance
Natural Resources Management
VITAL Virtualization Layer – Integrated Development
Application
Silos
Information
Silos &
Fragmentation
IoT for Smart
Industries
IoT for Smart
Buildings
IoT for Urban
Transport
Platform &
Applications
Platform &
Applications
Platform &
Applications
IoT for Law
Enforcement
Platform &
Applications
Technical Silos
Fragmented ICOs Access, Fragmented Intelligence, Fragmented Security, Limited Data Sharing,
Limited Integration
3
© 2016 VITAL Consortium
VITAL Architecture
Loosely Coupled Modules (REST, JSON-LD)
IoT Systems are accessed via a Virtualized Abstract PPI
(Platform Provider Interface)
IoT data are modeled according to a common (VITAL)
ontology (extending W3C SSN)
Added Value Functionalities (CEP, Discovery, Filtering)
are provided via Virtualized Interfaces (VUAIs), but (also)
through PPIs for specific platforms
VITAL Provides a range of development & management
tools
4
© 2016 VITAL Consortium
VITAL City Management Platform (1)
5
© 2016 VITAL Consortium
VITAL City Management Platform (2)
6
© 2016 VITAL Consortium
VITAL City Applications Development Tool
 Contains all VITALrelated nodes
 There is one node for
each piece of
functionality that
each one of the
integrated
components provides
 R Node provides
integration with R
project
7
© 2015 VITAL Consortium
IoT Analytics Disciplines (incl. BigData)
Statistics &
Machine
Learning
Visualization
& HMI
IoT Analytics &
Data Mining and
Knowledge Discovery
IoT Data
Collection &
Intoperability
Databases
(SQL, noSQL, HDFS,
Cloud,..)
8
© 2016 VITAL Consortium
Data Processing and Analytics
Workflow in Smart Cities
Source: Scottish Cities Alliance, “Smart Cities Maturity Model and Self-‐Assessment Tool Guidance Note for
completion Of Self-‐Assessment Tool”, January 2015.
9
© 2016 VITAL Consortium
JSON
VITAL PADA
Module
Dynamic Data
Discovery
Common
Semantics
JSON-LD
Contexts
Linked Data
10
IoT Platform Agnostic
Analytics
VITAL PPI
Semantic Unification &
Interoperability
Data (Streams) Collection
VITAL IoT Analytics Pipeline
VITAL
Development
Tool
R Node for
Analytics
Functions
© 2016 VITAL Consortium
Use of Cross Industry Standard Process for Data
Mining (CRISP-DM)
 VITAL Data Scientists
use CRISP-DM like
model
 Sample / Prepare /
Model Data
 Test / Validate and
Evaluate DM
Mechanisms (off-line)
 Deployment using
VITAL Platform (on-line)
Shearer C., The CRISP-DM model: the new blueprint for data mining, J Data Warehousing (2000); 5:13—22.
11
© 2016 VITAL Consortium
VITAL Validating Use Cases
Scenario
Scope
City
Smart
Working
Patterns
Use IoT to connect supply
chain to smart retail, smart
services, smart logistics
London
Improve Environmental
Performance and optimize
the efficiency of the
transport network
Istanbul
-Urban Regeneration -
Smart Traffic
Management
-Smarter Services-
12
© 2016 VITAL Consortium
Camden Data Collection
 DataSet:
o Camden Footfall Data from all locations for 10 days
(23/10/2015-02/11/2015)
 Preprocessing:
o Measurements from different sites and locations have
been merged together
o Analyses applies to the entire dataset, rather than the
individual datasets coming from the individual sources
 Problem:
o What is the peak hour for Camden market?
13
© 2016 VITAL Consortium
3-Hour Accumulation of People
 Visualization of 3hour accumulation
of people over the
Hour-of-Day
 No discernible
pattern of
correlation
between the two
variables.
14
© 2016 VITAL Consortium
People Coming-In vs. Hour Per Day
 Relation between number of
people coming-in (in an
hour), and the actual Hourof-Day
 There is a clear peak in
certain hours, but there are
many instances during the
same hours in which the
number of incoming people
is very low
15
© 2016 VITAL Consortium
Highlight of VITAL Machine
Learning Toolbox
 Quantitative Association Rule Mining (QARM)
algorithms going beyond all currently known QAR
techniques, specifically customized.
 Utilizes domain knowledge and is highly
parallelizable and distributable in nature.
 State–of –the-art fusion of user-based, item- based,
and content-based algorithms
o Achieving 64 times faster response times than SOTA
(e.g., Apache Mahout) while improving results by a
whopping 100%.
16
© 2016 VITAL Consortium
Applying QARM
 Mining all quantitative rules from the dataset comprising the
following features:
o LocationNumber, SiteNumber Hour-Of-Day, Day-Of-Week,
AcceptedIn, AcceptedOut, AInMinusOut, 3HourCumAInMinusOut,
HourlyAInPercChg, DailyAInPercChg, HourlyAOutPercChg,
DailyAOutPercChg
 Algorithm produced all valid non-dominated rules whose
antecedent is the 3-hour accumulation of people
o Requirement that this quantity must exceed the value 10, having
minimum support of 10% and minimum confidence of 80% in the
dataset.
 More than 1000 rules produced – top 20 can serve as basis for
planning:
o Production of rules for certain locations vs. time of day
17
© 2016 VITAL Consortium
Smart Traffic Management Scenario # 1
• Incident Detection:
- When a sharp decrease in average speeds is detected for a road segment,
a notification is generated to inform about a potential incident
18
© 2016 VITAL Consortium
Smart Traffic Management Scenario # 2
• Sensor Failure Detection:
-By comparing speeds collected from sensors & floating cars,
automatic notification is generated when a contradiction is detected,
i.e. mismatch in road segment colors
19
© 2016 VITAL Consortium
Smart Traffic Management Scenario # 3
• Traffic Prediction:
-Traffic estimation up to an hour is generated by using Modified Linear
Regression algorithm & calculations are made using stored data considering
external conditions such as national holidays & other events.
Data Flow Diagram
Historical & Live
Traffic
Data w/
External
Conditions
Processing &
Calculations
Predicted Traffic Status
(up to an hour)
Instant
Condition
20
© 2016 VITAL Consortium
VITAL Added-Value for Analytics
 Integrated cross-platform and cross-context approach to
the development & deployment of IoT Analytics
applications in smart cities
o Emphasis on Semantic Interoperability
 Added-value intermediary (proxy) between all different
IoT deployments and systems in the smart city
 Provides access to cached, aggregated, integrated data
(Data-as-a-Service)
 Open Source Solution targeting Smaller Cities (e.g., ~ up
to 200.000 inhabitants) which cannot afford the Cost of
Enterprise Scale Solutions from giant vendors
21
© 2016 VITAL Consortium
Conclusions
 IoT Analytics provide a huge potential for extracting
knowledge and deriving insights about humans’ behaviour
and the physical environment
o IoT Data remain largely unexploited
o Interoperability is an issue
 IoT Analytics present their own challenges:
o Velocity: Streams with high ingestion rates
o Semantic Interoperability: Alleviate the fragmentation of IoT
systems
o Lack of Tools: IoT Development tools are not enough
 VITAL is providing the means to confront some of these
challenges
22
© 2016 VITAL Consortium
VITAL
VIRTUALIZED PROGRAMMABLE INTERFACES FOR INNOVATIVE COST-EFFECTIVE
IOT DEPLOYMENTS IN SMART CITIES
Thank You!
IoT Analytics with the VITAL Smart Cities
Platform
Prof. John Soldatos
Athens Information Technology, Greece
Belgrade, June 2, 2016
© 2016 VITAL Consortium