Transcript An Post - BIWA Summit 2017

DOING MORE
WITH
BUSINESS ANALYTICS
AND
BIG DATA
Tony Cassidy | Vertice | CEO
Charlie Berger | Senior Director |Oracle
OUR MISSION
To provide world class postal,
distribution and financial
services with unrivalled local
community access and global
connections
AN POST’S REPUTATION
Best Performing Semi-State and the 2nd Most Reputable Indigenous Irish Firm1
1.7m customers in Post Offices each week
56% of adults visit a Post Office at least once a week
84% of adults visit a Post Office at least once a month
1. RepTrak 2014 Study, Corporate Reputations
MAILS
 9000 collection points
 2700 vehicles
 Automated Mails Processing
 4000 delivery routes
 2.1 million homes
 2.8 million pieces of mail per day
 87 million events tracked for
barcoded product
 Nationwide operations
MAILS AUTOMATION
 4 mail centres
 Combined processing speed of up
to 623,000 mail pieces per hour

4,500 Intermec CN50 mobile devices

Capture of delivery data at point of delivery

Customer signature captured electronically

80% of tracking data online within 3 mins

97% of tracking data online within 20 mins
Largest Retail Network in Ireland
Largest Retail Network in Ireland
RETAIL NETWORKS
 Fully Automated networks
 1100+ Post Offices
 130M+ Transactions per annum
 POS Technology
 Backend systems
 Huge range of customer offerings;
 Mails
 Bill Payment
 Savings
 Banking
 Welfare
 Insurance
 FX
 PostMobile
 Top-up
 Continuous Development & Innovation
INNOVATION and BIG DATA ANALYTICS PLATFORM - Enabling
 Continuous Focus on
Innovation
 Leveraging Oracle
Technology Investment to
benefit Customer
 Business Analytics helping to
drive performance
An Post Insight
DATA SOURCES
Mails Processing Systems
Quality External Stats
HR Attendance Data
Mails Volumes
Complaints/ Enquiries
Delivery Manpower Data
Collections
Track and Trace
Payroll Data
Retail Systems
Financial Systems
Quality External Stats
DATA SOURCES
Mails Volumes
 Overnight data load processes,
continuous
HR Attendance Data
monitoring, data quality controls
 System deployed to almost 600 interactive
users covering Finance, Manpower
Costs and
Complaints/ Enquiries
Mails Operations and now Retail
 KPI data displayed to operational staff in mails
Delivery Manpower Data
processing plants nationwide
Collections
 Analysis through Interactive Dashboards
 DV, Answers, Smartview deployed for
Track and Trace
additional Self Service analysis
Payroll Data
 Matured & active processes for Governance &
Retail Systems
Financial Systems
Change Management
DOING MORE With BIG DATA ANALYTICS PLATFORM
 Leveraging existing solution
 Extending solution to incorporate
comprehensive Data Layer for Retail
business
 Analysis of transactions, product,
office and performance data available
when needed
 Improved Information access
supporting key business processes
BIG DATA ANALYTICS PLATFORM + Hybrid Cloud
 Private Cloud – Managed Service:
 Exadata in 2 An Post Data centres
 An Post Business Critical Data in a
Private Secure Cloud
 Public Cloud - PaaS:
 Cloud Backup
 Cloud Archiving
 An Post Non Business Critical Data in
a Public Secure Cloud
 EU Oracle Data Centres
 BICS for DV
 Public Cloud - SaaS:
 PBCS Prototype
BIG DATA ANALYTICS PLATFORM
 Business Analytics and EPM and
APEX:





OBIEE
BI Apps
Essbase
Financial Planning
APEX
 Data Integration Suite:
 Oracle Data Integrator
 Oracle Golden Gate
 Oracle Golden Gate CS
 Big Data Analytics Platform:
 “Oracle Information Management
Reference Architecture”
 Oracle Advanced Analytics
 Oracle 12c Multitenant
Database
 Oracle Exadata
 Oracle Adv. Compression
BIG DATA ANALYTICS PLATFORM + Data Integration Suite
 ODI 12c
 Redeployment of Architecture
 Much more productive and cohesive UI
 Ready for use with Hadoop Adapters as
needed and so ODI for Big Data.
 Batch Processing
 1 Example:
 Legacy Batch ETL Process = 11 Hours
 Introduce Exadata and 12c DB = 4 Hours
 Performance Tune = 36 Mins
 Opened up the Overnight Batch Windows to
additional processing, when applied to
multiple batch loads
 Golden Gate and GGCS
 NEW “Retail” Subject Areas are Near Real Time
with Analytics updated in 15 min intervals…
 Further Utilisation fully across platform for
Near Real Time Analytics.
 Pertinent Data LIFT to all CS via GGCS
BIG DATA ANALYTICS PLATFORM + Business Analytics
 OBIEE 11g
 Upgrade to 11g Architecture in 2015
 12c Upgrade on long finger
 Data Visualisation
 Deploying Data Visualisation aspects NOW!
 Prototype of DV in BICS (and soon OACS!)
 Data Externalisation to Customers
 Essbase
 Legacy MIS migrated within APBI and OBIEE
Dashboards, parallel run. Legacy Finance and
Retail Data cubes built iteratively over time.
 Removal of some legacy “Spread marts” built
of the back of prior version of Essbase.
 NEW Retail DW and Financial DW ensures
single source of truth, readily available data in
analytic format. With Succinct use of Essbase!
 And soon Essbase ongoing = OACS Mid Level!)
 Financial Planning




Planning and Forecasting deployed in 2015
Wider Group Budgeting added in early 2016
Strategic 5 year plan added in Mid 2016
PBCS Migration in Prototype!
BIG DATA ANALYTICS PLATFORM +
 Possible Future additions – 2017:

“as and when” qualified defined needs in
the Solution Architecture;
 Big Data Appliance
 Big Data SQL
 Big Data Discovery
 Oracle Stream Analytics
 Oracle Big Data Cloud Service
 Overall Solution:






Structured, Semi-structured and
unstructured
Future proofed for Growth
Big Data ongoing which is Readily Available
for Business Analytics
Both Enterprise and Self Service Analytics
Mature Data Governance
Single Source of the Truth Across All
Business Areas!
CONCEPTUAL OVERVIEW + Big Data “Journey”
Actionable
Events
Actionable
Information
Actionable
Insights
Structured
Enterprise
Data
Data
Event Engine
Data
Data Factory
Reservoir
Streams
Enterprise
Information Store
Reporting
Other
Data
Execution
Innovation
Events
& Data
Discovery Lab
Discovery
Output
BIG DATA ANALYTICS PLATFORM + Advanced Analytics
 Oracle Advanced Analytics 12c




Deployed in early 2016
7 Projects Outlined
1st Deployed
2nd and 3rd Progressing to UAT
 Big Data Analytics Platform
 OAA will predominantly utilise the data being
stored in the APBI Big Data Analytics platform
 OAA Innovation Lab




Pilot work for all Projects
Integrated with main Architecture
Access to Prod and Test environments
Typical data volumes being process >630M records
in <2s
 Push out and integrate into Production processes
 Seamless integration of OAA with architecture
 Continuous improvement process
 Feedback process
 Data enrichment
Oracle Advanced Analytics—Best Practices
Nothing is Different; Everything is Different
1. Start with a Business Problem
Statement
7. Automate and Deploy Enterprise-wide
6. Quickly Transform “Data” to “Actionable
Insights”
5. Be Creative in Analytical Methodologies
22
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
2.
Don’t Move the Data
3.
Assemble the “Right Data” for the
Problem
4. Create New Derived Variables
Brendan Tierney
Oracle ACE
Director, Author,
etc.
1. Start with a Business Problem Statement
Clearly Define Problem
“If I had an hour to solve a
problem I'd spend 55
minutes thinking about the
problem and 5 minutes
thinking about solutions.”
― Albert Einstein
23
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Be Specific in Problem Statement
Poorly Defined
Better
Predict citizens likely to need
certain Social Services
• Based on past employees that voluntarily left:
• Create New Attribute EmplTurnover  O/1
Predict customers that churn
• Based on past customers that have churned:
• Create New Attribute Churn  YES/NO
Target “best” customers
• Recency, Frequency Monetary (RFM)
Analysis
• Specific Dollar Amount over Time Window:
• Who has spent $500+ in most recent 18 months
How can I make more $$?
• What helps me sell soft drinks & coffee?
Which customers are likely to
buy?
• How much is each customer likely to spend?
Who are my “best
customers”?
• What descriptive “rules” describe “best
customers”?
Detect transactions that could
be fraudulent
• Which transactions are the most anomalous?
24
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Then roll-up to physician, claimant, employee, etc.
Data Mining
Technique
Be Specific in Problem Statement
Poorly Defined
Better
Predict citizens likely to need
certain Social Services
• Based on past citizens that needed service:
• Create New Attribute Needed Service  O/1
Predict customers that churn
• Based on past customers that have churned:
• Create New Attribute Churn  YES/NO
Target “best” customers
• Recency, Frequency Monetary (RFM)
Analysis
• Specific Dollar Amount over Time Window:
• Who has spent $500+ in most recent 18 months
How can I make more $$?
• What helps me sell soft drinks & coffee?
Which customers are likely to
buy?
• How much is each customer likely to spend?
Who are my “best
customers”?
• What descriptive “rules” describe “best
customers”?
Detect transactions that could
be fraudulent
• Which transactions are the most anomalous?
25
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Then roll-up to physician, claimant, citizen, employee, etc.
Data Mining
Technique
Be Specific in Problem Statement
Poorly Defined
Better
Predict citizens likely to need
certain Social Services
• Based on past employees that voluntarily left:
• Create New Attribute Needed Service  O/1
Predict customers that churn
• Based on past customers that have churned:
• Create New Attribute Churn  YES/NO
Target “best” customers
• Recency, Frequency Monetary (RFM)
Analysis
• Specific Dollar Amount over Time Window:
• Who has spent $500+ in most recent 18 months
How can I make more $$?
• What helps me sell soft drinks & coffee?
Which customers are likely to
buy?
• How much is each customer likely to spend?
Who are my “best
customers”?
• What descriptive “rules” describe “best
customers”?
Detect transactions that could
be fraudulent
• Which transactions are the most anomalous?
26
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
• Then roll-up to physician, claimant, citizen, employee, etc.
Data Mining
Technique
1. Start with a Business Problem Statement
Identify and/or Create the Right Target Field
 Target behavior
that is of interest
to you e.g. certain
or different
behaviors
27
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
1. Start with a Business Problem Statement
Identify and/or Create the Right Target Field
 Target “best” customers
– Stratify into Low, Medium,
High and Very High
– Explore node
– Transform node
28
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
2. Don’t Move the Data
Data Preparation & Adv. Analytical Process Runs In-Database
Additional relevant
data and “engineered
features”
Historical or Current Data
to be “scored” for
predictions
Oracle Database 12c
Historical data
Assembled
historical data
Sensor data, Text, unstructured
data, transactional data, spatial
data, etc.
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Predictions &
Insights
3. Don’t Move the Data
Data Sources
 Data Source node
 Aggregate node
 Join node
30
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
3. Assemble the “Right Data” for the Problem
 Collaborate with Domain Experts, IT and Data
Analysts
– Business Domain—people who know the business
 Marketing & Sales
 Customer Service
 Operations
– Information Technology—people who have access
to data
– Data Analysts—people who have data analysis skills
Business
Domain
(Statisticians/Data Miners/Data Scientists)
 Demand active participation & buy-in




31
“These leads are no good”
“That data is old”
“This model is 100% accurate”
“Why didn’t you use this data?”
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
Information
Technology
Data
Analysts
4. Create New Derived Variables
 Transform node
 Aggregate node
 Text node
 All generate
associated
SQL code
32
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
4. Create New Derived Variables
New Server Functionality
 Text Mining Support
Enhancements
– This enhancement greatly simplifies
the data mining process (model build,
deployment and scoring) when text data is
present in the input:
 Manual pre-processing of text
data is no longer needed.
 No text index needs to be
created
 Additional data types are
supported: CLOB, BLOB, BFILE
 Character data can be specified
as either categorical values or
text
33
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
SQL Pattern Matching in action
Example: Find W-Shape*
Stock price
Find a W-shape pattern
in a ticker stream:
•
34
Output one row each
time we find a match to
our pattern
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.
1
9
13
19
days
SELECT first_x, last_z
FROM ticker MATCH_RECOGNIZE (
PARTITION BY name ORDER BY time
MEASURES FIRST(x.time) AS first_x,
LAST(z.time) AS last_z
ONE ROW PER MATCH
PATTERN (X+ Y+ W+ Z+)
DEFINE X AS (price < PREV(price)),
Y AS (price > PREV(price)),
W AS (price < PREV(price)),
Z AS (price > PREV(price)))
* For conceptual clarity, the statement is simplified and ignores an always-true start event.
See the notes or documentation for further explanation
More Data Variety—Better Predictive Models
Engineered Features – Derived
attributes/variable that reflect domain
100%knowledge—key to best models
• Increasing sources of
relevant data can
boost model
accuracy
Naïve Guess
or Random
Responders
Model with “Big Data”
and hundreds -thousands of input
variables including:
• Demographic data
• Purchase POS
transactional data
• “Unstructured data”,
text & comments
• Spatial location data
• Long term vs. recent
historical behavior
• Web visits
• Sensor data
• etc.
100%
Model with 20
variables
Model with 75
variables
Model with 250
variables
0%
Population Size
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
5. Be Creative in Analytical Methodologies
Leveraging a Variety of Data Sources and Types
SQL Joins and arbitrary SQL
transforms & queries – power of
SQL
Transactional
POS data
Modeling
Approaches
Consider:
• Demographics
• Past purchases
Generates SQL
• Recent purchases
scripts and
• Comments & tweets
Inline predictive
model to
augment input
data
Advanced Analytics
Unstructured data
also mined by
algorithms
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
workflow API for
deployment
Oracle Advanced Analytics DB Option
In-Database Machine Learning Algorithms*—SQL &
Classification
•
•
•
•
•
Decision Tree
Logistic Regression (GLM)
Naïve Bayes
Support Vector Machine (SVM)
Random Forest
Advanced Analytics
& GUI Access
Clustering
Predictive Queries
• Hierarchical k-Means
• Clustering
• Orthogonal Partitioning Clustering • Regression
• Expectation-Maximization
• Anomaly Detection
• Feature Extraction
Attribute Importance
A1 A2 A3 A4 A5
A6 A7
Regression
•
•
•
•
•
•
Multiple Regression (GLM)
Support Vector Machine (SVM)
Stepwise Linear Regression
Linear Model
Generalized Linear Model
Multi-Layer Neural Networks
Anomaly Detection
• 1-Class Support Vector Machine
• Minimum Description Length
• Unsupervised pair-wise KL div.
Market Basket Analysis
Feature Extraction & Creation
• Nonnegative Matrix Factorization
• Principal Component Analysis
• Singular Value Decomposition
• Apriori – Association Rules
Text Mining
• All OAA/ODM SQL ML support
• Explicit Semantic Analysis
Time Series
• Single & Double Exp. Smoothing
Open Source R Algorithms
• Ability to run any R package
(9,000+)via Embedded R mode
+ Ability to Mine Unstructured, Structured & Transactional data
+ Partitioned Models
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
7. Automate and Deploy Enterprise-wide
BI Dashboards + Predictive/Actionable Insights
38
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
8. Feedback Loop!
With each iteration we are influencing / changing
the behavior of the customers/people
39
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
CONCEPTUAL OVERVIEW + Big data
Actionable
Events
Actionable
Information
Actionable
Insights
Structured
Enterprise
Data
Data
Event Engine
Data
Data Factory
Reservoir
Streams
Enterprise
Information Store
Reporting
Other
Data
Execution
Innovation
Events
& Data
Discovery Lab
Discovery
Output
CONCEPTUAL OVERVIEW + Big data
Discovery Lab
Operational Insights
Information Management
Transformation
Big Data Pilot
Factors To Consider:
Sponsorship
Budget
Use Cases
Timeline
Skills
Big Data Application
SUMMARY
 Continuous Modernisation & Innovation
 Ability to quickly respond to new business
opportunities
 Well positioned for future opportunities &
challenges
 Doing More!
 Reaping rewards of investment in Oracle Technology,
Business Analytics success to date & Big Data ongoing!
 Award Winning!
Thank You
FUTURE PROOFING
THROUGH TECHNOLOGY
www.Vertice.ie