Transcript Big Data - Lanyon Events

“Innovation through Prediction”
- Hybrid Cloud Big Data Platform
Learn. Predict. Influence.
John Andrew
Oracle Enterprise Architect
[email protected]
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
2
Agenda
1
Innovation Business Drivers
2
Use Case Context
3
Reference Architecture Context
4
Hybrid Cloud Solution Context
5
Q&A
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
3
What if …
1. You knew what products your customers would be the most
likely to buy in advance?
2. You could maximize your profits by determining the highest
price a customer will pay for a product?
3. You could optimize customer service to resolve concerns
proactively before they become issues?
and finally….
4. Political parties had a way of determining and influencing
the voters to vote for them?
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
4
What does it mean for Business…
Increased Customer Drastically Reduced
Satisfaction and
Business Risk
Revenue
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
Increased
Efficiency and
Productivity
5
Core business themes and building blocks …
Predicting and
Influencing
Model Accuracy
Anytime & Anywhere
availability
Time to Value and
Economy of Scale
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
6
Google Trend Analysis as of Oct 2015
Prediction
Predictive
Analytics
Model
Big Data
Accuracy Foundation
Economy
Cloud
Of scale Deployment
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
7
Hindsight -> Insight -> Foresight
Descriptive
Analytics
Predictive
Analytics
Prescriptive
Analytics
Business • Reports
Expectation • Alerts
• Discovery
• Forecasting
• Trends
• Relationships
• Prediction
• Next Best Action
• Influencing
Architecture • RDBMS / SQL
Pattern • (x)OLAP
• Warehouse
• Analytics
• Models
• Semantics
• Machine learning
• Decision Hub
• Optimization
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
8
More Data + Variety Data -> Better Predictive Models
Model with “Big Data” and
hundreds -- thousands of input
variables including:
• Customer sentient data
• Competitors data
• Environmental data
• Spatial location data
• Long term vs. recent
historical behavior
• Sensor data
• etc.
Failure Prediction Accuracy
• Increasing sources of
relevant data can boost
model accuracy
100%
100%
Naïve Guess or
Random
Model with 20 variables
Model with 75 variables
Model with 250 variables
0%
Population Size
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
9
Agenda
1
Innovation Business Drivers
2
Use Case Context
3
Reference Architecture Context
4
Hybrid Cloud Solution Context
5
Q&A
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
10
Predictive Analytics Use Cases
Predictive Pricing
(Competitive, Dynamic,
and Demand)
Fraud Prevention
(Box tops)
Predicting Influence
(Product and Customer)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
11
Collecting All Pricing related Data… “Data Wrangling”
Enterprise Data
Non Enterprise Data
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
12
“Training” Pricing model … Machine Learning
Customer
Preference
Consumer
Price Index
Historical
Pricing Avg
Inventory
Turnover
Segment
ID
Customer
Segment
Producer
Price Index
Yes
80%
.12
.60
FM
HV
78%
No
60%
.34
.15
ANP
MV
65%
No
65%
.12
.30
FR
LV
60%
No
50%
.18
.35
PUR
MV
55%
Yes
78%
.16
.70
NUTR
HV
80%
No
95%
.53
.40
RB
LV
90%
No
74%
.45
.25
CGF
HV
75%
No
70%
.66
.38
SFI
MV
65%
Have an algorithm determine what is different and the importance of
the differences in the green and gray metrics to figuring out preference
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
13
Advanced Analytics Algorithms
More than just linear regression to help predict the future and discovery relationships
Algorithms works across data sets (Relational and Non-Relational)
Classification
Clustering
Attribute Importance
Logistic Regression
Hierarchical k-Means
Minimum Description Length
Decision Trees
Hierarchical O-Cluster
Principal Components Analysis
Naïve Bayes
Support Vector Machines
Regression
Expectation-Maximization
Anomaly Detection
One-Class SVM
Feature Extraction
Nonnegative Matrix Fact(NMF)
Singular Value Decomposition(SVD)
Linear Regression
Support Vector Machines
Multi-Layer Neural Networks
Text Mining
Collaborative Filtering (LMF)
Tokenization
Theme Extraction
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
14
Current State Constraints and Gaps
1. Analytical Challenges
•
Misspecification and using a sample to estimate the model
•
Resource (memory) constraints of analytical scripts (R scripts)
•
Analyze data without help from IT
2. Data Management Challenges
•
Complexity and Cost issues resulted using smaller data sets for analytics
•
Smaller the data sets, less accurate analytical outcomes
•
Data latency issues increased as the result of data exists in multiple places
3. Deployment Challenges
•
Up front large CapEx to build and deploy the Platform
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
15
Agenda
1
Innovation Business Drivers
2
Use Case Context
3
Reference Architecture Context
4
Hybrid Cloud Solution Context
5
Q&A
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
16
f(predictive analytics) = ( unified data +
predictive models +
@scale)
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
17
Architecture Vision
Creating an Unified Data + Advanced Analytic Platform
for the Era of Big Data and the Cloud
Simple
Unify
Secure
Resilient
• Any data size
• Enterprise (All) Data • Control Access
• Reliable
• Any data variety
• Analytical Models
• Protect
• Timely
• Any platform
• User Interaction
• Integrity
• Elastic
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
18
Architecture Themes
Architecture Fit
Financial Fit
Operational Fit
As-is Data Discovery
At-source Data Analytics
At scale and Performance
Reduced $ per Model
CapEx to OpEx Transition
Lower TCO
Automated Infrastructure
Unified Management
Simplified Support Model
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
19
On-Premises Architecture Pattern – Ingestion to Analytics
3rd Party Data
Cloud Service
❶
❷
Data Ingestion
Service
Big Data Service
❸
Big Data
Discovery
Service
Big Data SQL
xls
csv
Enterprise Data
Warehouse
❹
❺
Client R Engine
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
20
Cloud Architecture Pattern – Ingestion to Analytics
❶
Data Preparation
Cloud Service
xls
csv
❸
Big Data
Discovery Cloud
Service
❷
Big Data Cloud
Service
Big data SQL
3rd Party Data
Cloud Service
❹
Database Cloud
Service
❺
Client R
Engine
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
21
Hybrid Architecture Pattern – Ingestion to Analytics
3rd Party Data
Cloud Service
❶
Big Data
Preparation
Cloud Service
xls
csv
❷
Big Data Cloud
Service
Enterprise Data
Warehouse
❹
❸
Big Data
Discovery Cloud
Service
❺
Client R
Engine
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
22
Analytic Reference Architecture
Built it once and use it multiple times
Functional
Analytical
Services
3
2
Foundational
Analytical
Services
1
4
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
23
Reference Architecture Data Flow …
Data Sources
Data Ingestion
Oracle
Sales &
Inventory
Data Reservoir
(Semi and Unstructured)
Structured
Continuous
Data
Warehouse
Orchestration,
Integration,
Lineage,
and
Preparation
IBM
Competitors
Data
Discovery Lab
Analytical Lab
Enterprise
Business Users
Development
Analytics
Data
Discovery
Structured
Daily Updates
Archive
Mainframe
Meta
data
Spark
Teradata
Data Consumption
Data Storage and Management
Unstructured
Hadoop (HDFS)
NoSQL
Data Warehouse
(Structured)
On-demand
Outside
Data
• Public Sources (Free)
• 3rd Party Sources (Pay)
•
•
•
•
•
•
Import and Ingest
Cleanse and Normalize
Repair and Standardize
Classify and Extract
Augment and Enrich
Visualization
Enterprise
Data
Search
Visualize
Transform
Share/Subset
Analytical
Models
Analytic
Engine
• Data Movement
• Data Access
Structured
•
•
•
•
Unified
Secure
Access
Data
Mining
Analytic
Engine
In-memory
Processing
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
• Regression & classification
• Anomaly detection
• Segment analysis
BI and
Analytics
• Unified models
• Sense and respond
• Mobile interaction
24
Agenda
1
Innovation Business Drivers
2
Use Case Context
3
Reference Architecture Context
4
Hybrid Cloud Solution Context
5
Q&A
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
25
Oracle Analytics as a Service Platform
Private, Public and Hybrid Cloud deployment options
Graph Analytics
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
26
Predictive Pricing Hybrid Cloud Solution realization
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
27
Unified Analytical Model Execution
Client R Engine
R Functions
ORE packages
Oracle Database
In-db
stats/dm
User tables
Database
Server
Machine
Big SQL Services
B
Data Mining
Oracle R Advanced Analytics for Hadoop
Enterprise R
Oracle DB Advanced Analytics
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
28
Kick Start Your Hybrid Cloud Big Data Strategy
1. Guiding Success Factors
• Integrate business and technology vision (Identify stockholders that will carry
over to implementation). Focus on the next 12 months
• Identify your target architecture
₋ Avoid tunneling on one use case
• Keep in mind Big Data is not a cure-all
• Hadoop is a complementary to your existing EDW. It could very well be your
“System of truth” but most likely not your “System of record”
2. Jump start your Innovation with
• Oracle proven reference architecture
• Oracle proven Analytical platform
• Oracle proven hybrid cloud deployment
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
29
Why Oracle Hybrid Cloud Analytical Platform?
Discover and
Predict – Fast
Simplify Access to
All Data
Govern and Secure
All Data
Enterprise-Grade Cloud Capabilities
Performance
Integration
Availability
Elasticity
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
Manageability
30
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
31
Copyright © 2015 Oracle and/or its affiliates. All rights reserved.
32
33