Transcript CON2161 Big Data in Financial Services: Technologies, Use Cases and Implications

CON2161
Big Data in Financial
Services: Technologies, Use
Cases and Implications
Jim Acker
Global Solution Manager for Big Data
Industry Business Unit, Financial Services
1
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Understanding the Drivers
Executives frustrated with their data gathering and distribution systems
Executives’ Biggest Data Management Gripes:*
#1
Don’t have the right systems in place to gather the
information we need (38%)
#2
Can’t give our business managers access to the
information they need; need to rely on IT (36%)
#3
Systems are not designed to meet the specific needs
of our industry (29%)
#4
Can’t make sense of the information we have and
translate it into actionable insight (25%)
#5
Information is no longer timely by the time it makes it
to our business managers (24%)
* Source: Oracle Overload to Impact Study 2012
2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
The data problem just got a lot bigger
Leveraging untapped data for commercial gain
571
New
Websites
3
695,000
204,166,667
Status updates
Emails
510,040
Comments
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
2,000,000
Search Queries
The Big Data Opportunity
Big Data: Techniques and
Technologies that Enable Enterprises
to Effectively and Economically
Analyze All of their Data
4
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Big Data is ALL Data
Unstructured, Semi-Structure and Structured
What is the
main difference in this data?
There is always structure. But its not formally defined
or anticipated.
Social Media, RSS feeds, Videos, DOCs, PDFs, Graphics
Volume, Velocity, Variety, Value
These Characteristics Challenge your
Semi-Structured. Does not conform to DB tables, but
Existing
Architecture
still contains
tags or semantic elements.
Emails, Thought
log files, machine
generated content
and your
Processes
5
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Contrast in Big Data Models
Demands a new holistic look into data architecture
Relational DB
Distributed File System
HDFS
Schema on
Read
No / Minimal
Extreme Scale
Batch / slow – getting faster
Minimal
Flexibility and time to value
6
SQL
Map-Reduce
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Data Model
Scale
Processing
Security
Advantages
Schema on
Write
Explicit
Large Scale
Real time and batch
Robust
Optimized and familiar
RDBMS
Pulling it ALL Together for Business Value
 Create value from the full range of data sources
– Its about using ALL your data
– No more sampling
 Value First
– Let the data drive the questions, or …
– Test a hypothesis against all your data
 Still Need Information Management
– Once you find value, incorporate IM
– Big Data is NOT a Silo
7
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
A Word of Caution
Gartner Hype Cycle for Big Data
You are
Here
8
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Big Data in Financial Services
9
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Big Data is About Analytics
ALL
DATA
10
Copyright © 2013,
2012, Oracle and/or its affiliates. All rights reserved.
BETTER
DECISIONS
FASTER
ACTION
Big Data Use Cases Today
Correlating Diverse Data Sets
Finding and Monetizing
Unknown Relationships
Drive Opportunity
Reduce Cost
11
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Big Data Solutions for Financial Services
Two main patterns for how customers are using Big Data
 IT Optimization
• ETL and batch processing
• Mainframe offloading
• Extended Data Warehouse
• Archiving
 Big Data Analytics
•
•
•
•
12
Customer 360
Cross-selling / Geo-fencing
AML / Anti-Fraud
Pricing Management
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
•
•
•
•
Omni-channel CX
Payment Analytics
Risk Management
Compute Offload (VAR)
IT Optimization
13
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Big Data Usage Pattern
ETL and Batch Processing Workloads on Hadoop
SQL
DW & BI
•
•
•
Scalable
Integrate
Flexible
Cost
Effective
SQL
Analytics
NoSQL
Web
Mainframe
14
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Regions Bank
Objectives
 Meet ever evolving regulatory requirements
 Consolidate existing deposit, loan and
customer databases
Solution
 Big Data Appliance and Exadata ODS for
single, reliable, cleansed data source
 ODS is single landing zone and archival
repository for internal, external, structured,
semi-structured, and unstructured data
15
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Results & Benefits
•
Faster access to all their data
•
Reduced IT costs by eliminating duplicate
data stores
Thomson Reuters
Objectives
 Maximize cross-sell opportunities
 Lower cost and complexity
Solution
 “Oracle's engineered systems… are geared
toward high performance big data delivery - and
that is exactly the type of work we do”
Rick King
Chief Operating Officer for Technology
Thomson Reuters
 Economically capture all customer activity
Upsell/Cross Sell
 Testing 50M events/sec ingest rates into
the Oracle Big Data Appliance
 Feeds Exadata EDW for customer
profitability & segmentation analysis
16
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Research
Applications
Event Capture
& Store
EDW
Sandbox & DR
BDA
Exadata
Interactive
Analytics
Exalytics
Big Data Usage Pattern
Business
Intelligence
Expand Data Warehouse with Granular Data Store
•
Online
•Data
Scalable
Factory
• Flexible
• Cost
Effective
Σ
Σ
Data Warehouse
Marts
Archiving
17
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Tier 1 Global Bank
New Information Management Architecture
Objectives
 End-to-end business information
environment that provides accurate,
transparent and timely information to
shareholders, regulators and management
Solution
 7 Exadata Racks
 16 Node Hadoop Cluster – 33TB
 Oracle Loader for Hadoop (pending)
18
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Results & Benefits
 Reduce complexity and risk of changes
 Reduce cost of operation
 Increased stability & performance
Big Data Analytics
19
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Big Data Usage Pattern
Scale-out Information Discovery
Continuous
• Online
•Data
Scalable
Factory
On-Demand
• Flexible
Ad-hoc
• Cost
Effective
20
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Credit Suisse
Increased sales through instant access to information
Objectives
 Enable customers to learn about stocks and
increase buying confidence
 Cultivate the advisor-client relationship
online and acquire smaller clients
Solution
 Information Discovery on pooled research
data sets in multiple unstructured formats
 Oracle powers their internal application that
advisors utilize to quickly find information on
financial metrics
21
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Results & Benefits
 Incremental sales for Bank based on this
application for 5 years.
 Improved customer relationships
Big Data Usage Pattern
Instant Responses based on Historical Analysis
Event
Decisions
Integrate
•
•
•
•
22
Online
Scalable
Flexible
Cost
Effective
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Business
Intelligence
NoSQL for Fraud Scoring
Objectives
 Combine data sources for complex scoring
 Detect, alert analyst with low latency
 Handle burst seasonal transaction volumes
Solution
Transaction Authorization
Processor
Financial Services coordinated theft prevention
Application Data Ingestion
NoSQL DB Driver
 Oracle Coherence cluster for real time
transaction object management
 Oracle NoSQL Database for fraud model
and customer profile management
 Oracle Database for statistics and fraud
modeling-related data
23
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Results & Benefits
 Simple data model, flexible transactions
 Scalable, Low Latency data management
 Easy configuration and administration
 Enterprise Support
Real-time Location-Based Offers
Tier 1 Global Bank
Objectives
 Increase revenue through real-time,
location based offers
Solution
 Customer profile enrichment with Big Data
 Capture credit card POS and merchant data with
event processor
 Determine geo location of POS and nearby bank
wholesale customers
 Leverage real-time decision engine to generate
offer to mobile device
24
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Tier 1 Global Bank
Offer Workflow
Locate and identify customer
Capture
credit card
transactions
& identify
customer
location
Select next best offer
Derive next
best offer
using
customer
information
and
propensity
Evaluate
offers
based on
customer
location
Make offer
through
mobile text
message
Enrich propensity based on acceptance/rejection
Identify next best offers
25
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Make offer
Analyze
customer
acceptance/
rejection
System Architecture
Oracle Big Data at Work
FACTORY
EXECUTION
REAL TIME EVENT CAPTURE
Near time/Batch to
perform
model update
Event Capture and
Co-relation
POS DEVICE
Temporal cache
based customer
identification
INTELLIGENT INTERVENTION
PLATFORM
Statistical modeling –
Propensity, segments
etc.
Natural language
processing
Intent and semantic
inference
Advanced model free
visualization
DATA VISUALIZATION LAYER
Real time decision
WEBSERVICES
MQ
ATM MACHINE
Adaptive self- learning
Routing
Rules
Mapping
LEGEND
Real-time/Near
Time, Batch
NEXT
BEST
ACTION
Integration
adapters
SMART PHONE
APP
Real time intervention
– click to chat, click to
call
Near real-time analysis
and dashboarding
Near time/Batch for
acceptance/rejection
data
MapReduce + NLP
ETL/Real-Time
Derived outputs- intent,
segment, enhanced
customer mastering
DATA PROCESSING LAYER
BANK REPOSITORIES
Client profile, historical transactions, Good life
data, segment info, profit info, risk info, Opt-in
information etc.
KEY VALUE PAIRS
STAGING
Map information, social
networks, device logs,
smart app interfaces etc.
Structured, Nonstructured, Semistructured
DATA TRANSPORT LAYER
DATA STORAGE LAYER
26
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Product Roadmap
27
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Engineering the Oracle Big Data Solution
Decide
Oracle Real-Time
Decisions
Oracle BI
Enterprise Edition
Endeca Information
Discovery
Apache
Flume
Cloudera
Hadoop
Applications
Oracle Event
Processing
Oracle
NoSQL
Database
Oracle R
Distribution
Stream
28
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Oracle Big Data
Connectors
Oracle
Advanced
Analytics
Data
Warehouse
Oracle Data
Integrator
Oracle
Database
Acquire – Organize – Analyze
In-Database
Analytics
Unified Analytics APIs
Why Make Big Data a Divided World?
VS
29
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Goal: Unified Data Analytics Environment
• All Data Online and Ready
to Use
• Large Scale
Systems
• Cost Effective
30
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
VS
• Real-Time
Analytics
• Thousands of
Users
• Secure and
Available
Unified Data Analytics Environment
Unified Analytics API
SQL
R
Hadoop
RDBMS
IB
Management Framework and Tools
Unified Analytics Processing Platform
31
MR
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Analyze Data across your Oracle Systems
SQL Analytics on ALL data
SQL
 Expand the data pool for
analytics leveraging Hadoop
Hadoop
Oracle Database
 Stream Hadoop resident data
through Big Data Connectors
for SQL processing
 Use the full power of Oracle
SQL on all data
 Or use Oracle Loader for
Hadoop to integrate data in
Oracle Database
32
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
IB
Analyze Data across your Oracle Systems
R Analytics on ALL data
R
 Expand the data pool for
Hadoop
Oracle Database
analytics leveraging Hadoop
 Improve scalability and
performance for R without
changes to your programs
 Dynamically leverage Hadoop
through Big Data Connectors
to execute R analytics
33
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
IB
Unified Data Analytics Environment
 All Data Online and
Ready to Use
 Large Scale
Systems
 Cost Effective
34
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Real-Time
Analytics
 Thousands of
Users
 Secure and
Available
Unified Big Data Environment
&
VS
35
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Oracle Big Data Solution
Decide
Apache
Flume
Cloudera
Hadoop
Applications
Oracle Event
Processing
Oracle BI
Enterprise Edition
Oracle
NoSQL
Database
Oracle R
Distribution
Stream
36
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
•Oracle
Complete
Big Data
Connectors
• Integrated
Oracle Data
Integrator
• Scalable
Endeca Information
Discovery
Oracle
Advanced
Analytics
Data
Warehouse
Oracle
Database
Acquire – Organize – Analyze
In-Database
Analytics
Oracle Real-Time
Decisions
37
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
38
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.