Transcript CON1446_Huang-CON1446 - Oracle NoSQL Database a..

1
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Oracle NoSQL Database and
Big Data
Chao Huang
Senior Manager, Oracle NoSQL
Database development
2
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Relational Era
1970s - OLTP
 Relational Databases replace IMS
for ‘Business Data processing’
 Relational Databases become the
Replaced By
standard for OLTP
 SQL is better
 Query Optimizers beat programmers
3
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Edgar F. Codd
Big Data Era - Volume
What’s wrong with Sharding
 Unhappy Developers
 More code to write
 Unhappy DBAs
 Complicates backup, schema
changes, indexing, hotspots
 Unhappy users
 No cross-shard applications
4
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Big Data Era - Variety
What’s wrong with my Data
 More and more semi-structured
data sources
 Consumers are talking and so are
sensors
 Schema-less is the critical need for
agility
5
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Big Data Era - Simplicity
Simple Data Models
K-V
Logs, Sensors
6
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Real-time Ads
User Profiles
Trends Driving NoSQL Technology
Why NoSQL
 Horizontal Scale
–
Data Volume requires scaling on multiple machines
 Leading Technology giants
–
have embraced NoSQL - Google, Amazon, Facebook et al.
 New Kinds of Data
–
more semi-structured and unstructured data sources.
 New Speeds of Data
–
real time customer experience, real time sensor feedbacks
 Cost Sensitivity
–
7
Scale is not a yes/no question. Scale is an ROI question.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
What is Oracle NoSQL Database
Scalable, HA, Key-Value








8
Simple Key-Value Data Model
Horizontally Scalable
Highly Available
Simple administration
ACID Transactions at scale
Transparent load balancing
Elastic Configuration
Commercial grade software and
support
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Performance
YCSB
• 1.25M ops/sec
• 2 billion records
• 2 TB of data
• 95% read, 5% update
• Low latency
• High Scalability
9
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
NoSQL Use Cases
10
Web-Scale
Transaction Processing
Last-Mile
Content Delivery
Real-Time
Event Processing
High velocity, volume,
variety, low information
density data capture
Guaranteed low latency
lookups for end-customers
Real time events trigger rule
that perform low latency lookups
Web browsing, Shopping
Carts, CDR processing,
Sensor data capture
Advertising, Product
Recommendations, Online
Catalogs, Social Media
Medical Monitoring, Factory
Automation, Oil & Gas, Geolocation
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Use Case – Web Scale Transactions
Oracle NoSQL DB
 NoSQL DB for High velocity, High
Application
volume, High variety, Low
information density ACID
transactional data capture
 Hadoop and Data Warehouse for
analytics
 Applications: Web browsing, Web
Retail, CDR processing, Sensor
data capture
NoSQL DB Driver
Enterprise
Data Warehouse
Hadoop Cluster
Acquire
11
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Organize
Analyze
Use Case – Last Mile Content Delivery
Oracle NoSQL DB
Business Users
End Customers
 Platform for real-time content
delivery
 Content & market segmentation
Acquired and Analyzed in Hadoop &
RDBMS
 NoSQL provides low latency content
lookup and delivery to endcustomers
 Applications: Advertising, Marketing
segmentation, Location-aware
content delivery
Multi-Reporting
Dimensional
Application
NoSQL DB Driver
Hadoop Cluster
Acquire, Analyze, Prepare
12
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Content Delivery
Use Case – Real Time Event Processing
Oracle NoSQL DB
Oracle Event Processor
 Real time events trigger rule
execution in Oracle Event
Processor
 OEP rules perform low latency
lookups to Oracle NoSQL DB for
additional data
 Applications: Utilities, Medical
Monitoring, Factory Automation,
Oil & Gas Exploration, Geolocation Tracking
13
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Rule Action
Low Latency
Lookups
NoSQL DB Driver
NoSQL for Social Online Betting
Challenge
Objectives
 Scalable in-play sports betting platform
 Increase new business revenue
 Improve operational efficiency
 “Oracle NoSQL Database enabled the rapid,
scalable processing of incoming XML, ensuring
high available and guaranteed event ordering.”
James Anthony
Chief Technology Officer
Passoker
Solution
Solution
 Match in-play bets with incoming events
 Promote interaction between customers
 Scale system with customers and events
Real-Time, In-Play Sports Betting
XML
App
Providers
 Feeds MySQL database for revenue tracking
and operational reporting
14
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Customers
Event Capture
& Store
Accounting &
Operations
NoSQL DB
MySQL
NoSQL for Scalable PaaS
Challenge
Benefits
 Provide special purpose application server
 Low latency application object persistence
services to financial institutions
 Provide cost competitive subscriptions
 Flexible data format and serialization techniques
 Highly reliable data store that can scale as the number
of hosted apps and app objects grow
Solution
 Oracle NoSQL database for unstructured data
capture and application object persistence
 Oracle database for business analytics and
insight into the data collected
15
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Application
NoSQL DB Driver
NoSQL for Oracle Communications Mgmt
Challenge
Benefits
 Improve billing and revenue management
 Extend Coherence data caching to disk
 Calculate charges for any service combo
 Manage growth in data volume, 400M customers
 Provide scalable CDR processing
 Handle extreme TXN volumes with low latency
 Coherence cluster for real time event rating
 NoSQL database for rated event persistence and
consumption by downstream systems
 Coherence memory optimization using NoSQL
database for out of band data
16
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
 Always on and highly available
OCM Rated Event s Processor
Solution
Application Data Ingestion
NoSQL DB Driver
Compatibility
Oracle NoSQL
 Supported OS – Linux and
Solaris 10
 Oracle NoSQL DB is written in
Java
 Java and C APIs
 Bindings available for Jython,
Jruby, Clojure, Groovy, Rhino
17
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Features - Sharding
Auto-Sharding
Hash Fn
 Provides linear scale-out of write ops/sec
 No need to develop sharding logic in
Partitions
Application
 Intelligent Driver – aware of hash
…
Shard 0
function and shard location
Shard 11
 Hash function to map a key to a partition
 Each partition is routed to a single shard
18
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Rep
Node
Master
Rep
Node
Replic
a
Rep
Node
Replic
a
Rep
Node
Master
Rep
Node
Replic
a
Rep
Node
Replic
a
Features - Replication
High Availability
 Master-Slave replication
 Supports Heterogeneous platform
hardware/OS/JVM
 Provides linear scale out for read
ops/sec
19
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Shard 0
Rep Node
Master
Rep Node
Replica
Rep Node
Replica
Features – Configurable CAP
Greater Flexibility
 Configurable Durability per operation
 Configurable Consistency per
operation
 ACID by default
 Transaction scope is single API call
 Records share same major key
 Multiple operations supported
20
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Features - Failover
Automatic Failover
Replication factor = 5
 Automatic election of new Master
 Rejoining nodes automatically
Rep Node
Replica
Rep Node
Master
Rep Node
Replica
synchronize with the Master
Rep Node
Replica
 Isolated nodes can still service reads
 All nodes are symmetric
21
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
New Master
Rep Node
Replica
Enterprise Integration – Extensions
 Integration with Oracle Database – External Tables
 RDF Graph for NoSQL – Support for SPARQL
 Event Processing with Oracle Event Processor
 Integration with Hadoop
22
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
Why Oracle NoSQL Database
 It’s Oracle
 A trusted vendor here for the long term
 Scalable, Available with Predictable Latency
 Differentiating Features
 Always-On Elastic Processing
 Configurable ACID Transactions at scale
 Easy to use Smart Data Center Topologies
 Integration with the Oracle technology stack
23
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
24
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.
25
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.