Transcript Scaling MySQL in the Cloud Presentation

ScaleDB
Transactional Shared Disk storage engine for MySQL
Moshe Shadmon, Founder, CTO
The product
DBMS/Apps
MySQL-ScaleDB Interface
Storage Engine
Storage Layer
ScaleDB Benefits
 Enables open source DBMS (eg: MySQL) to operate as
clustered, high end database
 Scales applications in cloud environments
 No need to change the applications
 Provides high performance
 Provides unified data store
Shared Disk vs. Shared Nothing
Masters
Slaves
Shared Nothing
Shared Disk
SQL Server
DB2 (Unix)
MySQL
PostgreSQL
Oracle RAC
IBM - DB2, IMS (MF)
ScaleDB
Shared Disk vs Shared Nothing
Shared Nothing
Shared Disk
Storage
Per DBMS machine
Shared storage (NAS/SAN)
Scalability
By data partitioning
By adding CPU
Scaling complexity High
Low
High Availability
Master-Slave
Multi-Masters
HA complexity
High
Low
Implemented By
MySQL
PostgreSQL
SQL Server
DB2 (Unix)
Oracle RAC
IBM on mainframe (DB2/IMS)
ScaleDB
Shared disk provides “out of the box” scalability and availability
Currently available with the high end commercial databases only
The Virtualized Cloud Database
OSS DBMS
Storage Engine
Server 2
Server 1
MySQL Server
VM
VM
VM
VM
VM
OSS OSS OSS
DBMSDBMS
DBMS
OSS OSS
DBMSDBMS
ScaleDB ScaleDB ScaleDB
ScaleDB ScaleDB
Local Disk
Shared Nothing
Shared Storage
Shared Disk
Deploying ScaleDB
Application Layer
Database Layer
(Physical or VM nodes)
ScaleDB
Application
Node 1
Node 2
DBMS
DBMS
ScaleDB
ScaleDB
Node N
…
ScaleDB
ScaleDB Cluster Manager
Standby Cluster Manager
Storage Layer
Shared Storage
DBMS
Shared Storage
Benefits of ScaleDB Technology
 High Availability – No single point of failure
 If a node fails – a different node takes over
 If the cluster manager fails – the standby cluster manager is
initiated to manage the cluster
 Scalability
 No limit on the number of nodes that can join the cluster
 Algorithms to minimize the need to communicate
 Performance
 Innovative, high performance, general purpose indexing
ScaleDB’s Internal Architecture
Applications
ScaleDB
Cluster Manager
Global Buffer
Manager
Global Lock
Manager
Global Recovery
Manager
XML
ScaleDB API
Transaction Manager
Lock Manager
Local Lock
Manager
Global Lock
Coordinator
Index
Manager
Data
Manager
Buffer Manager
Log Manager
Recovery
Manager
Storage
Manager
File System
Storage Devices
Storage Devices
ScaleDB
Node
Distributed Lock Manager
ScaleDB
Cluster Manager
H List
DBMS X
H List
Table Y
H List
Block Z
W List
ScaleDB
Node
Node 1
R/W
Node 2
R/W
Read Customer
Info
Node 1
R/W
Node 2
R/W
Index translates
logical id to Row ID
Node 1
R
Node 2
R
Shared
DBMS lock
Shared
Table lock
Shared
Row lock
Node 2
R/W
File System
Storage Devices
Storage Devices
Shared Disk in the Cloud
 Cloud provides Scalability and HA
 Scalability - Enables dynamic allocation of computing and
storage
 Availability – Multiple servers over the same physical data
 High performance
 Reduces Management Overhead
 Reduced tuning and partitioning support
 No manual slave promotion or replication
Performance / Scalability
 Benchmarks
 Join operations using Multi-Table index
 Sysbench
 TPCC
 Example multiple MySQL + ScaleDB nodes on Amazon EC2
 Demo
Benchmark multi-table index
Queries/Sec
60
50
40
30
20
10
0
Engine X Join
ScaleDB MTI
ScaleDB 2 Nodes
Sysbench Performance on EC2
2303 TPS
Transactions Per Second
2033
1764
1494
Other Engine
1224
ScaleDB
954
775 TPS
685
415
1
1
2
3
4
5
6
Number of Nodes
7
8
Current & Future Work
 Optimizing TPCC – soon to be published
 Integrating with other environments
 GA – later this year
 Demo