Transcript CSE880_Presentation_Cloud

ElasTraS:
Database for the Cloud
CSE880
Professor Pramanik
Shayne Rashedan & Chongguang Bi
1
Agenda
●
●
●
●
●
●
●
Background
Motivation and Challenges
Examples
Architecture
Schema Level Partitioning
Live Database Migration
Evaluation
2
Pay per use model
3
What is cloud computing
● Software as Service, Apps are located in Cloud, software
experiences are delivered through the internet
● Platform as Service, Internet based platforms for developers who
want to create services but don’t want to build their own cloud
● Infrastructure as Service, Cloud users rent storage, computation
and maintenance from cloud providers (pay per use utilities)
● Data as Service, Infrastructure for web-scale data mining and
knowledge discovery
4
Challenges
● Large numbers of application databases (or tenants) with
diverse schemas, unpredictable load and transactions.
● Scaling out using clusters of commodity servers and sharing
resources among tenants (multitenancy)
● Pay-Per-Use: paying a small subscription fee on per usage basis
every time the software is being used (e.g. hourly)
● Elastic Scaling: When the load increases, add more servers to the
database tier to distribute the load, and vice versa.
● Fault-tolerant, self-managing and highly available to support
mission critical applications
5
ElasTranS: Elastic, Scalable, Self-Managing
6
Examples:
● Key-value stores
- Bigtable, PNUTS and Dynamo, serving hundreds of thousands of concurrent
requests
● Schema Level Partitioning
- Microsoft SQL Azure ,Relational Cloud and a prototype from Yahoo!, standard
RDBMSs (such as SQL Server or MySQL)
● Decoupled shared storage
- Project Deuteronomy and Hyder, persistent data layer is separate from the
logic that executes operations on that data,on the other hand in Amazon EC2,
each node is data node and the transaction processor
7
Examples:
● Live data migration
- VM pages, iterative phase-by-phase copying pages, migrates live
processes and network connections using OS and networking primitives
- Zephyr, for shared nothing database architectures such as Cloud SQL
Server or Relational Cloud. The persistent data is stored in disks locally
attached to every node. Hence, the persistent data must also be migrated
● Advanced transactions at Scale,
- Megastore, Percolator, ecStore and G-Store
- Sinfonia, two phase commit (2PC) protocol to guarantee atomicity of
operations in a distributed set of nodes
8
Architecture of ElasTraS
Transaction
Manager
Server
Partitions of Data
Owning
Transaction
Manager
Row Groups
Append-only Multiversion Data
9
Architecture of ElasTraS
User / Client
Routing
Routing
Routing
Routing
OTM
Routing
Routing
OTM
OTM
OTM
Data Storage
Data Storage
Data Storage
Data Storage
TM Master
10
Static Partitioning
11
Online Multi-player Game
12
Row Groups
13
Row Groups
14
Row Groups = Partitions
15
Why TPC-C Tree Schema?
When amounts of data and transactions are large, assign some jobs to more nodes….
Assign all data and transactions related to Michigan State University Football Team to another node?
How??
16
Why TPC-C Tree Schema?
MSU
Spartan Stadium
UM
Michigan Stadium
MSU
Partition Keys
UM
MSU
Player 1
Information 1
MSU
Player 2
Information 2
UM
Player 3
Information 3
UM
Player 4
Information 4
MSU
Player 1
Game 1
Score
MSU
Player 1
Game 2
...
MSU
Player 2
Game 3
...
MSU
Player 2
Game 4
...
UM
Player 3
Game 5
...
UM
Player 3
Game 6
...
UM
Player 4
Game 7
...
UM
Player 4
Game 8
...
17
Snapshot & Multiversion Data
Problem 1
Distribute
Transactions?
Read
A
Manager
Data
Read
B
Problem 2
Read Lock?
Write Lock?
Read
Manager
Data
A
Write
B
18
Snapshot & Multiversion Data
Problem 1
Just Read Snapshot!
Read
A
OTM
Data
Read
B
Problem 2
Just Read Snapshot!
Read
OTM
Data
A
Write
B
While “A”s are reading,
B can update database.
19
Albatross Migration Timeline
Source Ownership
Steady State
1.Begin Migration
Destination Ownership
2.Iterative Copying
3.Atomic Handover
Steady State
Clients
TM Master
20
Challenges
● Minimum node unavailability achieved by iteratively copying database
cache and state of active transactions
● Elastic Load Balancing, rule-based model, if OTMs load is above the
threshold, new OTMs are added, if load is below a min-threshold, partitions
are consolidated into a smaller number of nodes.
● Partition assignment algorithm optimization problem with Greedy
Heuristic based on load statistics of partitions
● Atomic ownership handover, ensure efficient and ACID execution of
transactions, transactions must execute during migration, synchronizing
commit logs of two nodes
● Aborted transactions, minimize loss of data, restart aborted transactions in
destination
● Failure Handling, how to recover migration state after failure
21
Elastic Load Balancing
Data
Data
OTM
User
OTM
User
TM Master
User
OTM
Data
OTM
Data
22
Evaluation
Benchmarks:
Yahoo! Cloud Serving Benchmark
Key-value stores
3 tables, 10 VARCHAR(100) column per table
TPC-C Benchmark
9 tables describing wholesale supplier
5 Transactions: NewOrder, Payment, OrderStatus,
Delivery, StockLevel
23
Evaluation
Comparing with Albatross:
S&M: Stop and Migrate
stop serving, flush, transfer ownership
F&M: Flush and Migrate
flush, stop serving, transfer ownership
24
Elastic Load Balancing
Data
Data
Data
User
Data
Migration
OTM
User
OTM
User
TM Master
User
OTM
Data
OTM
Data
25
Evaluation
Albatross:
Low Unavailability
Low Latency
26
Evaluation
Albatross:
Almost no additional cost for read operations
Migration costs much time
27
Evaluation
Albatross:
Low Latency
Long Migration Time
28
Evaluation
Not Enough Scale
Scale is Large Enough
Load is Low
Migrate to Low Scale
29