Transcript Cloud Computing: Why it Will Change Your Life and Why You

What Should the Design of CloudBased (Transactional) Database
Systems Look Like?
Daniel Abadi
Yale University
March 17th, 2011
Does the Cloud Force Us to Build
New Database Systems?
Traditional vendors argue ‘no’
But there is increased desire for certain things:
– Horizontal scalability to leverage cloud elasticity
Traditional solutions use high-end SANs, but this is not a
cloudy concept
Shared-nothing is the most effective way to achieve
horizontal scalability in the cloud
– Virtualization can result in wild fluctuations of node
performance
Do not want to operate at the speed of slowest node
– Virtual machines in the cloud have orders of
magnitudes higher mortality rates
Greedy owners like to murder the poor VMs
Fault tolerance must be treated as a first class citizen
The Problem With Traditional
Database Solutions
Available shared-nothing solutions do not
achieve high transactional throughput
– Distributed concurrency control and commit protocols
are expensive
Need more research to reduce this overhead
Database systems generally optimize everything
in advance
– Adaptive execution and optimization frameworks must
be high on the research agenda
Database systems treat faults as a rare event
– Machine failures cause transactions to abort
– Recovery from a REDO log is slow
Common Solutions
Drop A or C of ACID
– Relaxing consistency makes replication easy,
facilitates fault tolerance
– Relaxing atomicity reduces (or eliminates) need for
distributed concurrency control
Examples: SimpleDB, BigTable (HBase),
Cassandra, PNUTs, SQL Azure, sharded
MySQL, etc.
– Often called NoSQL systems
(Dropping ‘C’ also helps with CAP, but this is
only part of the story)
Whither ACID in the Cloud?
People still want ACID
– Engineers at Google, Facebook, Amazon,
Twitter, etc. are a very loud minority
– NoSQL should not be the only option in the
cloud
Needed research:
– Building an ACID-compliant, horizontally
scalable, fault tolerant database for the cloud
One Potential Idea
Get replication to work right out of the box
– Today’s systems generally act, then replicate
Complicates semantics of sending read queries to
replicas
Need confirmation from replica before commit
(increased latency) if you want durability and high
availability
In progress transactions must be aborted upon a
master failure
– Want system that replicates then acts
Therefore …
Instead of weakening ACID, strengthen it!
– Guaranteeing equivalence to SOME serial
order makes active replication difficult
Running the same set of xacts on two different
replicas might cause replicas to diverge
Disallow any nondeterministic behavior
Disallow aborts caused by DBMS
– Disallow deadlock
– Distributed commit much easier if you don’t
have to worry about aborts
Consequences of Determinism
Replicas produce the same output, given the same input,
– Facilitates active replication
Only initial input needs to be logged, state at failure can
be reconstructed from this input log (or from a replica)
Active distributed xacts not aborted upon node failure
– Greatly reduces (or eliminates) cost of distributed commit
Don’t have to worry about nodes failing during commit protocol
Don’t have to worry about affects of transaction making it to disk
before promising to commit transaction
Just need one message from any node that potentially can
deterministically abort the xact
This message can be sent in the middle of the xact, as soon as it
knows it will commit
If This Works Then …
Node failure does not cause transaction
failure
– Fault tolerance will be extremely high
Can run distributed transactions without an
expensive commit protocol
– Shared-nothing becomes much more
attractive