Transcript Franklin

The Three E’s of Big Data
and What DB People can do
About Them
Michael Franklin – UC Berkeley
Beckman Database Get Together
October 14, 2013
UC BERKELEY
The Big Data Problem - Nutshelled
omething’s
gotta
give:
Money
Time
Massive
Diverse
and
Growing
Data
Quality
2
The 3 E’s of Big Data:
xtreme lasticity
verywhere
Extreme Elasticity Machines
Option #1 – Build
your own Cluster/WSC
46K Servers
(2010 estimate)
Option #2 – Rent Machines from AWS
x Servers needed
Option #3 – Try your luck on the Spot
Market
x Servers needed
(US East –
Saturday Sept 28
Extreme Elasticity Algorithms
Agarwal et al., BlinkDB: Queries with Bounded Errors and Bounded Response Times
on Very Large Data. ACM EuroSys 2013.
Extreme Elasticity - People
Incentives
Fatigue, Fraud, & other Failure
Modes
Latency & Prediction
Work Conditions
Interface <-> Answer Quality
Task Structuring
Task Routing
6
Extreme Elasticity
• Approximate Answers
• ML Libraries and Ensemble Methods
Algorithms • Active Learning
• Cloud Computing – esp. Spot Instances
• Multi-tenancy
Machines • Relaxed (eventual) consistency/ Multi-version methods
People
• Dynamic Task and Microtask Marketplaces
• Visual analytics
• Manipulative interfaces and mixed mode operation
The Challenge
Extreme Elasticity
+
Tradeoffs
+
Integration
= Extreme Complexity
The Good News: We
already know how to do this
(kinda)!
End Users tell the system
what they want, not how to get it
✦
SQL
Result
MQL
Model
MLbase: Progress
MQL Parser
ML Library
ML Developer
API
Release
d
(Contracts)
Query
Planner /
Optimizer
Runtime
initial
release:
Spring
2014
For More Information
UC BERKELEY
amplab.cs.berkeley.
edu
[email protected]
du