Transcript Slide Presentation

A Peer-to-Peer Database Server based on BitTorrent
John Colquhoun
Paul Watson
www.neresc.ac.uk
Introduction
• When a database server receives queries faster than it can process
them, performance becomes unacceptable
• Similar problems solved for file-sharing by the use of P2P
technologies
• Can we utilise the combined CPU, memory, network and disk of
individual clients to reduce the server load?
• We examine how P2P techniques could be applied to databases
and introduce the Wigan P2P database, derived from the BitTorrent
file-sharing protocol
• Potential applications in e-Science and e-Commerce
www.neresc.ac.uk
2
System Architecture
Advertise: SELECT *
FROM t
Query: SELECT id, value
FROM t
WHERE t.Id < 100
Advertise: SELECT id, value
FROM t
WHERE t.Id < 100
Tracker
Query: SELECT id, value
FROM t
WHERE t.id < 100
Advertise: SELECT id, value
FROM t
WHERE t.id < 100
Advertise:
SELECT
id, value
Query:
SELECT
id, value
FROMt t
FROM
WHEREt.Id
t.Id<<10
10
WHERE
www.neresc.ac.uk
3
Initial Implementation
• A simulator of Wigan
• The TPC-H benchmark database was used to evaluate
the design
• Identified cases where Wigan offered a performance
advantage over a Client-Server database and those
areas where it did not
• Showed Wigan can outperform client-server DB when:
• There are enough peers to reduce the load on the seed
• There is sufficient overlap between the queries
• The system is sufficiently busy so a traditional database server is
overloaded
www.neresc.ac.uk
4
Live Wigan system
• Used algorithms developed for the simulator
• Written in Java and uses OGSA-DAI
• Also uses the TPC-H benchmark database
www.neresc.ac.uk
5
Live System Experiments
• Experiments – running the 15 TPC-H queries that MySQL
can process
• Five peers starting at 10 second intervals
• Of these, Wigan could answer eight
• Remainder require more advanced SQL features
• The remaining queries were sent to the seed in Wigan
• Three different scenarios
• Standard Client-Server (accessing MySQL via JDBC)
• Wigan
• Wigan in a warm-cache scenario (to see how Wigan would behave if multiple
peers have been available for some time)
www.neresc.ac.uk
6
Live System Results
600
Average response time (s)
500
400
300
200
100
0
Client-Server
Wigan With No Warming
Wigan With Warming
Method
www.neresc.ac.uk
7
Live System Results
• Client-Server is quite slow because the queries are
complex, involving many joins and aggregations
• Wigan in a cold-cache scenario does not offer a
performance advantage
• Only the seed is able to provide data
• Peers have to send all requests to the seed, hence queues form there
• Wigan in a warm-cache scenario outperforms ClientServer
• Other peer (s) able to provide the results, so reducing the load on the seed
• Peer (s) with the query results do not have so much data to scan or complex joins
to perform
• Validates results from the simulator regarding when
Wigan does and does not perform well
www.neresc.ac.uk
8
Future Work
• Testing Wigan with “real” e-Science data from existing
projects at Newcastle University
• Connect Wigan to an open-source implementation of
Amazon’s SimpleDB to explore the effect of the Cloud
and updates
• Example of usage in both three-tier architecture and an
e-Science application
www.neresc.ac.uk
9
Summary
• We designed, implemented & evaluated the Wigan P2P
Database System
• Derived from the popular BitTorrent file-sharing protocol
• The first database server that uses P2P to scale over
multiple peers
• Showed P2P databases can scale to outperform
traditional Client-Server databases
www.neresc.ac.uk
10