Transcript Proposal Presentation. - Computer Science

BENCHMARKING
DATABASES
By Samy Kabangu
Supervisor : Mr. John Ebden
Computer Science Department
Rhodes University
BACKGROUND
DATABASE WARS!


There is a high competition in the Database computing
industry.
Spectacular claims are made by different vendors in
favour of their products in terms of performance.
Here is one of the claims published in
a white paper by Microsoft.
“ Microsoft SQL Server has steadily gained ground on other
database systems and now surpasses the competition in
terms of performance, scalability, security, developer
productivity, business intelligence (BI), and compatibility with
the 2007 Microsoft Office System. It achieves this at a
considerably lower cost than does Oracle Database 11g.”
Microsoft, 2009
The question is: “should the public, database
professionals, database users and academics
believe in such claims ?”
Industry Standard Benchmarks



Benchmarking a database is the process of
performing well defined tests on that particular
database in order to evaluate its performance.
Industry standard database benchmarks provide a
platform for benchmarking various database
products.
The obtained results are used to compare and rank
database products.
Some of the Issues with the Industry
Standard Benchmarks





Though they provide means of knowing the performance of
database products, their adoption might be a little difficult.
They are performed on specific hardware platform and operating
system.
They simulate real world workload that might not reflect the actual
workload of a specific organisation database system.
Some vendors use techniques such as caching the data and SQL
statement into memory(RAM) in order to avoid disk access so as to
improve performance
Vendors cite the benchmarks tests results that favour their product.
STATEMENT OF THE PROBLEM

Our objective is to come up with a best way of testing
a database management system on several different
hardware and software platforms. The variables are:

64 bit + quad processor @ 2.66GHz each + 2.96G
RAM with 32 bit and then 64 bit software,

32 bit P4 @ 2.8GHz 1 GB RAM with 32 bit software,

An investigation into to the most critical
benchmarking techniques and factors in tuning
database management systems will be the main focus
of this project.
INTENDED APPROACH
 We will firstly provide ourselves with a sound
understanding of the factors that influence the
performance of a database management system.
 Which will enable us in the investigation of
some of the well known industry standard
database benchmarks such as the TPC.
 Run some TPC benchmark tests on SQL
Server 2008.
EXPECTED RESULTS



Implementation of sound practice techniques in
tuning a database in order to obtain an
optimum performance from it.
To be able to answer the question:”Why using
the TPC Benchmarks is better than using other
industry standard benchmarks?”.
Observe the performance of Microsoft SQL
server 2008 on a core 2 Quad processor if it will
match the one published by the TPC-H run on
mainframe type hardware?
EXPECTED RESULTS



Can new software (Microsoft SQL Server
2008 – 32 and 64 bit versions) being run on
slow hardware match the performance of
old software (Microsoft SQL Server 2005)
run on fast hardware?
The primary rigging database methods used
by vendors to increase the performance of
their products.
Can SQL Server “beat” Oracle (as per white
paper mentioned above).
FUTURE EXTENSION
This project can be extended to the
design of database benchmarks.
QUESTIONS?