Transcript Database Tuning Principles, Experiments and Troubleshooting

Database Administration and
Performance Tuning
--Introduction
1
Course Structure
• Lecturers:
–
–
–
–
Dr. Yanghua Xiao (肖仰华)
Email: [email protected]
Mobile phone: 13918452801
http://gdm.fudan.edu.cn/GDMWiki/Wiki.jsp?pa
ge=Dbtune
2
Goals of the Course
• Appreciation of DBMS architecture
• Study the effect of various components on the
performance of the systems
• Tune your application or system built upon DBMS
• Performance criteria for choosing a DBMS
• Tuning principles
• Troubleshooting techniques for chasing down
performance problems
3
Text/Reference Books:
•
•
•
•
•
•
•
•
•
•
Dennis Shasha and Phillipe Bonnet: Database Tuning : Principles Experiments
and Troubleshooting Techniques. Morgan Kaufmann Publishers. 2002
(released in June 2002). TEXT.
Dennis Shasha: Database tuning : a principled approach. Prentice Hall, 1992.
REFERENCE (a good reference if cannot get the text book)
Database Management Systems, 3rd edition. Raghu Ramakrishnan & Johannes
Gehrke, McGraw-Hill, 2002.
Hector Garcia-Molina, Jeffrey D. Ullman, and Jennifer Widom: Database
Systems -- The Complete Book. Prentice Hall, 2001.
Sitansu S.Mittra, Database Performance Tuning and Optimization,
Springer,2003
G. J. Vaidyanatha, K. Deshpande and J. Kostelac: Oracle Performance Tuning
101. Osborne/Mc-Graw-Hill. 2001. REFERENCE.
Jim Gray (ed): The Benchmark handbook : for database and transaction
processing systems. M. Kaufmann Publishers, 1991. REFERENCE.
Richard J.Niemiec, Oracle Database 10g Performance Tuning tips and
techniques, Mc Graw Hill Education,2009
牛新庄, DB2 数据库性能调整与优化, 清华大学出版社, 2009
胡百敬,等, SQL Server 2005 Performance tuning, 电子工业出版社，2008
4
Copyright:
Many slides belong to the tutorial:
Database Tuning
Principles, Experiments and Troubleshooting Techniques
Dennis Shasha ([email protected])
Philippe Bonnet ([email protected])
And lecture notes provided by
Database Management Systems, 3rd edition.
Raghu Ramakrishnan & Johannes Gehrke
McGraw-Hill, 2002.
and some from the web …
6
What is Database Tuning
• Database Tuning is the activity of making a
database application run more quickly.
• “More quickly”
– usually means higher throughput,
– though it may mean shorter response time for
time-critical applications
• A 5% improvement is significant.
7
Why Database Tuning?
• Troubleshooting:
– Make managers and users happy given an
application and a DBMS
• Capacity Sizing:
– Buy the right DBMS given application
requirements
• Application Programming:
– Coding your application for performance
Viewpoint about DB tuning
• Easy
– Need not to struggle with complicated formulas and
theorems
• Difficult
– Need broad and deep understanding about
•
•
•
•
DBMS
Application
OS
hardware
9
Why is Database Tuning hard?
sql
commands
PARSER
OPTIMIZER
EXECUTION
SUBSYSTEM
DISK
SYBSYSTEM
CACHE
MANAGER
LOCKING
SUBSYSTEM
The following query
runs too slowly
select *
from R
where R.a > 5;
LOGGING
SUBSYSTEM
What do you do?
MEMORY
CPU
DISK/
CONTROLLER
NETWORK
Where to tune?-physical
11
Where to tune? -logical
12
Optimization at each level
• Conceptual level
– E.g. normalization vs denormailization
• Internal level
– E.g. tablespace management, index strategy
• External level
– E.g. Query optimization, optimal access plan
13
How to tune?
process
• Process for DB tuning
– Localize the problem by starting at the location
where the problem surfaces for the first time
and trace it to the root
– Fix the root cause and test to ensure that the
problem does not reappear
– Look for any adverse side effects caused by the
fix
14
How to tune?
Troubleshooting Methodology
Troubleshooting Methodology:
– Troubleshooting (what is happening?)
– Hypothesis formulation
•
•
What is the cause of the problem?
Apply tuning principles to propose a fix
– Hypothesis verification (experiments)
15
Tuning Principles
• Think globally, fix locally
• Partitioning breaks bottlenecks (temporal
and spatial)
• Start-up costs are high; running costs are
low (disk transfer, cursors)
• Render unto server what is due unto server
• Be prepared for trade-offs (indexes and
inserts)
16
Think globally, fix locally
• Think globally to find the root case or best
solution for bottlenecks
– For example : High disk I/O may be caused by:
•
•
•
•
No index resulting into table scan
Log and data file reside in the same disk
Fail to increase the db buffer
…..
• Fix locally to minimize the adverse side-effect
– E.g. Optimize the critical query
17
Partitioning breaks bottlenecks
• Bottleneck: the ONE part of the system limits the overall
performance
• Partitioning
• Reducing the load on bottlenecks either by dividing the load over
more resources or by spreading the load over time
• Partitioning in space (physical resource)
– E.g. distribute account data to each branch
• Partitioning in logical space
– E.g. lock contention on free list will cause the free list to be a bottleneck
– Divide the free list into several pieces, each thread only lock one free list
• Partitioning in time
– Long transactions may lock resource for a long time, use large portion of
buffers, slowing down short transactions
– Serialize long transactions when there are little short transactions
18
Side-effect of partitioning
• Suppose we need to query data across branch of a
bank
• Merging data causes extra communication cost
• Lesson
– When you find a bottleneck, first try to speed up that
component; if that doesn’t work, then partition
Tuning Principles
• Start-up costs are high; running costs are low
– Start-up costs include
• Disk access
– Reading 64k segment from disk is less than twice as expensive as reading 0.5k
– Frequently scanned tables should be consecutively laid out on disk
– Vertical partitioning tables with hundreds of columns
• Data transfer
– Latency of sending message dominate the whole costs
• Packing data and send large chunk of data rather than small ones
• Query processing
– Compile often executed queries
• System calls
– Opening a call is expensive
– Cache connections
– Reduce the number of start-ups
20
An example:
Time =
Rule of
Thumb
Seek Time +
Rotational Delay +
Transfer Time +
Other
Random I/O: Expensive
Sequential I/O: Much less
• Ex:1 KB Block
» Random I/O:  20 ms.
» Sequential I/O:  1 ms.
21
Render onto server what is due
onto Server
• Task allocation between the server and the application
programs
– Factors:
• Relative computing resources and load of client,
application servers and data server
– Should checking be done in the middle tier?
• Where information is located?
– Trigger costs less cost than Polling
• Whether the database task involves the user interaction?
– Interaction with screen or user takes long time
– Separate the user interaction from a long transaction
22
Be prepared for trade-offs
• Performance vs cost/resource
– Adding Buffer size need more RAM
•
•
•
•
Query performance vs storage cost for index
Query performance vs Update performance
……
Keep in mind
– You want speed, how much are you willing to pay?
23
Tuning Mindset
1.
2.
3.
4.
5.
Set reasonable performance tuning goals
Measure and document current performance
Identify current system performance bottleneck
Identify current OS bottleneck
Tune the required components eg: application,
DB, I/O, contention, OS etc
6. Track and exercise change-control procedures
7. Measure and document current performance
8. Repeat step 3 through 7 until the goal is met
24
Date
Subject
Literature
1
Sep 14
Introduction (slides
2
Sep. 21
Schema Refinement(slides)
RG19-20,SP4.2
3
Sep. 28
Data Storage(slides)
HG 11.1-11.5
4
Sep.30
Storage Refinement(slides)
SP 2.5, HG 11.611.7
5
Oct. 7
Disk Organization(slides)
HG 12
6
Oct.14
Index Tuning-Convential Index(slides)
HG 13.1-13.2
7
Oct.21
Index Tuning-B+tree(slides)
RG 10, HG 13.3
8
Oct.28
External Sorting(slides),Index Tuning-Hash
Index(slides), Performance Tuning(slides)
HG13.4,RG 11, SP
3
9
Nov. 4
Operator Evaluation(slides)
RG14
10
Nov. 11
Query Optimization(slides)
RG15
11
Nov. 18
Query Tuning(slides)
SP 4.6-4.7
12
Nov. 25
Overview of Transaction Management(slides), Lock
Tuning(slides)
HG 16-17, SP 2.2
13
Dec. 2
Log Tuning(slides)
HG 18, SP 2.3
14
Dec.9
Troubleshooting(slides)
SB 7
)
Materia
(RG 1.5-1.8, RG
4.2); SP 1
database des
Reading and
Programming
DB Tuning Pr
2, Example 3