Transcript Load Test Report

Load Test Report
<Project> <Release #>
<Date>
Copyright © 2011 by the Commonwealth of Pennsylvania. All Rights Reserved.
Contents
•
•
•
•
•
•
•
•
•
Project Overview
Release Application Changes
Release Performance Tuning
Load Test Approach
Load Test Methodology
Load Testing scenario enhancements
Load Test Comparison
SQL Execution Comparison
Batch Volume Testing
• Approach
• Methodology
• Test Comparisons
2
Project Overview
<Provide a description of the project and subsystems involved with this load
test >
3
Release Application Changes
<Provide a description of the changes that are part of the work orders in this
load test. A load test may include multiple initiatives so make sure to mention
all. Highlight, if any, functionality that may have a performance impact.
This information may come from communiques that are distributed to clients
post production.
List out major PCRs, if any, that are also part of the maintenance release>
4
Release Performance Tuning
Indexes applied
<List indexes that were added or modified due to performance improvements >
Query Tuning
 <SQL ID> - <Functional Area> <PCR #>
5
Load Test Approach
• Integrated load tests were conducted with:
--- --- <List of Applications and Versions> were tested with <count of
scripts> scripts representing the application – including: <list of
subsystems>.
--- --- <List of Applications and Versions>, <count of vUsers> vUsers were
calculated to represent 100% load for <subsystem; include more as needed
separated by commas>, 100% of the expected production application load
for <application> based on the average usage during a two hour time
period.
6
Load Testing Scenario Changes
The scripts below have been added to measure the
performance of the <functional area> of <application>:
•
•
<script name 1>
<script name 2>
7
Load Test Comparison Overview
• Column Descriptions
• Column I – <application> current production metrics at 100%
load prorated to 2 hrs.
• Column II – <application> Integrated average load test results 1
for <applications and versions>
• Column III – <application> Integrated average load test results 2
for <applications and versions>
• Column IV, V – <application> Integrated load test results with all
modules <applications and versions>
8
Load Test Comparison
<Application>
Production
Metrics
<Applications and
Versions>
Average for Last
Release 1
<Applications and
Versions>
Average for Last
Release 2 (if
needed)
<Applications and
Versions>
Baseline I
<Date>
<Time>
<Applications and
Versions>
Baseline II
<Date>
<Time>
Column
I
II
III
IV
V
Test Volume
-
<Application> –
100%
<Application> –
100%
<Application> – 100%
<Application> – 100%
# of Virtual Users
-
<vUsers>
<vUsers>
<vUsers>
<vUsers>
Total Passed Transactions
-
<Transactions>
<Transactions>
<Transactions>
<Transactions>
Total Failed Transactions
-
<Transactions>
<Transactions>
<Transactions>
<Transactions>
% Processor Time (Web
App Server)
-
<Server> - <Percentage>
<Server> - <Percentage>
% CPU Utilization
(Database Server)
-
<CPU>
<CPU>
<CPU>
<CPU>
Average Database
Connections
-
<DB Connections>
<DB Connections>
<DB Connections>
<DB Connections>
Average Throughput
(bytes/second)
-
<Thoroughput>
<Thoroughput>
<Thoroughput>
<Thoroughput>
Average Response Time
(<Project>)
-
<Response Time>
<Response Time>
<Response Time>
<Response Time>
<Server> - <Percentage> <Server> - <Percentage>
Load Test Comparison
<Application>
Production
Metrics
<Applications and
Versions>
Average for Last
Release 1
<Applications and
Versions>
Average for Last
Release 2 (if
needed)
<Applications and
Versions>
Baseline I
<Date>
<Time>
<Applications and
Versions>
Baseline II
<Date>
<Time>
Column
I
II
III
IV
V
Test Volume
-
100%
100%
100%
100%
Average Response Time
<Functional Area>
-
<Response Time>
<Response Time>
<Response Time>
<Response Time>
<Business Metric
Count>
<Business Metric
Count>
<Business Metric
<Business Metric Count> <Business Metric Count>
Count>
Business Metrics
<Functional Area>
**Add additional functional
areas as needed
10
<Project> SQL Execution Statistics
SQL_ID
Prorated
Value for 2
hrs
Load Test
<Date>
Execution
Count
Load Test
<Date>
Execution
Count
Load Test
<Date>
Execution
Count
Load Test
<Date>
Execution
Count
Load Test
<Date>
Execution
Count
Load Test
<Date>
Execution
Count
Gap in
Executions
11
SQL Execution Comparison
•
•
<Applications and Versions>
•
Percentage of top 10 queries covered during test: xx%
•
Percentage of the top 30 queries covered during test: xx%
Primary reasons for the coverage difference:
•
<Typical users may interact with the application in a more random
pattern, whereas step execution during load test mimics steps in
linear and repetitive fashion>
12
Batch Volume Testing Approach
•
Testing of modified batch:
•
<Describe the functional changes made to the batches>
•
Batches were executed through the <Application>
OpCon schedule while pointed to the Load database
•
Measure run times and server utilization of the updated code
run against Load DB.
•
Compare new code/Load database statistics against old
code/Production.
13
Batch Volume Test - I
<Date>
Job Name
Description
Run Time
(min)
Start Time
End Time
Average
DB
CPU%
Average
APP21
CPU%
14
Batch Volume Test - II
<Date>
Job Name
Description
Run Time
(min)
Start Time
End Time
Average
DB
CPU%
Average
APP21
CPU%
15