Transcript A Case Study of Web Server Benchmarking

A Case Study of
Web Server Benchmarking
Using
Parallel WAN Emulation
Carey Williamson
Rob Simmonds
Martin Arlitt
University of Calgary
Network Emulation




A hybrid performance evaluation
methodology that combines aspects of
implementation with simulation modeling
A network emulator is a network simulator
with an interface that allows client
applications to interact with it in real-time
“A simulator that talks back” (IP packets)
Why? Provides a reliable, repeatable test
environment for distributed applications

Internet games, video conferencing, Web, ...
Web Benchmarking with IP-TNE
2
Simulation
Real World
IP-TNE


The Internet Protocol Traffic and
Network Emulator (IP-TNE) is a
network emulator based on IP-TN
(packet-level IP network simulator)
Enables interaction between IP
based clients via an IP-TN
simulated network (in real time!)

Distributed applications can interact
with IP-TNE without modification
Web Benchmarking with IP-TNE
4
IP-TNE Overview
Web Benchmarking with IP-TNE
5
IP-TNE Overview (cont’d)



CCTKit with real-time extensions
provides an environment for fast
network emulation (PDES)
IP-TNE provides routing methods
suitable for shared environments and
dedicated test environments
Now has HTTP and TCP client models
that can be used for Web server
benchmarking
Web Benchmarking with IP-TNE
6
So What?




Flexible routing model support
High-performance packet reading and
writing via raw sockets (1 Gbps)
Can model an arbitrary IP internetwork
Detailed IP protocol models



IPv4, ICMP, ping, traceroute, pchar, MTU, ...
Supports parallel execution on
shared memory multiprocessors
“Blazingly fast!” - CLW, 2002
Web Benchmarking with IP-TNE
7
Example: Web Benchmarking
Client 1
Client 2
...
Web Server
Client 3
Client C
Web Benchmarking with IP-TNE
8
WAN Emulation (1 of 3)
Client 1
Client 2
“Centralized” Approach
Web Benchmarking with IP-TNE
...
Web Server
Client 3
Client C
9
WAN Emulation (2 of 3)
Client 1
Client 2
“Shim” Approach
(NISTnet, DummyNet, WASP)
Web Benchmarking with IP-TNE
...
Web Server
Client 3
Client C
10
WAN Emulation (3 of 3)
Client 1
Client 2
Our IP-TNE Approach
Web Benchmarking with IP-TNE
...
Web Server
Client 3
Client C
11
Objectives of Case Study




Evaluate new approach to WAN
emulation, and demonstrate feasibility
Confirm prior results by Nahum et al.
on effects of WAN conditions on
Web server performance
How fast can Apache Web server go?
How fast can IP-TNE go?
Web Benchmarking with IP-TNE
12
Experimental Setup

IP-TNE on Compaq ES-40 (4 CPU)
Apache (1.3.23) on another ES-40
Gigabit Ethernet (1 Gbps) in between
OS is Compaq Tru64 (v5.1A)

ANML for defining network model




e.g., simple regular WAN topology
Web Benchmarking with IP-TNE
13
Web Benchmarking with IP-TNE
14
Web Benchmarking with IP-TNE
15
Web Workload Model



Static content only
Closed-loop workload generator
Fixed-size Web objects



Small (1 KB)
Large (64 KB)
Variable-size Web objects




Median 3 KB
Mean 9 KB
Pareto heavy tail (alpha = 1.2)
Zipf-like document popularity profile
Web Benchmarking with IP-TNE
16
Performance Metrics

Two primary metrics



HTTP transaction rate (trans/sec)
Network throughput (Mbps)
Several secondary metrics




Response time
Connection failure rate
Packet loss rate
...
Web Benchmarking with IP-TNE
17
Results (Fig. 4a)

For 1 KB transfers with HTTP/1.0:






Single client: 170 transactions/sec
Transaction rate scales up with number of
clients up to about H = 32
Transaction rate flattens, then drops sharply
as num clients is increased more (closed loop)
Peak rate achieved: 3800 trans/sec
Peak throughput approximately 40 Mbps
Transaction rate is (strongly) inversely related
to the client round trip time (RTT)
Web Benchmarking with IP-TNE
18
Web Benchmarking with IP-TNE
19
Web Benchmarking with IP-TNE
20
Web Benchmarking with IP-TNE
21
Web Benchmarking with IP-TNE
22
Web Benchmarking with IP-TNE
23
Results (Fig. 4b)

For 64 KB transfers with HTTP/1.0:






Single client: 18 transactions/sec
Transaction rate scales up with number of
clients up to about H = 32
Transaction rate flattens, then drops slightly
as num clients is increased more
Peak rate achieved: 220 trans/sec
Peak throughput approximately 115 Mbps
Transaction rate is (weakly) inversely related
to the client round trip time (RTT)
Web Benchmarking with IP-TNE
24
Web Benchmarking with IP-TNE
25
Results (Fig. 4c)

For variable-size transfers with HTTP/1.0:







Single client: 60 transactions/sec
Transaction rate scales up with number of
clients up to about H = 32
Transaction rate flattens, then drops
as num clients is increased more
Peak rate achieved: 1300 trans/sec
Peak throughput approximately 90 Mbps
Transaction rate is inversely related to the
client round trip time (RTT)
Behaviour is in between 1 KB and 64 KB results
Web Benchmarking with IP-TNE
26
Web Benchmarking with IP-TNE
27
Results (Fig. 5a)

Concurrent connections with
HTTP/1.0:



Single client: 600 transactions/sec
Qualitatively similar results to before,
except that fewer clients are needed to
drive the server to full load
Conceptually concurrent connections are
no different than adding more clients
Web Benchmarking with IP-TNE
28
Web Benchmarking with IP-TNE
29
Results (Fig. 5b)

Persistent connections with HTTP/1.1:




Single client: 300 transactions/sec
Qualitatively similar results to before,
except that transaction rate is about
70% higher than for HTTP/1.0
(since multiple HTTP req’s per TCP conn)
Peak transaction rate 6500 trans/sec
Much less dependency on RTT effects
Web Benchmarking with IP-TNE
30
Web Benchmarking with IP-TNE
31
Results (Fig. 5c)

Pipelined persistent connections with
HTTP/1.1:




Single client: 800 transactions/sec
Qualitatively similar results to before,
except that transaction rate is about
100% higher than for HTTP/1.0
Peak transaction rate 7600 trans/sec
Much less dependency on RTT effects
Web Benchmarking with IP-TNE
32
Web Benchmarking with IP-TNE
33
Results (Fig. 6a)

Effect of WAN RTT delays:



Increasing the per-link propagation delay
increases the client RTT delay, which in
turn reduces the transaction rate and
throughput (as expected)
As RTT increases, more and more clients
are needed in order to drive the Web
server to full load
Similar to [Nahum et al. 2001]
Web Benchmarking with IP-TNE
34
Web Benchmarking with IP-TNE
35
Results (Fig. 6b)

Effect of bandwidth asymmetry:



For asymmetric access technologies such
as ADSL (Asymmetric Digital Subscriber
Line), the upstream link from the client
to the server can sometimes be the
bottleneck for TCP, even though it is
primarily carrying ACKs only
Depends on normalized bandwidth ratio
Greater asymmetry, worse performance
Web Benchmarking with IP-TNE
36
Web Benchmarking with IP-TNE
37
Results (Fig. 6c)

Effect of WAN packet losses:



Decreasing the router queue size at the
bottleneck link increases the packet loss
ratio (as expected)
As the level of packet loss increases,
the HTTP transaction rate and the
network throughput decrease
Similar results to [Nahum et al. 2001]
Web Benchmarking with IP-TNE
38
Web Benchmarking with IP-TNE
39
Summary and Conclusions




The IP-TNE is a useful tool for Web
server benchmarking
Demonstrates feasibility of WAN
emulation using a single computer
Confirms prior results by Nahum et al.
studying the effects of WAN
conditions on Web server performance
Demonstrates performance
advantages of HTTP/1.1
Web Benchmarking with IP-TNE
41
Future Work?

Increase traffic to your Web site!!!
Guarantee 20% increase in traffic!
1000’s of new clients per month!!!!!

Send $$$ to [email protected]


Web Benchmarking with IP-TNE
42
Future Work with IP-TNE












Validation of IP-TNE (and IP-TN)
Benchmarking IP-TNE vs IP-TNE
Benchmarking Web caching appliances
Evaluating SRPT scheduling in WAN setting
Connection/packet-level scheduling algorithms
Evaluating CATNIP approach to TCP/IP
Evaluating portable (wireless) Web servers
Workload sensitivities (Zipf, Pareto, corr, mods)
Experiments with dynamic content (CGI, etc)
Asymmetric networks, Ensemble-TCP
Parallel TCP connections: friend or foe?
Evaluating effect of TCP SACK in WAN
Web Benchmarking with IP-TNE
43