Transcript Presentation Templats

© Asifuddin Mohammad
Empirical Model for HTTP Network
Traffic
Asifuddin Mohammad
Club presentation
27 February 2009
Abbreviated Paper Title
1
© Asifuddin Mohammad
Empirical Model for HTTP Network
Traffic
• Paper I
– An Empirical Model of HTTP Network Traffic
– Bruce A. Bah, INFOCOM ‘97
• Paper II
– Empirical Models of TCP and UDP End–User Network Traffic
from NETI@home Data Analysis
– IEEE 2006
27 February 2009
Abbreviated Paper Title
2
© Asifuddin Mohammad
An Empirical Model of HTTP Network
Traffic
[Mah-INFOCOM-1997]
Asifuddin Mohammad
Club presentation
[email protected]
http://www.people.ku.edu/~asifm
27 February 2009
© 2009–Asifuddin Mohammad
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Abstract
•
The workload of the global Internet is dominated by the Hypertext
Transfer Protocol (HTTP), an application protocol used by World Wide
Web clients and servers. Simulation studies of this environment will
require a model of the traffic patterns of the World Wide Web, in order
to investigate the performance aspects of this increasingly popular
application. We have developed an empirical model of network traffic
produced by HTTP. Instead of relying on server or client logs, our
approach is based on gathering packet traces of HTTP network
conversations. Through traffic analysis, we have determined statistics
and distributions for higher-level quantities such as the size of HTTP
items retrieved, the number of items per “Web page”, think time, and
user browsing behavior. These quantities form a model can then be
used by simulations to mimic World Wide Web network applications in
wide-area IP internetworks.
1 January 2000
Abbreviated Paper Title
4
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Outline
•
•
•
•
•
•
•
•
Introduction
Background
Prior Work
Methodology
Model
Experimental Results
Conclusions
References
1 January 2000
Abbreviated Paper Title
5
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Introduction
•
•
•
•
•
•
•
•
Introduction
Background
Prior Work
Methodology
Model
Experimental Results
Conclusions
References
1 January 2000
Abbreviated Paper Title
6
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Introduction
• Develop an empirical model for HTTP
– Accurate models of the system under study to yield useful
data
– Provides a synthetic workload to simulation of wide area IP
internetwork
– Based on network packet traces
• At Lowest Level
– Describes the sizes of individual web files
• At highest level
– Describes the browsing behavior of the user
1 January 2000
Abbreviated Paper Title
7
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Background
•
•
•
•
•
•
•
•
Introduction
Background
Prior Work
Methodology
Model
Experimental Results
Conclusions
References
1 January 2000
Abbreviated Paper Title
8
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Background
• WWW
– Collection of documents
– Each document consist of # of files
– E.g. multipart document = text (in HTML) + images
• HTTP request-response protocol
• HTTP uses TCP for reliable transfer
– Non-Persistent TCP
– Persistent TCP was proposed but not implemented
1 January 2000
Abbreviated Paper Title
9
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Prior Work
•
•
•
•
•
•
•
•
Introduction
Background
Prior Work
Methodology
Model
Experimental Results
Conclusions
References
1 January 2000
Abbreviated Paper Title
10
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Prior Work
• Approaches taken to characterize internet application
– Server logs
– Client logs
– Packet or Traffic traces
• Server Logs
– Logs ranging from operational monitoring to collecting the
demographic info about the user
– Easiest way
– 2 dis-adv
• No easy capture of users access pattern across multiple servers
• No HTTP overhead capture
1 January 2000
Abbreviated Paper Title
11
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Prior Work
• Client Logs
–
–
–
–
No problem in capturing user accesses b/w multiple servers
Characterization of client side caching of documents
Need the browser code to log request size
Supporting variety of browsers may be difficult if modified
• Packet Trace
– Analyzing the HTTP packet trace taken from a subnet
– Used in number of traffic studies ; eliminates dis-adv
– 2 dis-adv
• More effort to reconstruct the TCP connection contents
• Effects of client caching of documents are difficult to determine
1 January 2000
Abbreviated Paper Title
12
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Methodology
•
•
•
•
•
•
•
•
Introduction
Background
Prior Work
Methodology
Model
Experimental Results
Conclusions
References
1 January 2000
Abbreviated Paper Title
13
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Methodology
• Packet Trace approach
–
–
–
–
–
No higher level info like actual file accessed
TCPdump packet capture utility
More than dozen networks with 100’s of hosts
TCP port 80
Recorded packet loss of 0.014%
1 January 2000
Abbreviated Paper Title
14
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Model
•
•
•
•
•
•
•
•
Introduction
Background
Prior Work
Methodology
Model
Experimental Results
Conclusions
References
1 January 2000
Abbreviated Paper Title
15
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Model
• Empirical distribution for different quantities
1 January 2000
Abbreviated Paper Title
16
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
•
•
•
•
•
•
•
•
Introduction
Background
Prior Work
Methodology
Model
Experimental Results
Conclusions
References
1 January 2000
Abbreviated Paper Title
17
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
• Few Results
1 January 2000
Abbreviated Paper Title
18
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
1 January 2000
Abbreviated Paper Title
19
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
1 January 2000
Abbreviated Paper Title
20
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
• Page Length:
1 January 2000
Abbreviated Paper Title
21
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
27 February 2009
Abbreviated Paper Title
22
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
1 January 2000
Abbreviated Paper Title
23
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
1 January 2000
Abbreviated Paper Title
24
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
1 January 2000
Abbreviated Paper Title
25
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
1 January 2000
Abbreviated Paper Title
26
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Experimental Results
• Server Selection
– No proper results
– Approximate the distribution to Zipf’s law
• Zipf’s Law
– Probability of selecting the ith most popular item in a set is
proportional to 1/i
1 January 2000
Abbreviated Paper Title
27
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Conclusions
•
•
•
•
•
•
•
•
Introduction
Background
Prior Work
Methodology
Model
Experimental Results
Conclusions
References
1 January 2000
Abbreviated Paper Title
28
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
Conclusions
• An Empirical Model is developed
1 January 2000
Abbreviated Paper Title
29
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
References
•
•
•
•
•
•
•
•
Introduction
Background
Prior Work
Methodology
Model
Experimental Results
Conclusions
References
1 January 2000
Abbreviated Paper Title
30
© Asifuddin Mohammad
Empirical Model for HTTP Network
Traffic
• Paper I
– An Empirical Model of HTTP Network Traffic
– Bruce A. Bah, INFOCOM ‘97
• Paper II
– Empirical Models of TCP and UDP End–User Network Traffic
from NETI@home Data Analysis
– Charles R. Simpson, Jr. Dheeraj Reddy, George F. Riley
– IEEE 2006
27 February 2009
Abbreviated Paper Title
31
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data
Analysis
•
Abstract
The simulation of computer networks requires accurate models of user
behavior. To this end, we present empirical models of end–user
network traffic derived from the analysis of NETI@home data. There
are two forms of models presented. The first models traffic for a
specific TCP or UDP port. The second models all TCP or UDP traffic for
an end–user. These models are meant to be network–independent and
contain aspects such as bytes sent, bytes received, and user think
time. The empirical models derived in this study can then be used to
enable more realistic simulations of computer networks.
1 January 2000
Abbreviated Paper Title
32
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Outline
•
•
•
•
•
•
•
•
Introduction
Background & Related Work
Methodology
Experimental Results
Simulation Results
Future Work
Conclusions
References
1 January 2000
Abbreviated Paper Title
33
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Introduction
•
•
•
•
•
•
•
•
Introduction
Background & Related Work
Methodology
Experimental Results
Simulation Results
Future Work
Conclusions
References
1 January 2000
Abbreviated Paper Title
34
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Introduction
• Simulation
– Popular method to evaluate characteristics of networks
• Need
– For accurate Models for Simulator component
• E.g. End-User Traffic
– To update the Models frequent
1 January 2000
Abbreviated Paper Title
35
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Background & Related Work
•
•
•
•
•
•
•
•
Introduction
Background & Related Work
Methodology
Experimental Results
Simulation Results
Future Work
Conclusions
References
1 January 2000
Abbreviated Paper Title
36
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Background & Related Work
• Expansion of Bah’s empirical distribution
• Network Intelligence at Home (NETI@home)
– Software is used instead of packet trace
– Reports end to end flow summary stats
• Bah’s study was conducted on specific port
– TCP port 80
• Experiment conducted on any given TCP or UDP port
1 January 2000
Abbreviated Paper Title
37
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Methodology
•
•
•
•
•
•
•
•
Introduction
Background & Related Work
Methodology
Experimental Results
Simulation Results
Future Work
Conclusions
References
1 January 2000
Abbreviated Paper Title
38
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Methodology
• 2 categories of models
– TCP or UDP port
– Aggregate of all port-specific models
• Dataset collected over one year period
– Oct 1,2004 to Sep 30,2005
– 36 million TCP flows and 93 million UDP flows
• Aspects
–
–
–
–
Bytes send and bytes received
User think time
Consecutive contacts
Contact selection
1 January 2000
Abbreviated Paper Title
39
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Experimental Results
•
•
•
•
•
•
•
•
Introduction
Background & Related Work
Methodology
Experimental Results
Simulation Results
Future Work
Conclusions
References
1 January 2000
Abbreviated Paper Title
40
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Experimental Results
• Bytes sent
1 January 2000
Abbreviated Paper Title
41
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Experimental Results
• Bytes Received
1 January 2000
Abbreviated Paper Title
42
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Experimental Results
• User Think Time to the same IPs
1 January 2000
Abbreviated Paper Title
43
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Experimental Results
• User Think Time to differing IPs
1 January 2000
Abbreviated Paper Title
44
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Experimental Results
• Consecutive Contacts
1 January 2000
Abbreviated Paper Title
45
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Experimental Results
• Contact selection
1 January 2000
Abbreviated Paper Title
46
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Simulation Results
•
•
•
•
•
•
•
•
Introduction
Background & Related Work
Methodology
Experimental Results
Simulation Results
Future Work
Conclusions
References
1 January 2000
Abbreviated Paper Title
47
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Simulation Results
• Implemented in GTNets
– Data collected for NETI as well as Bah’s empirical Models
– Network topology used is shown
1 January 2000
Abbreviated Paper Title
48
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Simulation Results
INET@home
1 January 2000
Bah’s Model
Abbreviated Paper Title
49
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Future Work
•
•
•
•
•
•
•
•
Introduction
Background & Related Work
Methodology
Experimental Results
Simulation Results
Future Work
Conclusions
References
1 January 2000
Abbreviated Paper Title
50
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Future Work
• Useful to Model Idle time
• Determine Correlation between different aspects of
this Model
– E.g. correction between bytes sent and byte received
• Enhancements to consecutive contacts and contact
selection using a memory based model
– Markov model
• Extend to other protocols beyond TCP or UDP
• Model network-dependent characteristics of internet
• Develop a analytical models from empirical
1 January 2000
Abbreviated Paper Title
51
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Conclusions
•
•
•
•
•
•
•
•
Introduction
Background & Related Work
Methodology
Experimental Results
Simulation Results
Future Work
Conclusions
References
1 January 2000
Abbreviated Paper Title
52
© Asifuddin Mohammad
Empirical Models of TCP and UDP End–User
Network Traffic from NETI@home Data Analysis
Conclusion
• This paper developed an empirical model for the
number of bytes sent, number of bytes received, the
user think time to the same destination, the user
think time to a different destination, the number of
times a destination will be contacted consecutively,
and the popularity of specific destinations.
1 January 2000
Abbreviated Paper Title
53
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
References
•
•
[1] D. P. Anderson and et al. SETI@home: Search for extraterrestrial intelligence at home. Software on-line:
http://setiathome.ssl.berkeley.edu, 2003.
[2] C. Barakat, P. Thiran, G. Iannaccone, C. Diot, and P. Owezarski. Modeling internet backbone traffic at the flow level. IEEE
•
[3] P. Barford and M. Crovella. Generating representative web workloads for network and server performance evaluation. In
•
[4] J. Cao, W. S. Cleveland, Y. Gao, K. Jeffay, F. D. Smith, and M. C. Weigle. Stochastic models for generating synthetic HTTP
source traffic. In IEEE INFOCOMM, March 2004.
[5] Y.-C. Cheng, U. Holzle, N. Cardwell, S. Savage, and G. M. Voelker. Monkey see, monkey do: A tool for TCP tracing and
replaying. In Proceedings of USENIX Technical Conference, June 2004.
[6] H.-K. Choi and J. O. Limb. A behavioral model of web traffic. In ICNP, 1999.
[7] M. Christiansen, K. Jeffay, D. Ott, and F. D. Smith. Tuning RED for web traffic. IEEE/ACM Transactions on Networking,
9(3):249–264, June 2001.
[8] S. Floyd and V. Paxson. Difficulties in simulating the internet. IEEE/ACMTransactions on Networking, 9(4):392–403, August
2001.
[9] J. B. Grizzard, C. R. Simpson, Jr., S. Krasser, H. L. Owen, and G. F. Riley. Flow based observations from NETI@home and
honeynet data. In Proceedings from the sixth IEEE Systems, Man and Cybernetics Information Assurance Workshop, pages 244–
251, June 2005.
[10] F. Hernandez-Campos, A. B. Nobel, F. D. Smith, and K. Jeffay. Understanding patterns of TCP connection usage with
statistical clustering. In IEEE MASCOTS, 2005.
[11] F. Hernandez-Campos, F. D. Smith, and K. Jeffay. Generating realistic TCP workloads. In Computer Measurement Group
•
•
•
•
•
•
•
•
•
•
Transactions on Signal Processing – Special Issue on Networking, 51(8), August 2003.
ACM SIGMETRICS, 1998.
International Conference, December 2004.
[12] L. Le, J. Aikat, K. Jeffay, and F. D. Smith. The effects of active queue management on web performance. In ACM
SIGCOMM, pages 265–276, August 2003.
[13] B. A. Mah. An empirical model of HTTP network traffic. In IEEE INFOCOMM, April 1997.
1 January 2000
Abbreviated Paper Title
54
© Asifuddin Mohammad
An Empirical Model of HTTP Network Traffic
References
•
[14] G. F. Riley. The Georgia Tech Network Simulator. In Proceedings of the ACM SIGCOMM workshop on Models, methods and
tools for reproducible network research, pages 5–12, 2003.
[15] C. R. Simpson, Jr. NETI@home. Software on-line: http://neti.gatech.edu, 2003. Georgia Institute of Technology.
[16] C. R. Simpson, Jr. and G. F. Riley. NETI@home: A distributed approach to collecting end-to-end network performance
measurements. In PAM2004 - A workshop on Passive and Active Measurements, April 2004.
[17] F. D. Smith, F. Hernandez-Campos, K. Jeffay, and D. Ott. What TCP/IP protocol headers can tell us about the web. In ACM
•
[18] J. Sommers, H. Kim, and P. Barford. Harpoon: A flow–level traffic generator for router and network tests. In ACM
•
•
•
SIGMETRICS, pages 245–256, 2001.
SIGMETRICS, June 2004.
•
[19] M. Weigle, K. Jeffay, and F. D. Smith. Delay–based early congestion detection and adaptation in TCP: Impact on web
performance. ACM Computer Communications Review, 28(8):837–850, May 2005.
•
[20] J. Xu andW. Lee. Sustaining availability of web services under distributed denial of service attacks. IEEE Transactions on
Computers, 52(2):195–208, February 2003.
1 January 2000
Abbreviated Paper Title
55