Transcript Random Thoughts on Application Performance and

Random Thoughts on Application Performance
and Network Characteristics
Brian L. Tierney
[email protected]
Distributed Systems Department
Lawrence Berkeley National Laboratory
9/29/04
GGF
Questions
• 1) Applications have different requirements of the various
network attributes (ie bandwidth, packet loss, latency,
jitter). How would you create classifications of applications
such that network interfaces could be created to optimize
performance for each classification?
• 2) What can the network do better to create better
performing applications for each of the classifications?
• 3) What can the applications do to make better use of the
network?
• 4) Can standard applications interfaces (sockets) and/or
protocols be created to enable the ideas in 2 and 3?
9/29/04
GGF
Application Classifications
• Types of applications
– Loss sensitive applications
– Jitter sensitive applications
– Latency sensitive apps
• All would benefit from more network knowledge
• All require some form of QoS
– High Bandwidth applications
• “Fixing” TCP could make network monitoring unnecessary for these
types of applications
• Should need nothing more than next generation TCP and parallel
streams
• Higher-level services like LSL should also help
• Important to define exactly what problems we are trying to solve
– What applications are most important
9/29/04
GGF
Ubiquitous Network Monitoring
• Image an internet with the following ability:
– A DNS query (e.g.: gethostbyname() ) returned all of the
following:
•
•
•
•
•
•
•
IP address
Lat/long
Min/max/average delay over the past hour
Min/max/average available bandwidth over the past hour
packet loss metric
out of order packet metric
jitter metric
9/29/04
GGF
Network Monitoring: Open Issues
• Many Issues!
– Use passive or active monitoring?
– End-to-end? 1st router to last router? 1st router to end
host?
– What should be reported? min/max/ave/std dev/? Time
series? Over what time interval?
– Prediction? (“ask the network oracle”?) Over what time
interval?
– What about QoS capabilities? How does QoS effect the
above results?
• Exactly how will applications use this data?
• Need to prioritize which application classes will gain the
most benefit
9/29/04
GGF
Make Applications Smarter?
• Do applications need to be “network aware”, or should they
use higher level services which are “network aware”?
• Many simple things are not being done today
– Long pipelines
– Large TCP buffers
– More application buffering
• Need better services to hide network from the application
– Examples:
– Logistical Network Layer (M Swany)
– Buffering Service (Kangaroo)
– Forward error correction service
9/29/04
GGF
Better Bulk Transfer
• Assuming data is replicated in multiple locations (not always true…),
always just download in parallel from multiple sources (bittorent model)
• Example:
– Copies of a file exist in 20 location
– Divide the file into blocks (eg: 256K), and download in parallel from
5 servers
• Servers chosen based on shortest RTT
– If blocks from a given server are coming in too slow, try a different
server
– Should be easy to saturate the incoming host, while not overloading
any server
• Monitoring is all application level.
– Save performance results to use as estimate for the next time
9/29/04
GGF
New API’s
• Higher level APIs are needed
– but what should they look like?
9/29/04
GGF
Conclusion
• Need to determine exactly what network monitoring data
applications (or network services) really need and will
actually use
• New GGF Network Monitoring for Applications Research
Group (NMA-RG) has been formed to precisely deal with
these issues
– http://dsd.lbl.gov/NMA-RG/
9/29/04
GGF