Transcript On the efficient detection of elephant flows in aggregated

On the efficient detection of
elephant flows in aggregated
network traffic
Javier Rivillo Lopez
Jose Alberto Hernandez
Iain W. Phillips
Networks and Control Group
Research School of Informatics
Loughborough University
[email protected]
[email protected]
[email protected]
LCS, 2005
Outline
• Motivation
• Flow analysis
• Detection method
• Experiments and results
• Conclusions
Motivation (I):
Definitions
• Flow: unidirectional set of packets of the same
•
transport protocol sharing the same source and
destination IP addresses and ports.
Elephant flow: stream of packets which
contribute to network load substantially more
than the rest of the flows.
– A threshold must be defined by the network
managers/administrators.
– The threshold value depends on the network size.
Motivation (II):
Elephant and mice phenomenon
• Usually a few
•
•
flows carry
most of the
data
Trace example
from NLANR
router. In
future, MASTS.
0.1% of the
flows carry
nearly 83% of
the total traffic
Figure 1: A flow aggregation view of network traffic
Motivation (III):
Elephant and mice phenomenon
• Elephant flows
– Low-priority applications
– Large data transfer transactions and peer-to-peer file
sharing
• Mice flows
– Sensitive to delay, jitter and high loss rates.
– Voice over IP, online gaming, small http requests
• Under this phenomenon, Internet's best effort
•
delivery is not suitable.
The performance of the network can be
improved by detecting the elephant flows and
applying traffic engineering solutions
Flow analysis (II):
Flow duration
Figure 2: Flow duration histogram
• Most of the mice flows (92%) have very short duration (< 2 sec)
• Most elephants are long duration flows: heavy tail behaviour.
Flow analysis (III):
Mean interarrival time
Figure 3: Flow mean interarrival histogram
• Elephants have very low average packet interarrival time.
• So, a flow with high average packet rate and long duration is very
likely to be an elephant.
Detection method (I):
Sampling
• Requirement: Low computational cost
• Continuous monitoring not suitable:
– Requires huge amount of resources.
– Not scalable.
• Sampling is required.
• Random sampling not suitable because we lose
•
information about the packet interarrival and timing.
Solution: Windowing.
– Example: monitoring the network 20ms every 2 seconds.
• Monitoring 1% of the time, Sampling factor = 20ms/2sec = 100
Detection method (II):
Elephant detection algorithm
• Objective: identify flows with low packet interarrival
•
•
•
time (high packet rate) and long duration: Elephants.
Step 1. A flow has high packet rate when it has at least
Np packets in a sampling window.
Step 2. A flow is considered elephant when it has been
identified as high packet rated flow in Nw different
sampling windows.
Parameters:
– Np, Nw, w and T
• In future, the algorithm will be adaptative: The
parameters will be calculated automatically by the
system.
Experiments and Results
•
Results with w=20ms and T=2sec:
Figure 4: Flows identified as elephant traffic
• Np=2, Nw=2: 80% of the elephant flows are correctly identified,
they carry 89% of the total traffic and 0.12% of the mice flows are
misidentified as elephants
• Increasing Np and Nw, we get more Precision but less Recall.
Conclusions
• Identifying elephant flows for traffic engineer solutions
can improve the network performance.
• The properties of elephant and mice flows have been
obtained studying real traffic data.
• The long tail behaviour and high packet transmission
rate shown by the elephants have been used in the
elephant detection method explained.
• This scalable and low computational cost method uses
high sampling rate for early detection of elephant flows.
• We have shown in the results that it is a valid method
and its parameters may be adjusted for a tradeoff
between Precision and Recall in identifying the elephant
flows.
THANK YOU!