Transcript Document

Traffic Engineering of High-Rate
Large-sized Flows
Tian Jin, Chris Tracy, Malathi Veeraraghavan, Zhenzhen Yan
University of Virginia and ESnet
[email protected], [email protected]
July 8-11, 2013
Acknowledgment: UVA work is supported by DOE ASCR grants
DE-SC002350 and DE-SC0007341, and NSF grants, OCI1038058, OCI-1127340, and CNS-1116081, and ESnet work is
supported by DOE grant DE-AC02-05CH11231
1
Outline
• Problem statement & Motivation
– Example of ESnet measured load
– Adverse effects of “alpha flows”
• Hybrid Network Traffic Engineering
System (HNTES)
• HNTES evaluation
– NetFlow data collection
– Effectiveness
– Afflicted-flow packet percentage
2
Problem statement
• Flows generated by high-rate large-sized
file transfers are called alpha flows
– thresholds used in this paper: 1 GB in  1 min
• Previous work shows that alpha flows
– are the cause of burstiness of IP traffic
• Experiment shows adverse effects of alpha
flows on real-time A/V flows
• Problem: How can a provider identify such
alpha flows within their network and direct
them to separate QoS-controlled VCs?
3
Motivation: ESnet4
Core network for US Dept. of Energy Labs
Brookhaven National
Laboratory
PNNL
StarLight
BNL
MAN LAN
(32 A of A)
50
LLNL
LANL
FNL
ORNL
GA
IP
30/40/50G SDN
NLR 10G
MAN
Lab Link
IP router
SDN router
Optical
node
Lab
Steve Cotter, Chin Guok, Joe Metzger, Bill Johnston
Traffic surges on
ESnet interface
9 Gbps
Outgoing
traffic
Incoming
traffic
Jan. 12, 2013
Link rate: 10 Gbps
5
Motivation: Adverse
effects of alpha flows
• Used DOE 100G testbed
• Hosts: high-performance diskpts
TCP (alpha) flow
buffer buildups
ping flow
(delay-sensitive)
UDP flow
(background)
NEWY
BNL
6
Impact of alpha flows
on real-time flows
UDP flow
3 Gbps
6 Gbps
TCP flow
Pings: 1 per sec
Delay: 60 ms in 1-queue case
• Impact on ping flow
delay
– significant in 1-queue
configuration
– negligible in 2-queue
configuration
• Need separate virtual
queue for alpha flow
packets
Delay: 2.1 ms in 2-queue case
7
Outline
• Problem statement & Motivation
Hybrid Network Traffic Engineering
System (HNTES)
• HNTES evaluation
– NetFlow data collection
– Effectiveness
– Afflicted-flow packet percentage
8
Hybrid network traffic engineering system (HNTES)
- Intradomain identification/redirection of alpha flows
• Three steps
– Analysis of NetFlow
reports from ingress
routers to identify
address prefixes of
completed alpha
flows
– IDC creates L3
circuits between
ingress-egress router
pairs and configures
QoS
– IDC sets firewall
filters to direct
future alpha flows
with matching
address prefixes to
L3 circuits
Aging parameter (A): age out rules corresponding to
prefixes for which no alpha flows have been observed
9
Outline
• Problem statement & Motivation
• Hybrid Network Traffic Engineering
System (HNTES)
• HNTES evaluation
– NetFlow data collection
– Effectiveness
– Afflicted-flow packet percentage
10
Data collection for HNTES evaluation: NetFlow data
from 4 routers were collected for 7 months (214 days)
OP: observation point
router-1 & router-2: provider-edge (PE) routers
router-3: core router (REN peering)
11
router-4: core router (commercial peering)
Effectiveness Analysis
• Two types of effectiveness
– Cumulative effectiveness (Ci): percent of alpha
bytes (bytes reported in alpha NetFlow
reports) that would have been redirected in
period (1,i)
– Daily effectiveness (Ei): percent of alpha bytes
that would have been redirected on day i
• Choose aging parameter for:
– High effectiveness
– Stability in firewall-filter size
12
Aging parameter: tradeoff effectiveness
with size of firewall filter
Firewall filter size stable with aging parameter 30
Cumulative effectiveness > 90%
• graphs for router 1 (similar for other routers)
• 30 days is good compromise for aging parameter
13
Cumulative effectiveness (/24)
router-1 omitted as it is similar to router-2
Cumulative effectiveness
Boxplots for 214 values each
Provider edge routers
(single customers)
Peering routers
(router-3: REN;
router-4: commercial)
Why is cumulative
effectivness lower for peering
routers, esp. router-4?
14
Effectiveness comparisons
• Obs. 1: higher effectiveness for /24 than for /32
• Obs. 2: higher effectiveness for router-1 and router-2 than for
router-3 and router-4
• Obs. 3: fewer alpha prefix IDs for router-3 and router-4
15
Explanations
• Obs. 1: data-transfer node clusters are typically
located in the same /24 subnet; thus, repetition is
greater with /24 than /32
• Obs. 2 and obs. 3:
• Higher effectiveness for routers 1 & 2:
downloads from supercomputing facilities are
repetitive (a scientist accesses the same data
transfer nodes)
• Lower effectiveness for routers 3 & 4:
• fewer uploads to DoE labs than downloads
from DOE labs
• expect few, if any, scientific data transfers
16
from commerical peers (router-4)
Outline
• Problem statement & Motivation
• Hybrid Network Traffic Engineering
System (HNTES)
• HNTES evaluation
– NetFlow data collection
– Effectiveness
– Afflicted-flow packet percentage
17
Afflicted-flow packets
• B: set of non-alpha NetFlow reports for
flows that share alpha prefix IDs
• Divide B into four subsets in sequence
–
–
–
–
C: non-alpha reports of alpha flows
D  B-C: data-transfer reports (heuristic)
W  B-C-D: well-known ports
L: leftover = B-C-D-W
• Afflicted flows: W+L
18
Afflicted-flow packets
Percentage of afflicted-flow packets in samples of beta-flow (nonalpha flow) packets; across the 214-day period
• Tradeoff: /24 vs /32
– /32 has lower effectiveness: large % of afflicted-flow packets
will be impacted when an alpha flow is not redirected
– /24 has higher afflicted-flow packet percentage: small % of
afflicted-flow packets are adversely impacted
• Recommend /24 address prefixes for firewall filters
19
Conclusions
• Hypothesis: Most high-speed data transfer nodes have
static IP addresses, and alpha flows are created repeatedly
between the same source-destination subnets
– Validated for flows generated by dataset downloads as
observed at edge routers
• HNTES solution of determining src-dest address prefixes
of completed alpha flows & using these prefixes to set
firewall filters for future alpha-flow redirection is effective
for downloads from DOE labs
• Less effective for uploads esp. from commercial peering
links
–
But alpha-flow causing uploads are fewer
20