Transcript Bandwidth on the Internet

Bandwidth on the Internet
Mat Ford
ISOC Standards & Technology
ISOC Advisory Council Meeting, Hiroshima, Japan
Sunday November 8th 2009
Takeaways, upfront
•
Gross bandwidth growth more than catered for by new
capacity
•
•
•
•
•
•
No Internet ‘brownouts’ anytime soon
Broadband creating new realities for access nets
•
More bandwidth provisioning isn’t the answer
•
Additional capacity is quickly consumed (by design)
Interests of content providers, ISPs and users not always
well-aligned
We need more data
IETF has several streams of relevant work
ISOC has a panel on this topic during IETF76
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
2
Bandwidth, defined
•
A term that describes the amount of information that can be
passed through a communications channel in a given amount
of time; that is, the capacity of the channel. The bandwidth is
usually expressed in 'bits per second’…
•
•
Or bytes per second, leading to lots of confusion 
Used for expressing
•
Capacity of a channel or interface (e.g. 100Mbps ethernet i/f)
•
Rate of information transfer (e.g. downloading at 3.2KB/s)
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
3
Internet is sharing
•
•
•
•
Internet protocols have their origin in the desire to share
computers in diverse locations
Extension of the OS-paradigm of shared resources
Early objections to feasibility of
packet-switching centred on
the need for very large packet
buffers to handle uncontrolled
loads from end hosts
Leonard Kleinrock credited with
demonstrating the theoretical
feasibility of packet-switched
communications
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
4
Rules of the road
•
Resource sharing
• Central to notions of Internet’s value and
success
• Enables interconnection of diverse applications
• Over heterogeneous networking media with
diverse speeds
•
Sharing creates potential for demand to
outstrip supply
• Necessitates a sharing mechanism
• This is congestion control
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
5
How much bandwidth?
•
Cisco Forecasts 44 Exabytes per Month of IP Traffic in 2012
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
6
Some rough consensus?
Source
MINTS
Cisco
Observatory
Traffic volume per month
(exabytes)
5-8
11
9
Traffic annual growth rate
50-60%
46%
44.5%
Kenjiro Cho et al. see ~40%
growth per annum since
2005, for IX peaks
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
7
What about capacity?
•
Telegeography: Capacity growing by 50% per annum
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
8
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
9
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
10
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
11
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
12
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
13
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
14
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
15
Congestion
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
16
Definitions of congestion
•
Arrival rate > service rate (λ > μ)
•
•
Congestion = arrival rate/service rate (p=λ/μ)
In the presence of queues, congestion only present when
packets are dropped?
•
•
This is TCP’s definition of congestion
For network operators, congestion is usually measured over
longer timescales, e.g. 15mins
For economists, congestion occurs when increased use
would impose a cost on existing users
•
•
In this case congestion can occur before queues start to build or
packets are dropped
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
17
Evolution of congestion control
•
•
Congestion control originally handled by ICMP
Widespread incidents of congestion collapse in mid-1980s
•
•
Early TCP implementations had very bad retransmission behavior that
pushed the entire network into a state where most packets were lost
and the resultant throughput was negligible.
Addition of slow start, congestion avoidance, fast retransmit and
fast recovery algorithms to TCP
Image source: Internet Protocol Journal, Vol 9, No. 2
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
18
Standards evolution
•
RFC 793 ‘Transmission Control Protocol’, STD 7 (Sept.
1981)
•
Doesn’t specify any congestion control mechanism
•
Congestion control handled by ICMP Source Quench
•
RFC 1122 ‘Requirements for Internet Hosts –
Communication Layers’ (October 1989)
•
•
Mandates implementation of a congestion control algorithm
RFC 2581 ‘TCP Congestion Control’ (April 1999)
•
Defines the accepted mechanism
•
Required for the avoidance of congestion collapse
•
Flow-rate fairness
Ref: RFC 4614 ‘A Roadmap for TCP Specification Documents’
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
19
TCP
•
•
•
Implemented on end hosts
Regulates sending rate for a single flow
Doesn’t observe sending rate over time, or whether there are
multiple flows from a single end host
Doesn’t regulate aggregate demand
Doesn’t know where congestion occurred, when it occurs
TCP will keep getting faster unless
•
•
•
•
All data is sent
•
Congestion experienced
•
End system maxed out
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
20
Animation credit: Guido Appenzeller
TCP in action
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
21
TCP isn’t enough
•
•
In his original paper, Van Jacobson noted some limitations
While algorithms at the transport endpoints can insure the
network capacity isn’t exceeded, they cannot insure fair
sharing of that capacity. Only in gateways, at the
convergence of flows, is there enough information to control
sharing and fair allocation. Thus, we view the gateway
‘congestion detection’ algorithm as the next big step.’
•
Whether or not multiple flows share common bottleneck links
isn’t clear from the edge – only net ops have this view
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
22
Cable vs. DSL architectures
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
23
Slide credit: Bob Briscoe
TCP-fairness
capacity
• voluntarily polite algorithms in endpoints
• pushes until congested
• equalises rates of data flows
bandwidth2
bandwidth1
time
a game of chicken – taking all and holding your ground pays
unresponsive
flow3
(VoIP, VoD)
or start more ‘TCP-fair’ flows than anyone else (Web: x2, p2p: x5-100)
or for much more data than others (video streaming or p2p file-sharing x200)
•
24
2009-11-08
net effect of both (p2p: x1,000-20,000 higher traffic intensity)
Bandwidth on the Internet, ISOC AC Meeting
In defence of flow-rate fairness
•
•
Floyd and Allman (2008)
‘We do not, however, claim that flow-rate fairness is
necessarily an optimal fairness goal or resource allocation
mechanism for simple best-effort traffic. Simple best-effort
traffic and flow-rate fairness are in general not about
optimality, but instead are about a low-overhead service
(best-effort traffic) along with a rough, simple fairness model
(flow-rate fairness).’
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
25
What’s different this time?
•
•
•
•
•
Congestion collapse isn’t new
•
Happened in the 80s
•
Predicted again in the 90s (Metcalfe’s
gigalapse)
•
More predictions swirling around now
Growing diversity of uses/users
Increasing traffic volumes
Increasing economic value
Emergence of public policy debates
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
26
Impact of all-you-can-eat broadband
45%
Broadband
Usage Distribution
40%
(now Arbor Networks)
source: Ellacoya 2007
27%
20%
20%
15%
16%
8%
5%
4%
% of subscribers
% traffic
Source: Cho et al., Proceedings of ACM CoNEXT2008
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
27
A changed landscape
•
Services that are now dominant consumers of bandwidth
were simply not viable at pre-broadband access data rates
Natural suppression of user demand
Congestion is at the edge
Huge growth of mass-market broadband since 2000
•
•
•
•
‘a seismic shift in the nature of the congestion problem’
•
Bandwidth hungry apps and multiple users per connection
•
Raised expectations
•
Access nets now the dominant constraint on achievable throughput
•
Changing diurnal patterns of consumption
•
Growing symmetry of traffic patterns
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
28
Slide credit: Bob Briscoe
ISPs forced to subvert TCP
bit-rate
1. equal bottleneck flow rates
(TCP)?
2. access rate shared between active users,
bit-rate
time
bit-rate
time
bit-rate
time
but weighted by fee (weighed fair queuing, WFQ)?
3. volume caps
tiered by fee?
4. heaviest applications of heaviest users
throttled at peak times by deep packet inspection
(DPI)?
29
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
29
time
Graphic credit: Bob Briscoe
Congestion isn’t a bug, it’s a feature
•
•
•
Want transmissions to occur as fast as possible
Lack of congestion implies network could be more efficient
Techniques are available to enable the use of congestion
signals to control flow rates without lost
packets/retransmissions
bitrate

bitrate
time
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting

time
30
Slide credit: Bob Briscoe
Better alternatives
bit-rate
1. TCP
bit-rate
weighted
TCP
sharing
bit-rate
time
2.
(weighted)
fair
queuing
3. volume
caps
4. deep
packet
inspection
(DPI)
•
•
bit-rate
time
•
•
bit-rate
31
2009-11-08
light usage can go much faster
hardly affects completion time of
heavy usage
doesn’t have to shift into night
BitTorrent & Microsoft have
protocols to do this
time
but... punished by #2, #3 & #4
Bandwidth on the Internet, ISOC AC
Meeting
NOTE:
weighted sharing doesn't imply differentiated network service
time
•
just weighted aggressiveness of end-system's rate response to
congestion
time
Slide credit: Bob Briscoe
Limiting the horizon
•
Becoming impossible to deploy a new use of the Internet
•
•
Two confusable motives
•
Fairer cost sharing
•
Competitive advantage to own services
•
How to deconfuse? how to encourage fairer cost sharing?
•
•
Must negotiate arbitrary blocks and throttles en route
Make cost of usage transparent
Fixing Internet technology should avoid need for legislation
32
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
32
BBC iPlayer example
•
Original version used P2P to minimise BBC wholesale
bandwidth costs
Latest update, P2P technology disabled
•
•
•
Disliked by end-users
•
Disliked by ISPs
•
BBC claim wholesale bandwidth costs have fallen
iPlayer Desktop warns users to beware ISP bandwidth caps
and charges
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
33
Wrapping up
•
•
•
Recent studies conclude
•
P2P declining
•
Being replaced with video and other web2.0 content
•
Network capacity growing faster than traffic volumes
Interests of content providers, ISPs and users not always
well-aligned
Fair management of congestion is a whole network issue
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
34
We need more data
•
•
•
•
•
•
•
http://mitas.csail.mit.edu/
MINTS
CAIDA
M-Lab
Transparency wrt congestion management policies of net
ops is a good start
Mismanagement of congestion is stifling application
innovation
Want to avoid imposition of regulation that enshrines TCP
flow-rate fairness in law – there is ample scope for innovation
in congestion management mechanisms at both sub-second
and sub-month timescales.
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
35
Network operators play a key role
•
•
•
Determine supply of network resources
Limit demand (access link capacity)
Is there a role for network operators at shorter time scales?
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
36
An inflection point?
•
•
•
Can the Internet community move to a new notion of what
constitutes ‘fair’ resource allocation on the Internet?
TCP has ‘incumbent’s advantage’
If it ain’t broke, don’t fix it!
•
Has the environment changed so significantly that it is broke?
•
Who should have responsibility for prioritising traffic during
overload conditions?
How can the answer to that question be made to work in
practice?
•
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
37
Relevant IETF work
•
Conex BoF
•
•
Exposing expected congestion along the forwarding path of the
Internet
Ledbat WG
•
•
Congestion control algorithm for scavenger service
Alto WG
•
•
Protocols for better-than-random peer selection
MultipathTCP WG
•
•
Simultaneously use multiple paths in a single TCP session
Homegate BoF
•
Getting uniform set of requirements to aid deployment
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
38
ISOC Panel
•
Internet Bandwidth Growth: Dealing with Reality
•
11.45am – 12.45pm, Tuesday 10th November
•
Light lunch provided
•
Orchid West Room, ANA Crowne Plaza Hotel
•
•
Panellists
•
Leslie Daigle (Moderator)
•
Kenjiro Cho, IIJ
•
Richard Woundy, Comcast
•
Danny McPherson, Arbor Networks
•
Lars Eggert, Nokia Research Center, IETF Transport AD
Audiocast & report will be available
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
39
References
•
The Evolution of Internet Congestion
•
•
Steven Bauer, David Clark, William Lehr (MIT)
Internet Cost Transparency: Mending Value Chain Incentives
•
•
Bob Briscoe (BT)
ATLAS Internet Observatory 2009 Annual Report
•
•
C. Labovitz et al. (Arbor Networks, Inc.)
Observing Slow Crustal Movement in Residential User Traffic
•
Cho et al. (IIJ)
2009-11-08
Bandwidth on the Internet, ISOC AC Meeting
40