Transcript 20080120-litvanyi

Community Attribute Use in
Internet2 CPS
Caren Litvanyi
lead network engineer
peering team
Internet2 NOC
GigaPoP Geeks BOF
January 2008
Honolulu, Hawai’i
Commercial Peering Service
Outline
• Quick review of what CPS is.
• Quick notes on how to connect.
• So if I’m connected to CPS, how can I
use communities to control how my
routes are advertised?
• Discussion and feedback from all you
GPG’s.
Commercial Peering Service
CPS Background
CPS is Internet2’s “Commercial Peering Service”.
“Through CP Service members can leverage their
existing Internet2 Network investments to help
serve their commercial Internet needs, thereby
saving money on commodity Internet charges.”
CPS is included in the base connection fee, so it is
available for Internet2 Network connectors at no
additional cost.
http://www.internet2.edu/network/cp/
Commercial Peering Service
CPS Background
•
Implemented as an “overlay”
on the existing Internet2
Layer3 Network.
•
Uses MPLS Layer3 VPN,
VRF on same T640 routers.
•
Currently 4 commercial public
peering sites:
–
–
–
–
PAIX New York 10GE
Equinix Chicago 10GE
PAIX Palo Alto 10GE
SIX (Seattle) 1GE
•
Also PNI (private peerings).
•
Today has over 76,000 unique commercial
prefixes from approximately 50 peers, and
advertises about 850 connector prefixes.
Commercial Peering Service
Connecting to CPS - brief
• Call up the Internet2 NOC, open a ticket to connect to Internet2 CPS.
• You will need to enable 802.1Q VLAN encapsulation on your Ethernet
connection to Internet2, or frame-relay encapsulation on a SONET connection.
Add an additional VLAN (or DLCI) to carry CPS traffic.
• Assign IP addresses (/30 or /31) in the usual way. MTU is 1500.
• Supply the NOC with a list of ASs behind you, or reference an AS-SET object
you maintain.
• Supply the NOC with a prefix list of what you will advertise to CPS, or agree to
use your existing Internet2 prefix lists, or provide a diff.
• Set up the BGP peering - it’s with AS 11537 (same as R&E network).
• Pad towards your direct commercial providers as desired to shift inbound traffic
away from them, letting end commercial networks see the CPS path as “better”.
• Local-pref CPS higher, so your outbound traffic prefers CPS over your direct
commodity providers, as desired.
Commercial Peering Service
Of course, we didn’t cover…
• Analyzing your current commercial traffic patterns, if any, to get a
baseline.
• Checking that your circuit to Internet2 can handle the additional load
without affecting R&E traffic.
• Figuring out how, or if, this will interact with your existing bandwidth
shapers, firewalls, etc., if any.
• Considering how a circuit failure to Internet2 or one of your other
commercial providers will be handled.
• Figuring out how you will distribute this to downstream connectors,
how/if it will be measured/charged…
• Educating your downstream connectors.
Commercial Peering Service
But I want more control!
• We give you some! It not perfect, but it’s pretty easy to
understand and implement.
• You can attach certain communities to your prefixes you
advertise to Internet2 CPS that in turn, affects how we
advertise your prefix to commercial peers. (details next slide)
• You can, if you like, configure your network policy to allow
your downstreams to do this themselves.
• Combining this with adjustments to your import policy,
gives you better control.
• We also support blackhole routing for up to /24s.
Commercial Peering Service
Using communities in CPS
• Inbound traffic, outbound route policy:
– If you do not want CPS peer network X to send traffic to
you over Internet2 CPS, you can tag your prefixes with
the BGP Community 65000:<foo> where ”<foo>" is the
BGP ASN of peer network X.
– CPS has an outbound policy specific to each
commercial peer that will prevent the advertisement of
your prefix to it appropriately, CPS-AS<foo>-OUT.
– Note we do NOT do this “per location”, eg, “advertise
my route to Shaw (AS6327) in New York but not Seattle”.
Commercial Peering Service
Using communities in CPS
• For example:
– Suppose your downstream customer <bar> has called you up to say
they don’t want YouTube to send traffic destined to their dorm network
over CPS, ever, not even as a last resort. Though they want other
traffic to come across CPS destined to that network.
– In your BGP policy with CPS, apply policy outbound that tags that dorm
network prefix (or prefixes) with 65000:36561, since 36561 is YouTube’s
AS number.
– When CPS processes what it advertises to AS36561, it will leave out
those prefixes.
– YouTube will not have a path across Internet2 CPS to your
downstream’s dorm network. Traffic will not come in that way.
– That dorm network prefix will still be advertised to all other CPS
commercial peers.
Commercial Peering Service
Of course, this is not perfect…
• For example:
– YouTube is now moving behind the Google AS (AS15169).
– CPS has peerings with Google and with YouTube.
– Does this mean you should tag that dorm network prefix with
65000:15169 and 65000:36561?
Maybe, Maybe not.
• Therefore, in some ways, this is better for “traffic
engineering/balancing/management” than to “ensure” certain traffic
doesn’t come across CPS.
• For example, if you already have a decent path to LimeLight, and you
need to keep up a certain minimum bandwidth usage, you might want
to tag all your prefixes with 65000:22822 towards CPS. So CPS will
not advertise your prefixes to LimeLight at all.
Commercial Peering Service
But, I’d like CPS as a backup
• Inbound traffic, outbound route policy:
– If you want this “globally” regarding CPS, you can of course
simply pad towards CPS in hopes of influencing CPS
commercial peers.
– What if you want CPS to be your primary commercial path (for
the routes CPS offers), EXCEPT for traffic from peer Y -- for peer
Y, you want them to send your traffic across CPS only as a “last
resort”?
– The CPS outbound policy specific to each commercial peer
can pad your prefix with the Internet2 AS (AS11537) one, two, or
three times, whichever you choose.
– This may cause peer Y to see the path across CPS to your prefix
as less desirable, leaving it as a backup.
– Note we do NOT do this “per location”.
Commercial Peering Service
Using communities in CPS
• Specifically, if you want traffic from peer network
Y to prefer a different path, but want to use the
Internet2 CPS path as a backup, you can tag
your prefixes with 65001:<bar>, 65002:<bar>, or
65003:<bar> where ”<bar>" is the BGP ASN of
network Y.
• These communities will cause Internet2 to pad
the AS-PATH towards peer network Y 1, 2 or 3
times respectively (using AS11537 for the pad)
for those prefixes you tag.
Commercial Peering Service
Using communities in CPS
• For example:
– Suppose you see traffic from Akamai is preferring Internet2 CPS. For
whatever reason, you’d rather they get to you over a different path,
leaving the CPS path as a backup.
– In your BGP policy with CPS, apply policy outbound that tags your
prefixes with 65001:20940, since 20940 is Akamai’s AS number.
– When CPS processes what it advertises to AS20940, it will pad your
prefixes with one additional “11537” in the AS-PATH.
– If you find that is not enough to influence the inbound traffic from
Akamai, you can successively try 65002:20940 and 65003:20940.
– Your prefixes will not be padded towards any other CPS peers.
– Of course, this is not perfect either, and additionally peers may not be
letting decisions fall to AS-PATH length for their own reasons.
Commercial Peering Service
Using communities in CPS
• And there’s always the blackhole community:
– We check it’s your prefix first of course.
– only allowed /32 to /24.
– 11537:911.
– sets next-hop to discard
Commercial Peering Service
Discussion
• Comments?
• Suggestions?
Commercial Peering Service
Thank you!
http://noc.net.internet2.edu/i2network/commercial-peering-service.html
http://www.internet2.edu/network/cp/
[email protected]
[email protected]
Commercial Peering Service