Transcript backbone-intro

Implementing and Maintaining
an ISP Backbone
Kevin Butler
Sprint
Network
Seattle
Tacoma
Click here for a closer look at the
Sprint network in Washington state
Stockton
San Jose
Click here for
a closer look
at the Sprint
network in
Northern
California
DS3
OC3
OC12
OC48
Legend
Cheyenne
Kansas City
Click here for
a closer look
at the Sprint
network on the
East Coast
New York
Pennsauken
Relay
Wash. DC
Chicago
Roachdale
Anahei m
Atlanta
Pearl Ci ty in Hawaii is
a future network location
Fort Worth
Orlando
Tier 1 ISP Backbones
• Comprise some of the world’s largest IP
networks
• Tier 1 companies include Sprint, AT&T,
PSINet
• UUNET has the world’s largest IP data
network, presence on four continents and
future expansion into Latin and South
America
Service Level Agreements
• SLAs are an important and prestigious tool
in attracting and maintaining customers
• Comprised of uptime guarantees and
bounds on latency through various
geographic regions
• most ISPs currently have latency < 50ms
across the US
Supporting the Customer
• Quality and expertise of first-line customer
support varies wildly between companies
• depending on size, geographic location and
company focus, some front-line support
teams outsourced to third parties
• some in-house high level support teams
have skills equivalent or superior to NOCs
Network Operations Centres
• Generally the teams concerned with
backbone maintenance and support
• trend towards consolidation into “SuperNOCs” (eg. one for Americas, one for
Europe)
• specialisation within NOC for product
support (eg. dial, VPN, backbone NOCs)
NOC Tools
• NOCOL - Network Operations Centre On
Line (freeware UNIX)
• Mediahouse monitoring (mainly web)
• Micromuse Netcool (now owned by Lucent)
- used by MCI WorldCom, PSINet, BT
Dial Access
• Dial is a major selling point, especially with
customers who travel a lot or are their own
ISPs
• connections made through an Ascend MAX
TNT, which can support up to 720
concurrent callers
• back-end is a DS-3 into a backbone router,
routers advertised by an IGP (eg. RIP)
Dial-Related Technologies
• COBRA (Central
Office Based Remote
Access) allow building
of virtual POPs by
backhauling PRIs
• RADIUS (Remote
Authentication Dial In
User Service)
Integrated Services Digital
Network
• ISDN customers authenticate by RADIUS
similar to dial users
• underlying architecture similar but dial
equipment often administrated differently
• ISDN maintained within same AS as
backbone whereas dial often in its own AS
DS-1 and high-speed access
• Customer connections usually multiplexed,
come into DSU as a channelised DS-3
• gateway routers on ISP side usually Cisco
7500 series, increasingly using Cisco 12000
• customers connect using Cisco 1604, 2621,
some 3600 series, very large customers use
7500 series routers
Gateway Routers
• obtain routes from
customers usually
statically, but
sometimes by BGP
• usually run link-state
IGP within AS (eg.
OSPF, IS-IS)
• Cisco 7513 backplanes
1.8 Gbps while 12008
does 40 Gbps
Where does traffic go from
here?
• Most ISPs have two levels of networks
above the access router
• Metropolitan networks aggregate gateway
traffic, generally city-wide (if multiple
POPs in city)
• transit networks aggregate metro network’s
traffic, responsible for inter-city transport
ATM Switches
• Terminate long-haul
OC-12, OC-48 circuits
and metro rings
• Choice of vendor
contingent on ISP,
commonly Newbridge,
Fore Systems (ASX1000 and ASX-4000)
Example of an ATM interface
TR1.EG1:
interface ATM2/0
description To HA13.BLAH1 3C1
atm vc-per-vp 512
atm pvc 16 0 16 ilmi
!
interface ATM2/0.195 point-to-point
description To XR1.BLAH1 ATM6/0
ip address 146.188.200.98 255.255.255.252
ip router isis Net-Backbone
atm pvc 195 0 195 aal5snap
clns router isis Net-Backbone
Implementation of BGP
• BGP run between autonomous systems and
peers, as well as multi-homed customers
• monolithic AS broken up into BGP
confederations for ease of work
• routes controlled using access lists and
route maps
BGP
• Communities are destinations that share
common attributes (eg. through access-list
filters)
BGP table version is 23718690, local router ID is 205.150.242.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete
Network
*>i24.64.0.0/19
*>i24.64.0.0/14
*>i24.64.32.0/19
*>i24.64.64.0/19
*>i24.64.96.0/19
*>i24.64.192.0/19
*>i24.64.224.0/19
*>i24.65.0.0/19
*>i24.65.96.0/19
*>i24.65.128.0/19
Next Hop
198.133.49.7
198.133.49.7
198.133.49.7
198.133.49.7
198.133.49.7
198.133.49.7
198.133.49.7
198.133.49.7
198.133.49.7
198.133.49.7
Metric LocPrf Weight Path
100
0 6327 6172
100
0 6327 i
100
0 6327 6172
100
0 6327 6172
100
0 6327 6172
100
0 6327 6172
100
0 6327 6172
100
0 6327 6172
100
0 6327 6172
100
0 6327 6172
i
i
i
i
i
i
i
i
i
Advantages of BGP for User
• Allows for load-sharing and redundancy
• routes can be biased through AS path
prepending
• requirement is high-quality router with
close to 100% uptime to avoid connection
flaps and subsequent route dampening
Common Customer Issues
• Static routes on backbone - often difficult to
spot, can cause very strange routing results
• pull-up routes for netblocks smaller than
/24, required to avoid BGP dampening
• BGP recalculations - if done on a transit
router, entire backbone segments can
experience outages
Customer Requirements of
the Backbone
• Redundancy - networks are redundant but
card failures can take down whole routers
• physical connection to POP from customer
is SPF
• low latency - massive increases in demand
on backbone makes this difficult
• over $2 million a day spent on global
backbone upgrades
DSL: low cost, high speed
• DSL might phase out ISDN connections
• difficult to troubleshoot from network
standpoint
• connections pass through telco’s frame or
ATM cloud between DSLAM and VR
• RedBack SMS (Subscriber Management
System) 1000 commonly used as VR
RedBack SMS 1000
• Supports up to 4000 sessions
• OC-3 out to metro network
• traffic-shaping accomplished with profiles
atm profile samplecust
counters
shaping vbr-nrt pcr 1000 cdvt 100 scr 100 bt 10
Increasing Capacity
• Backbone capacity increasing at a huge rate
• Traffic engineering combined with high
backplane becoming increasingly important
• many ISPs turning to Juniper routers
• UUNET rolled out production OC-192c
with Juniper M160 running MPLS
Juniper Routers
• JUNOS supports
MPLS and RSVP
isis {
interface all;
}
ospf {
area 0.0.0.0 {
interface so-0/0/0 {
metric 15;
retransmit-interval 10;
hello-interval 5;
}
}
}
[edit]
Distributed DOS attacks
• Can be very detrimental to backbone (even
causing switch crashes)
• Combated by rate-limiting ICMP on routers
• Most effective defense is community-wide
egress filtering; requires co-operation
throughout the Internet
Canadian Network Challenges
• Geographically, population resides in
virtually a straight line across the south
• major focus is on southbound capacity to
the US
• CRTC regulations on telcos create different
arrangements
• heterogeneous network to the US,
integration a big issue
Questions?
• Anything I can clarify or expand on...
• Thank you!