Transcript No Slide Title

How to Secure Your Job:
Implement MPLS VPNs
[email protected]
NANOG24 / Miami, Florida
February 10-12, 2002
Agenda
• The Idea behind this Talk
• Lessons Learned
• The Service Architecture
• The Use of Route Reflectors
• Baby Jumbo Frames
• The LDP-ID
• Traceroute is no longer Your Friend
• Some Security Considerations
– no Religious Wars please :-)
2
Agenda
• The Idea behind this Talk
• Lessons Learned
• The Service Architecture
• The Use of Route Reflectors
• Baby Jumbo Frames
• The LDP-ID
• Traceroute is no longer Your Friend
• Some Security Considerations
– no Religious Wars please :-)
3
The Idea behind this Talk
• This talk should show some lessons we learned when
implementing and running a real life MPLS VPN network
• Nextra Switzerland is running a national MPLS VPN
network since 2.5 years (Q3 1999)
– about 30 PoPs, Cisco based
• The Nextra Group is running a pan-European MPLS VPN
network since about one year (Q1 2001)
– present in most European countries
– provides seamless VPN services between autonomous Nextra
companies
• I will not discuss whether MPLS is a good thing or not :-)
4
Agenda
• The Idea behind this Talk
• Lessons Learned
• The Service Architecture
• The Use of Route Reflectors
• Baby Jumbo Frames
• The LDP-ID
• Traceroute is no longer Your Friend
• Some Security Considerations
– no Religious Wars please :-)
5
Lessons Learned
A reload a day
keeps the TAC away
just kidding…
6
Agenda
• The Idea behind this Talk
• Lessons Learned
• The Service Architecture
• The Use of Route Reflectors
• Baby Jumbo Frames
• The LDP-ID
• Traceroute is no longer Your Friend
• Some Security Considerations
– no Religious Wars please :-)
7
Service Architecture
• Is entirely based on MPLS VPN ‘Routing Realms’
• Each Added Value Service has its own Routing Realm/VPN
• Each Customer has several Routing Realms/VPNs
• Customers are given access to Services by
interconnecting ‘Customer’ VPNs and ‘Service’ VPNs
• Limited Set of Building Blocks
–
–
–
–
–
8
Full Mesh Routing Realm
Hub/Spoke Realm
(Point-Point Realm)
Inter-Realm ‘Conduits’
Inter-Realm ‘Adapters’
Lego-Brick #1: Full-Mesh VPN
9
Lego-Brick #2: Hub & Spoke VPN
Hub
Spoke
10
Spoke
Spoke
Lego-Brick #3: Conduits
Resource
X Mbps
11
Lego-Brick #4: Adapters
Resource
12
The Four Network Building Blocks
13
Multiple VPNs per Customer
14
Managed Firewall IAS for PN
Internet
Nextra Managed Firewall
15
Standard VPN IAS
Internet
Customer Managed Firewall
16
Redundant Load Sharing PN
HSRP
POP
17
Redundant
Access
Disaster Recovery VPN
Primary
Computing Center
(IP: 10.5.5.0/24)
POP A
18
Fast (?)
Re-routing
Backup
Computing Center
(IP: 10.5.5.0/24)
POP B
PN Dial Backup
VPDN Home Gateway
ISDN
19
PartnerNet
PartnerNet
Customer
A
20
Customer
B
Agenda
• The Idea behind this Talk
• Lessons Learned
• The Service Architecture
• The Use of Route Reflectors
• Baby Jumbo Frames
• The LDP-ID
• Traceroute is no longer Your Friend
• Some Security Considerations
– no Religious Wars please :-)
21
The Use of Route-Reflectors
• You do not want to maintain a full VPNv4-iBGP mesh
• Use at least two VPNv4-RRs for redundancy reasons
• Do not use the IPv4-RRs as VPNv4 Route-Reflectors
– IPv4 Routing instabilities could affect VPN routing
– If PE reloads it will get all 100’000 IPv4 routes first. So it will take
several minutes until the VPN routes are restored
– If RRs are separated, both feeds are done simultaneously
– So you need at least four RRs --> what about core routers?
• Use global-local-massive-extreme unique RDs
– A different RD per VRF, not per VPN
– otherwise RR will break iBGP load-sharing
• Check next-hops of BGP routes you get from the RR :-)
22
Agenda
• The Idea behind this Talk
• Lessons Learned
• The Service Architecture
• The Use of Route Reflectors
• Baby Jumbo Frames
• The LDP-ID
• Traceroute is no longer Your Friend
• Some Security Considerations
– no Religious Wars please :-)
23
Baby Jumbo Frames
• MPLS labels are 4 Bytes long
• Available MTU on a link gets decreased by x*4 Bytes
• Due to Path-MTU-Discovery, the sender will detect this
• Path-MTU-Discovery seems to be broken sometimes
• Customer will blame you, not the broken PMTUD
– “everything worked fine before we moved to your MPLS network!”
• Workaround:
– allow 1500+(x*4) Bytes packets over Ethernet (Baby Jumbo Frames)
– Special config command: tag-switching mtu 1508
– oversized packets only allowed if oversize is caused by MPLS labels
• Before you migrate to MPLS, make sure all your routers,
interfaces and switches support Baby Jumbo Frames!
24
Agenda
• The Idea behind this Talk
• Lessons Learned
• The Service Architecture
• The Use of Route Reflectors
• Baby Jumbo Frames
• The LDP-ID
• Traceroute is no longer Your Friend
• Some Security Considerations
– no Religious Wars please :-)
25
The LDP-ID
• After a crash, a P router stopped forwarding any traffic...
interface loopback99
description test test test
ip address 222.222.222.222 255.255.255.255
• Like in OSPF, LDP uses highest loopback address as ID
• Unlike in OSPF, LDP-ID has to be reachable
LDP-ID not reachable  no LDP TCP-session  no labels!
• So always configure the LDP-ID manually!
26
Agenda
• The Idea behind this Talk
• Lessons Learned
• The Service Architecture
• The Use of Route Reflectors
• Baby Jumbo Frames
• The LDP-ID
• Traceroute is no longer Your Friend
• Some Security Considerations
– no Religious Wars please :-)
27
Traceroute is no longer Your Friend
Traceroute stops at PE router !
28
A Multi ISP Environment
Trans
it ISP
ISP B
ISP A
Traceroute stops at PE router !
29
Customer Complains about Strange Traceroute
CPE-Router#traceroute www.cisco.com
1 accesslink.company.com (10.150.73.17) 0 msec 4 msec 0 msec <-- PE (Europe)
2 r16b01-pos0-0-0.bb.nextra.net (141.22.5.2) 104 msec 104 msec 100 msec <-- P (Europe)
3 r08b02-pos1-0-0. bb.nextra.net (141.22.1.6) 104 msec 104 msec 104 msec <-- P (Europe)
4 r08b01-fe4-1-0. bb.nextra.net (141.22.5.49) 148 msec 200 msec 200 msec <-- P (Europe)
5 r10b01-pos4-0-0. bb.nextra.net (141.22.5.34) 160 msec 200 msec 200 msec <-- PE (USA)
6 Pos4-0.GW8.NYC4.ALTER.NET (157.130.21.37) 100 msec 100 msec 100 msec
7 172.ATM2-0.XR2.NYC4.ALTER.NET (146.188.180.62) 104 msec 104 msec 104 msec
8 188.at-1-0-0.TR2.NYC8.ALTER.NET (152.63.21.106) 108 msec 104 msec 104 msec
9 124.at-6-0-0.TR2.SAC1.ALTER.NET (152.63.6.14) 180 msec 176 msec 180 msec
10 190.ATM6-0.GW8.SJC2.ALTER.NET (152.63.52.181) 184 msec 188 msec 184 msec
11 cisco.customer.alter.net (157.130.200.30) 188 msec 188 msec 188 msec
12 pigpen.cisco.com (128.107.240.170) 188 msec 184 msec 184 msec
13 * www.cisco.com (198.133.219.25) 184 msec *
30
What a Professional Troubleshooter Told Us :-)
Hi,
I think I can explain what is happening. Any IP packets received with IP
options that require processing are process switched. That means that the
packet is sent to the Route Processor for forwarding, rather than switched
directly using CEF. If the RP is busy, then this may add to the latency.
The traceroute command uses IP options in the header, so these packets
will be process switched. Any other traffic, such as http, ftp etc will be
switched using CEF. So when they run the traceroute command, they are
seeing the latency of the packet being forwarded through the RP, rather
than the true latency of the packets being fast switched by CEF.
Regards,
Your professional troubleshooter
31
The Real Reason
• PE sends back ICMP Time Exceeded via appropriate
Routing Table
• P router has no routing information!
– Only information he has is receiving LSP, but LSPs are unidirectional
– So P router sends ICMP Time Exceeded via receiving LSP
– ICMP packet travels through the MPLS backbone to the other PE,
which then has the routing information to send the ICMP packet back
through another LSP to the sender
– So each traceroute probe travels through the whole MPLS backbone
twice
– TTL is high!
• Customer can accept this because his productive traffic
is not affected
32
But How to Explain this to a Customer?
CPE-Router#traceroute www.cisco.com
1 accesslink.company.com (10.150.73.17) 0 msec 4 msec 0 msec <-- PE (Europe)
2 r16b01-pos0-0-0.bb.nextra.net (141.22.5.2) 104 msec 104 msec 100 msec <-- P (Europe)
3 r08b02-pos1-0-0. bb.nextra.net (141.22.1.6) 15 msec 12 msec 22 msec <-- P (Europe)
4 r08b01-fe4-1-0. bb.nextra.net (141.22.5.49) 16 msec 17 msec 14 msec <-- P (Europe)
5 r10b01-pos4-0-0. bb.nextra.net (141.22.5.34) 160 msec 200 msec 200 msec <-- PE (USA)
6 Pos4-0.GW8.NYC4.ALTER.NET (157.130.21.37) 100 msec 100 msec 100 msec
7 172.ATM2-0.XR2.NYC4.ALTER.NET (146.188.180.62) 104 msec 104 msec 104 msec
8 188.at-1-0-0.TR2.NYC8.ALTER.NET (152.63.21.106) 108 msec 104 msec 104 msec
9 124.at-6-0-0.TR2.SAC1.ALTER.NET (152.63.6.14) 180 msec 176 msec 180 msec
10 190.ATM6-0.GW8.SJC2.ALTER.NET (152.63.52.181) 184 msec 188 msec 184 msec
11 cisco.customer.alter.net (157.130.200.30) 188 msec 188 msec 188 msec
12 pigpen.cisco.com (128.107.240.170) 188 msec 184 msec 184 msec
13 * www.cisco.com (198.133.219.25) 184 msec *
BUG!
33
Agenda
• The Idea behind this Talk
• Lessons Learned
• The Service Architecture
• The Use of Route Reflectors
• Baby Jumbo Frames
• The LDP-ID
• Traceroute is no longer Your Friend
• Some Security Considerations
– no Religious Wars please :-)
34
Configuration Integrity
• Sometimes config lines for a vrf routing enviroment
appear on the global routing process level
– Some buggy software versions
– Typo in vrf name
– Copy-Paste works too fast :-)
• This can be very very very dangerous
• Sometimes someone puts a interface into a wrong VRF :-(
• The problem is not when you break connectivity (easy to
spot) but when you add too much connectivity
• Sanity Check Tools!
35
Configuration Integrity
• How it should look like
• What I found on a router
router bgp 88
router bgp 88
[usual iBGP stuff]
redistribute rip
no auto-summary
default-information originate
!
address-family ipv4 vrf green
redistribute connected
redistribute rip
neighbor 211.27.213.9 remote-as 65500
neighbor 211.27.213.9 activate
default-information originate
no auto-summary
no synchronization
exit-address-family
36
Configuration Integrity
• How it should look like
• What I found on a router
router rip
router rip
version 2
redistribute bgp 88 metric 3
no auto-summary
network 211.27.213.0
!
address-family ipv4 vrf blue
version 2
redistribute bgp 88 metric 3
network 211.27.213.0
• Imagine what happens if you use
distribute-list 22 in
addresses from the 62/8 block!
no auto-summary
• Another reason for good filtering
exit-address-family
• (Preconfigure all possible redistributions with route-map “deny-all”)
37
Secure Your PE VTYs!
line vty 0 4
access-class 99 in
login authentication bbmethod
access-list 99 permit 10.25.25.5 0.0.0.0 <-- NMS box
access-list 99 permit 61.8.0.0 0.0.7.255 <-- BB-links
• What happens, if customer sends hostroute
for your NMS box?
• He will be able to telnet to your PE router!
38
Secure Your PE VTYs!
• only use hostroutes for NMS
– RIP, OSPF, EBGP has better admin distance than IBGP:-(
• put hostroutes to Null0 for NMS into all VRFs
– how to manage the CPEs?
• filter routing updates from customer
– do not accept all internal address ranges (NMS, BB)
– secure, but could impact customer connectivity
39
Secure Your PE VTYs!
• use better ACLs to secure the VTYs
interface loopback0
ip address 1.1.1.1 255.255.255.255
access-list 199 permit 10.25.25.5 0.0.0.0 1.1.1.1 0.0.0.0
access-list 199 permit 61.8.0.0 0.0.7.255 1.1.1.1 0.0.0.0
access-list 199 permit 10.25.25.5 0.0.0.0 61.8.0.0 0.0.7.255
access-list 199 permit 61.8.0.0 0.0.7.255 61.8.0.0 0.0.7.255
• needs entries for BB interface addresses as telnet
destination (hop-to-hop telnet)
– as BB interface addresses and loopback addresses are
not reachable within the vrfs, this should be secure
– needs a nice IP addressing scheme (or a clever config generation tool)
40
That’s it!
Thank you for your attention
Now, it is time for
–
–
–
–
41
questions
comments
corrections
flames
How to Secure Your Job:
Implement MPLS VPNs
[email protected]
NANOG24 / Miami, Florida
February 10-12, 2002