Transcript Master
Técnicas de Alta Disponibilidade para NAPs Marcelo Molinari – Foundry Networks do Brasil [email protected] © 2002 Foundry Networks, Inc. Agenda • • • • LINX topology overview AMS-IX topology overview Metro Ring Protocol Virtual Switching Redundancy Protocol ©2002 Foundry Networks, Inc. London Internet Exchange (LINX) © 2002 Foundry Networks, Inc. LINX Topology ©2002 Foundry Networks, Inc. LINX Topology • The LINX Network consists of two separate high- performance Ethernet switching platforms installed across seven locations. • Switches from two equipment vendors are deployed in two separate networks to provide an extra level of fault-tolerance, the logic being that both systems shouldn't develop the same fault at the same time. ©2002 Foundry Networks, Inc. LINX Topology • Two switches are installed in every LINX location, and the locations are interconnected by multiple 10 gigabit Ethernet circuits to form two physically separate backbone rings. • Most LINX members connect to both switching platforms, which reduces the impact of any downtime on a single network element. • Management of the logical redundancy of the network is done using MRP (Metro Ring Protocol). In the event of the loss of a network segment, MRP activates a redundant link within tenths of a second and restore connectivity. ©2002 Foundry Networks, Inc. LINX Aggregated Traffic Statistics ©2002 Foundry Networks, Inc. Amsterdam Exchange (AMS-IX) © 2002 Foundry Networks, Inc. AMS-IX Topology ©2002 Foundry Networks, Inc. AMS-IX Topology • AMS-IX is a distributed exchange, currently present at five independent co-location facilities in Amsterdam. • • • The AMS-IX topology is built around two hub/spoke arrangements. • Members with a 10GbE port are connected to Glimmerglass Networks photonic cross-connects. These L1 switches connect the member 10GbE ports to BigIron RX-16 or NetIron MLX-16 switches. The core switches are Foundry Networks NetIron MLX-32 switches. Members connected with GigE, 100Base-TX or 10Base-T ports are connected to Foundry Networks BigIron 15000 and BigIron RX-8 switches. ©2002 Foundry Networks, Inc. AMS-IX Topology • The two core switches run VSRP (Virtual Switch Redundancy Protocol) to define the active hub/spoke and to automatically fail over to the other, based on pre-defined triggers (e.g. link failure). • All edge switches from Foundry follow VSRP automatically. The Glimmerglass switches follow the VSRP failover based on software developed at AMS-IX. • Members can connect to the AMS-IX infrastructure at any of the five AMS-IX co-locations, at 100 Mbit/s, 1 Gbit/s or 10 Gbit/s. ©2002 Foundry Networks, Inc. AMS-IX Traffic Statistics ©2002 Foundry Networks, Inc. Metro Ring Protocol © 2002 Foundry Networks, Inc. Metro Ring Protocol (MRP) • Metro Ring Protocol is a Layer 2 protocol designed to provide SONET-like, high speed, fault tolerant, fast recovery for Metro Ethernet networks. • MRP SONET-like features provide: – Sub-second failover – Efficient use of bandwidth with topology groups (802.1s based) – Scalable protection for multiple VLANs – Large Scale L2 MANs with multi-ring support – Highly flexible network designs – Works with other L2 features • Runs on all Ethernet and PoS/SDH interfaces, including 10 Gigabit ©2002 Foundry Networks, Inc. How it works • A single node is defined as the Ring Master Node • All other nodes are defined as Ring Member Nodes • The Master Node prevents loops by blocking its secondary port • Ring Hello Packets are generated by the Master Node to check ring integrity • As long as the master sees its own Hello packets on the secondary port, ring health is verified, and secondary port remains blocked Primary (Forwarding) Secondary (Blocking) ©2002 Foundry Networks, Inc. Rapid Failover NetIron 400 • • • • • Hello packets are hardware forwarded by the nodes in the ring to ensure fastest failure detection. NetIron 400 Master considers ring broken if no Hello packets are received within 300 ms (3 consecutive Hellos are lost). By changing timers and using messages sent by the node where the ring broke, it is possible to achieve recovery times from 150 ms to 200 ms. NetIron 400 NetIron 400 If no Hellos are received, Master transitions secondary port into forwarding state to restore ring connectivity. To provide reliable flushing of stale MAC entries, Master sends 3 consecutive TCN notifications. FAULT NetIron 400 NetIron 400 NetIron 400 NetIron 400 ©2002 Foundry Networks, Inc. Link Restoration Link restored Preventing Temporary Loops • • • • • When an MRP port goes up, it goes into pre-forwarding mode to avoid the creation of temporary loops. NetIron 400 PF PF NetIron 400 NetIron 400 In pre-forwarding mode the port forwards no data, but only the ring hello packets from the master. NetIron 400 Master sees its own RHP, detects that ring integrity has been restored, puts its secondary port in blocking mode. From that point onwards, master sends RHPs with the Forwarding flag bit set, indicating that members should transition their ports from preforwarding to forwarding. The Forwarding flag bit is always set, as long as the master is blocking its secondary port. NetIron 400 F F NetIron 400 NetIron 400 NetIron 400 ©2002 Foundry Networks, Inc. Topology Groups Topology Group Master VLAN A VLAN running a control protocol (or more) that controls the active topology for the whole topology group. Control protocols: STP, RSTP, MRP, VSRP. Member VLAN A VLAN running NO control protocol of its own but rather follows the active topology of the master VLAN.. Member VLAN Group A group of VLANs running NO control protocol of their own but rather follow the active topology of the master VLAN. VLAN groups are defined via the “vlan-group” command. ©2002 Foundry Networks, Inc. Efficient use of Ring Bandwidth • • MRP Supports Multiple Topology groups within a Ring • • • Each Topology group contains a Master VLAN and Member VLANs An MRP Node can be both a Master node and Member Node for different topology groups Master VLANs generate Hello packets and block secondary ports 4094 VLANs can be divided among up to 255 Topology groups Topo Group 1 Topo Group 2 ©2002 Foundry Networks, Inc. Using Multiple Rings • There are 3 ring scenarios: • • • • • Single ring Rings that don’t overlap Overlapping rings that share links Each Ring runs its own instance of MRP A ring node can be Master for Multiple rings BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 BigIron 4000 Single Ring BigIron 4000 BigIron 4000 Non-Overlapping Ring BigIron 4000 BigIron 4000 Overlapping Rings Phase II Phase I ©2002 Foundry Networks, Inc. Example Scenarios – Phase I – High-speed 10 GE trunks for Metro rings or IXPs – Provides sub-second fault-detection and fail-over – Superior scalability: no limit on maximum number of nodes per ring – Counter rotating topology groups provide efficient use of bandwidth BigIron 4000 BigIron 4000 BigIron 8000 S6 Secondary Ring 2 port BigIron 4000 S1 Master node Primary BigIron 4000 port S5 Ring 1 BigIron 8000 S2 BigIron 4000 BigIron 4000 S5 BigIron 4000 Ring 3 S4 BigIron 4000 S4 S3 BigIron 4000 Ring 4 BigIron 4000 Master node BigIron 4000 ©2002 Foundry Networks, Inc. Example Scenarios – Phase II – Shared ring support – Increased reliability – Increased bandwidth BigIron 4000 BigIron 4000 BigIron 8000 S6 Secondary Ring 2 port BigIron 4000 S1 Master node Primary BigIron 4000 Master node S5 BigIron 4000 port Ring 1 BigIron 8000 S2 BigIron 4000 BigIron 4000 S5 BigIron 4000 Ring 3 S4 BigIron 4000 S4 S3 BigIron 4000 Ring 4 BigIron 4000 Master node BigIron 4000 ©2002 Foundry Networks, Inc. Interface Flexibility • Support for Mixed interfaces - 10Gig & Gig - Gig & 10/100 - 10Gig & PoS/SDH • Support for Trunked interfaces - 10Gig & Gig - PoS/SDH Slower link BigIron 8000 S6 BigIron 8000 S6 Secondary Secondary port port BigIron 4000 S1 Primary BigIron 4000 S1 Primary port S5 BigIron 8000 BigIron 8000 S2 BigIron 4000 Master node BigIron 4000 BigIron 4000 port S5 Master node S2 BigIron 4000 S4 S4 BigIron 4000 BigIron 4000 S3 S3 ©2002 Foundry Networks, Inc. MRP – Summary of Benefits • • • • • • Fast, sub-second, predictable fail-over functionality Maximizes ring bandwidth utilization Cost effective scalable solution for MAN resiliency Attractive alternative to STP Utilizes 802.3 Ethernet Packet standards and MACs Can be combined with other Foundry features to provide complete end to end MAN designs ©2002 Foundry Networks, Inc. Virtual Switch Redundancy Protocol © 2002 Foundry Networks, Inc. Virtual Switch Redundancy Protocol • VSRP provides an alternative to Rapid Spanning Tree Protocol (RSTP) in dual homed/mesh configurations, providing sub-second fail-over and recovery. • VSRP features provide: – Sub-second fail-over – Efficient use of mesh bandwidth – no blocked links – Block and unblock ports at the per-VLAN group level – Large scale L2 MANs with multi-tiered support – Highly flexible network designs – Configurable tracking options – Works with other L2 features • • Works with all Ethernet interfaces, including 10 Gigabit VSRP is based on VRRP-E & can provide L2 and L3 backup ©2002 Foundry Networks, Inc. How it works • • VSRP uses an election process to select a Master switch and up to 4 backup switches for each VLAN: higher configured priority wins; if equal, higher IP address wins. VSRP Master VSRP Backup BigIron 8000 Only the Master switch forwards data, while Backup switches block traffic on all VSRP configured interfaces within the VLAN (or the topology group). • Master switch sends Hello packets to all backup switches • Switches do not have to be VSRP aware. VSRP aware provides faster failover. • VSRP can track ports and decrease the priority of VSRP active switch in case a tracked port goes down. BigIron 8000 S1 S2 F F BigIron 4000 S3 B F BigIron 4000 VSRP Aware S4 B B BigIron 4000 S5 ©2002 Foundry Networks, Inc. Rapid Failover Master • A VSRP Backup switch monitors Hellos from the Master. • If no Hellos are received for Master Dead Interval (default 300 ms), Backup goes into Hold Down state, starts sending periodic Hellos. • Hold Down interval is by default 300 ms, and it allows for the election of a new master. • If the switch is elected as Master, it sets its port into forwarding state, sends 3 TCNs. • A VSRP aware switch receives TCN, and looks for the new master. Hellos of the new master will be received on a different port. • Backup NetIron 800 A VSRP aware switch shifts the MAC addresses learned on the failed port to the new port. B FAULT F Backup NetIron 800 NetIron 800 B NetIron 400 Master NetIron 800 Dn Backup NetIron 800 FAULT F NetIron 800 B NetIron 400 Mac Type:D = Dynamic, S = Static, H = Host, R = Router MAC Port Age Type 0060.f320.23a8 2 0 D 0030.1b07.0694 2 0 D 00d0.b758.88dc 2 0 D 0004.8039.5f00 2 0 D ©2002 Foundry Networks, Inc. Link Restoration (Orig. Master) Backup Switching Back to Original Master • • • • • • • Link restored Master NetIron 800 When the failed link is restored, the original Master remains as a Backup. Original Master receives inferior Hello from the current Master, so it immediately replies with its own Hello, switches into Hold Down state (300 ms), starts sending periodic Hellos Current Master receives superior Hello, so it switches into Backup mode. If no superior Hellos are received during Hold Down interval, original Master considers itself the new current Master, and sets its port in Forwarding mode. New Master sends out 3 TCNs. A VSRP aware switch receives TCN, and looks for the new master. Hellos of the new master will be received on a different port. A VSRP aware switch shifts the MAC addresses to the new port. F B Backup NetIron 800 NetIron 800 B NetIron 400 Master Backup NetIron 800 F Backup NetIron 800 B NetIron 800 B NetIron 400 Mac Type:D = Dynamic, S = Static, H = Host, R = Router MAC Port Age Type 0060.f320.23a8 1 0 D 0030.1b07.0694 1 0 D 00d0.b758.88dc 1 0 D 0004.8039.5f00 1 0 D ©2002 Foundry Networks, Inc. Efficient use of Uplink Bandwidth • • VSRP supports topology groups to fully utilize switches and links Topology groups are a collection of VLANs • Each yopology group contains a Master VLAN and Member VLANs • VSRP configured switches can be Master for some topology groups while backup other for others • 4094 VLANs can be divided among up to 255 Topology groups Master topology group 1 Backup topology group 2 BigIron 8000 S1 BigIron 4000 S3 Master topology group 2 Backup topology group 1 Hello Packets BigIron 8000 S2 BigIron 4000 VSRP Aware S4 BigIron 4000 S5 Topology group 1 = Master VLAN 1 Member VLANs 2 to 2048 Topology group 2 = Master VLAN 2049 Member VLANs 2050 to 4096 ©2002 Foundry Networks, Inc. VSRP Domains • • • • • VSRP can be configured in separate domains within the same VLAN to allow for larger topologies. NetIron 800 Topology groups can be designed to use unique paths in each domain. VSRP Domain 1 (VRID 1) A TTL value within the VSRP Hello packet controls how deep the packet goes into the network. VSRP Domain 2 (VRID 2) TTL is being decremented by 1 at each VSRP aware switch. Default TTL is 2, which allows Hello to traverse one VSRP aware switch to go to another VSRP active switch. VSRP Domain 3 (VRID 3) NetIron 800 NetIron 800 VSRP Active NetIron 400 NetIron 400 VSRP Aware VSRP Active NetIron 400 NetIron 400 VSRP Aware VSRP Active NetIron 400 NetIron 400 VSRP Aware ©2002 Foundry Networks, Inc. Intelligent Port Level Control • VSRP can be configured to run only on designated ports • Only VSRP configured ports are placed in blocking for Backup switches • Supports host attachments and dual port NICs: host facing ports are configured as VSRP free. • NetIron 800 NetIron 800 MRP Master NetIron 800 Works in combination with MRP to provide flexible Metro / Enterprise Ethernet designs. Backup NetIron 800 NetIron 400 NetIron 400 VSRP Aware Dual Homed Servers ©2002 Foundry Networks, Inc. VSRP-Aware Switches • • • Both VSRP-Aware and Non VSRPAware switches can be used as edge devices VSRP-Aware switches recognize the VSRP Hello packet sent by the Master, and then create an table which contains the VRID of the VLAN which sent the VSRP Hello plus the incoming port where the VSRP Hello was received. When the VSRP-Aware switch sees a Hello packet coming from a different port, it quickly moves the MAC address table entries to the new port. VSRP Master VSRP Backup BigIron 8000 BigIron 8000 S1 S2 F F BigIron 4000 S3 VSRP Aware B F BigIron 4000 S4 VSRP Aware B B BigIron 4000 S5 VSRP Aware ©2002 Foundry Networks, Inc. Non VSRP-Aware Switches • • Non VSRP-Aware switches can be used as edge devices, but they do not recognize VSRP Hello packets. MAC entries will age out or they will eventually be learned from the new port. • This results in slow convergence when the Master fails. • Solution is to configure “VSRP Fast Start” in the VSRP Master and Backup nodes. • VSRP Master VSRP Backup BigIron 8000 BigIron 8000 S1 S2 F F BigIron 4000 S3 Non VSRP Aware B F BigIron 4000 S4 Non VSRP Aware B B BigIron 4000 S5 Non VSRP Aware VSRP Fast Start disables and re-enable the ports before transitioning from Master to Backup. This causes a MAC address flush in the edge devices which makes convergence faster. ©2002 Foundry Networks, Inc. VSRP – Summary of Benefits • • Fast, sub-second protection without Spanning Tree • • Supports topology groups for full link utilization Combines both switching and routing redundancy – Provides default gateway redundancy if needed Can be combined with other Foundry features to provide complete end to end MAN designs ©2002 Foundry Networks, Inc. References • • • • • LINX - https://www.linx.net/ AMS-IX - http://www.ams-ix.net/ “I can feel your traffic” - http://www.jasinska.de/ MRP- http://www.foundrynet.com/pdf/wp-mrp.pdf VSRP - http://www.foundrynet.com/pdf/wp-vsrp.pdf ©2002 Foundry Networks, Inc. Thank You ! Marcelo Molinari – Foundry Networks do Brasil [email protected] © 2002 Foundry Networks, Inc.