Transcript ATF_Best Practice Design for an Avaya Fabric Connect Campus

March 25 – 27th, 2014 І Orlando, FL
Best Practice Design for
Campus Networks
Steve Emert, Avaya
Keith Nuehring, City of Cedar Rapids
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
#AvayaATF
Do
Do
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
Best Practice Design
for Campus Networks
City of Cedar Rapids
Fabric Connect Case
Study
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
#AvayaATF
Agenda
 Campus Best Practices Design with Fabric Connect




Deploying Fabric Connect in the Campus
Design Options with Compact Form Factor Switches
Capabilities to Ease Integration with Conventional Networks
Models to Extend the Fabric to the Wiring Closet Edge
 City of Cedar Rapids, Iowa case study
Keith Nuehring, IT Operations Manager, City of Cedar Rapids






Cedar Rapids network before SPB and Fabric Connect
Considerations and the decision to move to Fabric Connect
Network Redesign Goals and Objectives
Planning and Staging the Network Cutover
Cutover Weekend
Observations and Lessons Learned
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
3
Best Practice Network Designs for the Campus
 First…. The sales pitch… not that you haven’t heard it before!
 Use Fabric Connect!
 Whether a small/medium enterprise, large campus network, or a campus
network that is distributed across a city or even a larger geography
 Why?
 It will make your life easier
 It will free up your time and your staff’s time to work on more strategic
projects
 It will allow you to support new initiatives
improving your business – improve
time to service
 It will help simplify your efforts to
maintain PCI DSS or other security
compliance requirements or regulations
 It will save you time and money in
operational expenditures
 It will make your network run more smoothly
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
4
Deploying Fabric Connect in the Campus
 What SPB Network services to use, and where …
 Layer 2 VSNs
BEB
VLAN
200
BCB
BCB
I-SID
200
BEB
I-SID
200
VLAN
200
 Stretching VLANs across the campus
 Rack to Rack, Row to Row, Data Center to Data Center for VM
Moves
 Special purpose L2 networks
 Totally constrained networks if no IP interface created on BEBs
 Routable if IP interface configured
 STP BPDUs not transported across ISID
 STP becomes a “local construct” only for edge protection
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
5
Deploying Fabric Connect in the Campus
 What SPB Network services to use, and where …
 IP Shortcuts (GRT Route Redistribution to ISIS)
BEB
BCB
BCB
VLAN
100
VLAN
200
BEB
VLAN
300
Route redistribution
across ISIS
VLAN
400
 One-hop IP routing across the fabric
 Eliminates “transit” IP Subnets, simplifying the routing table
 Enhances security – end users cannot determine “core” routers by
workstation Traceroute
 Typical Uses
 Simplest migration from conventional IP routed network to Fabric
Connect
 Simply enable route redistribution to SPB/ISIS
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
6
Deploying Fabric Connect in the Campus
 What SPB Network services to use, and where …
 Layer 3 VSNs
BEB
BCB
BCB
BEB
VLAN
200
VLAN
100
I-SID
500
I-SID
500
VRF
VRF
VLAN
300
 Multiple isolated/segregated IP routed networks within a single fabric
infrastructure
 More efficient than conventional VRFs with a single instance of the routing
protocol (ISIS for SPB), single LSDB with info for all VRFs
 Typical Uses
 Fully routed private networks for security segregation/isolation
 PCI DSS, IP Video surveillance networks, SCADA or HVAC
 Mergers and acquisitions, partnerships – duplicated IP addresses are
allowed when in different VRFs
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
7
Deploying Fabric Connect in the Campus
 What SPB Network services to use, and where
NNI
IP
Shortcuts
VSP 8284
NNI
IP
Shortcuts
IP
Shortcuts
NNI
NNI
L2 VSN
IP Shortcuts
VSP 4450 /4850
VSP 7024
Q Tagged
Q Tagged
IP
Shortcuts
L2 VSN
VSP 7024
Q Tagged
L2 VSN
ERS 4800
Unicast only
ERS 4500
or
4500/4800
ERS 4800
Multicast
Clients
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
ERS 5000
8
Design Options with Compact Form Factor Switches
Port Capacity
 VSP 8284
Fabric Services
80 10Gig, 4 40Gig
BCB, L2 VSN,
IP Shortcuts,
L3 VSN in 2015
24 10Gig (SFP+,
10GBase-T), 8 port
MDA
BCB, L2 VSN
 VSP 4850

48 UTP
10/100/1000,
2 shared SFP,
2 10Gig SFP
BCB, L2 VSN,
IP Shortcuts,
L3 VSN
 VSP 4450
12 UTP, 36 SFP,
2 10Gig SFP
BCB, L2 VSN,
IP Shortcuts,
L3 VSN
L2 VSN only
 ERS 4800
48 UTP 10/100/1000
PoE+ or non PoE
Stackable
 VSP 7024
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
9
Fabric Connect Deployment Best Practices
 Establish a consistent, understandable and useful
naming/numbering system
 Backbone VLAN IDs 4051 and 4052 – per informational RFC
 System IDs – 00xx.xxxx.xx00
 Stay away from first two hex digits and last two hex digits
 Building or network . Subnet or other identifier . Switch number
in location
 Example: 0001.07a1.0100
 Nick-Names (ISIS Source Address) – x.xx.xx
 Base it on the System ID for ease of correlation
 Example: 1.a1.01
 System names (CLI Prompt names)
 Meaningful, avoid special characters – make it easy to type!
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
10
Fabric Connect Deployment Best Practices
 Ensure STP/MSTP/RSTP is disabled on NNI ports
 Default behavior on most switches
 Faster convergence when STP does not interfere with NNI
 Avoid putting CVLANs on NNI ports
 Except where needed during conventional network to Fabric
migration
 Network Edge connectivity and protection
 Spanning Tree Fast Learning or MSTP Edge-Port
 SLPP and SLPP Guard
 Can use SLPP even in some non-traditional locations
 VLACP
 Use is optional, but can improve NNI failover times
 Can specify Ethertype to be unique for the specific link
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
11
Capabilities to Ease Integration with Conventional Networks
 Current capabilities
 Split BEB for SMLT compatibility
 802.1Q tagged interfaces to non-SPB switches
 Transparent UNI
 Route redistribution between ISIS/SPB and other IGPs
 Upcoming capabilities
 Virtual IST
 Fabric Attach
 Fabric Connect over IP SFP Adapter
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
12
Route Redistribution between SPB/ISIS
and Other IGPs
RIP or OSPF used to exchange
routes with external routing
switch
Redistribute RIP or
OSPF to ISIS
OSPF/RIP/BGP enabled on external facing interfaces.
Redistribute ISIS Routes into other IGPs:
ip <rip|ospf|bgp> redistribute isis <create|enable|apply>
Redistribute direct/static/other IGP routes into ISIS:
ip isis redistribute <direct|static> <create|enable|apply>
ip isis redistribute <rip|ospf|bgp> <create|enable|apply>
Layer 3 Switch
Avaya SPB
Domain
Layer 2 Switch
IP Routed
Domain
Redistribute
Direct to ISIS
Layer 3 Switch
Avaya SPB
Domain
Edge VLANs/IP Subnets are present on
VLAN interfaces within the SPB BEB
switch, associated with ISIDs
Layer 2 Switch
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
13
Virtual IST for SMLT (vIST)
 Virtual IST Concepts
 Delivers SMLT/RSMLT w/ virtual IST capability.
 Ability to run IST over SPB fabric. Removing the
need for direct links between IST peers. Can be
deployed as conventional SMLT/RSMLT solution
with direct IST links.
 First phase: configure ISIS and SPB & IST
protocol between a cluster pair
 Virtual IST Benefits
Seamless & Painless Scalability
Virtual IST
 Further increased resiliency & more flexible
routing for IST connectivity. (no need for direct
IST trunk)
 Allows mixing of IST node types
(VSP8k, VSP4k, …)
 Future benefits for FA and distributed LAG
 Platform Implementation Timelines


Single, Unified, Logical Core
VSP8k Release 4.0 1H2014
VSP4k Release 4.1 2H2014
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
14
Models to Extend the Fabric to the Wiring Closet Edge
 ERS 4800 SPB L2 VSN
NNI
L2 VSN
NNI
 Fabric Attach
FA Client
FA Hosts
ERS 5600
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
15
Fabric Attach (FA)
 Concepts




Automatic attachment of non-fabric switches (e.g.
ERS 56xx) and hosts/devices (Servers, Cameras,
APs) to Fabric Connect networks.
Introduces FA Server, FA Switch & FA Devices
Uses signaling protocol to signal VLAN/ISID
memberships between attached hosts and nonfabric switches/devices to FA Server switches
(BEBs).
First step to Zero-Config-Edge: Establishes node
connectivity for FA hosts/devices out of the box
and auto attaches it to fabric
 Timelines (solution more widely available
in 2015)



FA Server: VSP7k, ERS4k demo now, VSP8k,
VSP9k, VSP4k 2015
FA Switch: ERS4k, ERS5k demo now
FA Devices: TBD
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
16
Fabric Attach Extends the Benefits of Fabric to the Wiring Closet
& Network Attached Devices

Customer Value





Technology
Automated identification and provisioning of
end points (e.g., wireless AP’s and cameras)
Simplified network provisioning for devices
outside the Fabric
Reduction in network configuration errors
Simplifies adds, moves, and changes


Builds on top of Fabric Connect architecture
Extends Fabric benefits to non-Fabric Connect
platforms AND endpoints / users
Client/Device identification, authentication and
authorization via Identity Engines

Fabric Attach
Switch ERS 5600
DHCP
Fabric Connect
Switch Stack of ERS 4800
Management Zone
Employee Zone
Contractor Zone
Guest Zone
Fabric Attach
Server ERS 4800
Fabric Attach
Switch ERS 4800
Fabric
Attach
Fabric
Attach
Fabric Attach
Access Point
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
Authentication &
Authorization
Authentication &
Authorization
17
Fabric Connect SPB over IP

Concepts






Benefits





Solution allows extension of Avaya’s Fabric Connect
fabrics over IP networks.
Introduces 100Mbps/1Gbps SFP compatible
adapter which tunnels Ethernet VLANs over IP.
The SFP “sleeve” can be inserted into an SFP/SFP+
NNI port and is used in conjunction with any
supported SFP (sorry, no 10Gig SFP+).
SFP adapter provides IP tunneling capabilities
for SPB NNI connections by adding additional IP
tunnel header to the SPB MAC-in-MAC packets.
IP connection MTU requirements: 1582 to 2000
bytes – work with the carrier to ensure compatibility!
Allows extending SPB/Fabric Connect over an IP
network. IP network can be campus backbone or
MAN/WAN IP MPLS network.
Full fabric capabilities remain intact over IP (except
MTU considerations).
Support for Hub and Spoke topologies (with up to
64 tunnels per adapter)
Allows extending IST over WAN solution with vIST
Timelines


Proof Of Concept occurring now
Production anticipated 2H2014, early 2015
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
18
Fabric Connect over IP Deployment
Scenario – Hub and Spoke
VSP4000
SPB Fabric
Site 1
VSP4000
Main Site
VSP4000
Site 2
ERS4800
WAN
VSP4000
4 interfaces
Site 3
VSP4000
Site 4
 Adapter establish multiple tunnels per device (POC restricted to 2)
 For POC hub site requires translation bridge (ERS4800) to convert
NNI interfaces into 1 uplink port
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
19
March 25 – 27th, 2014 І Orlando, FL
City of Cedar Rapids, Iowa
Fabric Connect/SPBm Implementation Case Study
Keith Nuehring – IT Operations Manager
©©2014
2014Avaya
AvayaInc.
Inc.Avaya
Avaya– –Confidential
Confidential& &Proprietary
Proprietary
DoDo
not
not
duplicate,
duplicate,
publish
publish
oror
distribute
distribute
further
further
without
without
the
the
express
express
written
written
permission
permission
of of
Avaya.
Avaya.
#AvayaATF
#AvayaATF
Cedar Rapids Network before SPB and Fabric Connect
 Many device types – ERS 8300, ERS 5530, ERS 4548, ERS 2526,
SR 1004, SR 3120
 With three primary locations, SMLT (two-switch cluster) didn’t really fit well
 ERS 5530’s required at least annual reboot to maintain stability
 Maintained a Spanning Tree ring to support smaller venues within the City
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
21
Considerations and the Decision to move
to Fabric Connect
 Should we stay with Avaya or move to another vendor?
 Nortel’s Chapter 11
 Uncertainty over whether Avaya would continue to support and expand
the networking business acquired with Nortel Enterprise Solutions
 Concerns with technical support after Avaya’s NES acquisition
 Concerns with stability and product life of existing equipment
 ERS 8300
 ERS 5530
 Attended ATF Orlando February 2012
 Came away from first ATF with decision to implement SPB
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
22
Network Redesign Goals and Objectives





New network in conjunction with new City Services Center
Upgrade from 1Gigabit to 10Gigabit City-wide network
Increase network resilience – target of milliseconds, not seconds
Consolidate Layer 3 routing into three main sites
Reduce number of devices types to support
 From …
 ERS 8300 and ERS 5500 both performing IP routing
 ERS 4000 series and ERS 2500 series edge switching
 Different platforms, different capabilities, different CLIs
 To …
 VSP 7000 SPB Fabric Switches and IP Routing
 ERS 4000 series and ERS 3500 series edge switching
 Single CLI common across entire network
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
23
Planning and Staging the Network Cutover
 Considerations in selecting the network products to use
 Large chassis switches not practical for City of Cedar Rapids
 VSP 7000 – 10 Gigabit switching, SPB Fabric, but no
simultaneous SPB Fabric and IP Routing
 VSP 4000 – Limited number of 10 Gigabit interfaces, semiexternal USB flash, does have IP Shortcut Routing
 Selected VSP 7000 both for SPB Switching Fabric and for IP
Routing external to the SPB Fabric
 Network design
 Three-site mesh topology with all links active
 NetApp storage and servers sync between sites across L2 VSN
 Training and Staging
 Used to-be live equipment for on-site hands-on training for staff
 Pre-built and tested entire network in City Services Center lab
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
24
New City of Cedar Rapids SPB Network Design
Involta Data Center
VSP 7000
L2 Fabric Core and in-building
IDF Distribution
VSP 7000
L3 Fabric Services
ERS 4800
1Gig Server Switching
Public Works
Central Fire
VSP 7000
L2 Fabric Core and in-building
IDF Distribution
City Hall
VSP 7000
L3 Fabric Services
Police Department
ERS 4800
1Gig Server Switching
Water
City Services Center
VSP 7000
L3 Fabric Services
Shown:
• Main site network core switching
and routing only
Not shown:
• Small, seasonal venues (parks,
rinks, etc.)
• In-building wiring closets
VSP 7000
L2 Fabric Core and inbuilding IDF Distribution
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
ERS 4800
1Gig Server Switching
25
Cutover Weekend
 Moved pre-configured equipment from City Services Center lab to
final location
 Plan and document all steps, label all cables
 Walk through migration steps in a dry run to test procedures
 With single-strand BX optics, could bring up basics of new network
simultaneously with old network for easing the migration
 Proactively open up Avaya Support case, have support engineer
review plan and configurations before cutover
 Problems? … a few …
 PVID set wrong on a switch
 Missed moving a cable, caused a routing loop
 VSP 7000 in Involta Data Center random rebooting
 Updated boot loader to fix
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
26
Observations and Lessons Learned
 SPB resiliency works
 One optic failed some time after cutover and the NNI was
bouncing
 No visible effect to users – even while link was bouncing for 21/2 hours
 Simply disabled link over weekend until optic could be
changed the next Monday
 Proved during cutover that even with a VSP 7000 failing, the
alternate switch and path would work to maintain the network
 VSP 7000 that was rebooting was one of two switches
connecting the NetApp storage!
 Simplified IP routing by consolidating into VSP 7000’s with VRRP
 Standardized on Direct and Static routing and VRRP
 Single CLI syntax on VSP 7000, ERS 4000, ERS 3500 is nice
 Was it worth making the change?
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
27
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
#AvayaATF
BEST OF ATF
SPEAKER AND TEAM AWARD
BE SURE TO
TWEET YOUR FEEDBACK
ON THIS PRESENTATION
#AvayaATF
Winners will be announced at closing of event
© 2014 Avaya Inc. Avaya – Confidential & Proprietary
Do not duplicate, publish or distribute further without the express written permission of Avaya.
#AvayaATF
29
#AvayaATF