Transcript Fabric Module - DFW Cisco Users Group

ACI OVERVIEW
Temi Ajasa - Systems Engineer
Allen McClure – Systems Engineer
Cisco Confidential
1
AGENDA
Application Centric Infrastructure Overview
Application Centric Infrastructure Policy Model
Nexus 9000 Hardware
Q and A
Cisco Confidential
2
APPLICATION-CENTRIC INFRASTRUCTURE
NEXUS
9000 SERIES
APPLICATION POLICY
INFRASTRUCTURE
CONTROLLER
INDUSTRY LEADING
ECOSYSTEM
APIC
OPEN STANDARDS OPEN SOURCE
Cisco Confidential
3
MARKET TRENDS
REQUIRES OPEN APIS, OPEN SOURCE APPROACH
HYPERVISOR
FRAGMENTATION
APPLICATIONS
PRIVATE/PUBLIC
CLOUD
Public Cloud
Hypervisor
42% OF BUSINESSES USE
MULTIPLE HYPERVISORS
PHYSICAL + VIRTUAL
60–80% OF WORKLOADS
VIRTUALIZED
HADOOP, BIG DATA AND
ANALYTICS
~21% OF PHYSICAL SERVERS
VIRTUALIZED BY 2016
Cisco Confidential
Service
Provider Cloud
Private Cloud
Enterprise IT
Organizations
2 OUT OF 3 US BASED
MIDSIZE FIRMS WILL USE
CLOUD SERVICES
INTEGRATED DEVELOPMENT
AND OPERATIONS
OPEN RESTFUL APIS, OPEN SOURCE
4
A NEW OPEN OPERATING MODEL IS REQUIRED
TRADITIONAL
NETWORK MODEL
TODAY’S SDN
DATACENTER MODEL
FUTURE
OPEN MODEL
Network of Boxes
Software-Based
Network Virtualization
Application Centric
Infrastructure
Needs Agility and Time
to Applications
Lacks Scale, Visibility, Security
Open Source, Open APIs
Physical and Virtual
Radical Simplification
Policy and Automation
Scale and Security
Visibility and Troubleshooting
More Complexity
Decreases Reliability
Disjointed Overlay and Underlay
Cisco Confidential
5
• Classical approach to connectivity requires mapping
the various connectivity service layers manually
• ACI directly maps the application connectivity
requirements onto the fabric
• Security is ‘always’ enabled
• Fabric is application aware
• Services inserted dynamically
Redirect and Load Balance Connectivity
IP Address, VLAN, VRF
Control & Audit Connectivity
Application Requirements
(Security – Firewall, ACL, …)
IP Addressing
IP Address,
VLAN, VRF
Application Requirements
Dynamic provisioning of connectivity explicitly
defined for the application
Application Specific Connectivity
Enable Connectivity
(The Network)
Cisco Confidential
6
ACI BUILDING BLOCKS
NEXT GENERATION
FUTURE
PROOF—SOFTWARE
NEXUS—TRADITIONAL
UPGRADABLENETWORKS
TO ACI
OPEN RESTFUL APIS
CENTRALIZED POLICY MODEL
OPEN SOURCE
APIC
SIMPLE, SECURE
CONTROLLER
PRICE
APIC
POLICY
MODEL
NEXUS
9500 and 9300
BUILT-IN LINE RATE
INNOVATIONS IN SOFTWARE HARDWARE AND SYSTEM DESIGN
PERFORMANCE
Cisco Confidential
PROGRAMMABILITY
POWER EFFICIENCY
SCALEOPTIMIZED
OUT WITHOUTNX-OS
COMPROMISE
COMMON
BUILDING
BLOCKS - ACCESS AND CORE
INTEGRATED
OVERLAY
40G NON-BLOCKING FABRIC
>_
>_
RESILIENCY:
IN SERVICE PATCHING,
UPGRADE, FAST RESTART
END POINT DIRECTORY
PORT DENSITY
50% SIMPLER
CODE BASE
ACI
FUTURE PROOF
UPGRADABLE
TO ACI
NETWORK
VIRTUALIZATION
SUPPORT
PROGRAMMABILITY
AND AUTOMATION
7
ACI: RAPID DEPLOYMENT OF APPLICATIONS ONTO
NETWORKS WITH SCALE, SECURITY AND FULL VISIBILITY
Physical
Networking
Hypervisors
and Virtual
Networking
Compute
L4–L7
Services
Storage
Multi DC
WAN and Cloud
ENABLED BY PHYSICAL AND VIRTUAL INTEGRATION
Cisco Confidential
8
APPLICATION CENTRIC INFRASTRUCTURE
INVESTMENT PROTECTION
Nexus 9500
APIC
Nexus 9300 and 9500
Physical
Networking
Hypervisors
and Virtual
Networking
Compute
L4–L7
Services
Storage
Multi DC
WAN and Cloud
Nexus 7K
Nexus 2K
Cisco Confidential
Integrated
WAN Edge
9
AGILITY: ANY APPLICATION, ANYWHERE—PHYSICAL AND VIRTUAL
COMMON APPLICATION NETWORK PROFILE
WEB
F/W
ADC
APP
ADC
DB
Extensible APIC
Scripting Model
SLA
QoS
CONNECTIVITY
POLICY
SECURITY
Security
POLICIES
Load
Balancing
QOS
BANDWIDTH
RESERVATION
AVAILABILITY
APPLICATION
L4-L7
SERVICES
STORAGE
AND
COMPUTE
APPLICATION
NETWORK PROFILE
HYPERVISOR
Cisco Confidential
HYPERVISOR
HYPERVISOR
10
OPEN SOURCE ACI POLICIES
APPLICATION-CENTRIC POLICY DEFINITION
Orchestration and
Automation
ACI Extensions
ACI Extensions
Network Controller
ACI API Extensions
ACI Fabric
Hypervisor
ACI Extensions
ACI Extensions
APIC
• Application policy model
• Open source technology
• Community driven
• Stand alone Nexus 9000 Switch,
traditional networks
• Application policy model with hardware acceleration
• Any application, any where
• Best in class scale and performance
• Real-time network telemetry
• Nexus 9000 ACI Mode
Cisco Open Source Solution
Cisco ACI Solution
OPEN SOURCE COMMUNITY
Cisco Confidential
11
SIMPLIFICATION
10,000S ACLS
COMPLEX QOS
MULTIPLE MANAGEMENT
POINTS
EXCESSIVE PROTOCOLS
FLOODING
Cisco Confidential
COMMON POLICY
DECOUPLE APPLICATION &
POLICY FROM IP INFRASTRUCTURE
IP NETWORK
CENTRALIZED
SECURITY AND
QOS POLICY
NO FLOODING
ROUTED NETWORK
FULL HOST MOBILITY
12
ELASTICITY AT SCALE / PAY AS YOU GROW
BUILT FOR THE GROWING COMMERCIAL ENTERPRISE TO THE LARGEST SERVICE
PROVIDERS
1 MILLION IPV4 / IPV6
END POINTS
8K MULTICAST GROUPS
64,000
TENANTS
576 40G PORTS WIRE-RATE
(PER SPINE)
APIC
60 TBPS CAPACITY
(PER SPINE)
(PER LEAF)
100K+
27648
35860
44652
13824
18632
11592
4854
5260
6912
8598
1286
2268
3456
22584
288 PORTS
Cisco Confidential
13
APPLICATION CENTRIC INFRASTRUCTURE SECURITY
SECURITY WITH ACI
Open APIs
APIC
Policy
Engine
Centralized Compliance and Auditing
Services Chaining Automated
APPLICATION
NETWORK PROFILE
Import / Export Policy via API
(Support for External Policy Engines)
Policy Separated from Network Forwarding
Complete Isolation with
Full Scalability and Security
Cisco Confidential
Engineering
Sales
HR Finance
Legal
Legal and Marketing
Marketing
ENABLING A DYNAMIC ENTERPRISE WITHOUT COMPROMISE
14
COMMON HARDWARE PLATFORM
TWO OPERATIONAL MODELS
TRADITIONAL NETWORKS
OPTIMIZED NX-OS
APPLICATION CENTRIC
INFRASTRUCTURE
Software
Upgradable
to ACI
PROGRAMABILITY— 1/10/40 GE, 100 GE READY PRICE/PERFORMANCE
Q4 2013
Cisco Confidential
APIC
Agility
Simplicity
Automation
and Visibility
Performance
and Scale
Security
Open
Q2 2014
15
NEXUS 9000 SERIES SWITCHES
• Family of fixed and modular switches
• Foundation for Application Centric Infrastructure (ACI)
• Runs in two operating modes – Cisco® NX-OS and ACI
• Delivers industry-leading 10/40 Gb platform for:
̶ Price and performance
̶ Power
̶ Programmability
• Establishes Cisco leadership in 40 Gb density and
performance
• Designed for future upgrade to 100 Gb
Cisco Confidential
16
BUILD A BETTER SWITCH




Merchant+ Foundation
State of the Art Mechanical Design
Object Oriented Programmable Operating System
Next Generation Development and Verification
Methodology
 Two Modes of Operation
 NXOS
 Fabric Mode ACIOS + APIC
Cisco Confidential
17
OVERVIEW
• High Port Density
• Line-Rate Performance on All Ports
• Low Latency
10 Gbps Ports
40 Gbps Ports
Nexus9508
1152
288
Nexus9516
2304
576
• VxLAN Bridging/Gateway/Routing
• Highly integrated switch and buffer functionality
Only 2-4 ASICs per line card
No buffer bloat
Mix of 28nm Cisco and 40nm Broadcom ASICs
• Power Efficiency
Platinum rated power supplies, 90-94% power efficiency across all workloads
3.5W per 10 Gbps Port
14W per 40 Gbps Port
• First modular chassis without a mid-plane
Unobstructed Front-to-Back airflow
Cisco Confidential
18
NEXUS 9500 – CHASSIS AND LINE CARD OPTIONS
40G Aggregation
36 ports 40G QSFP+
(Non Blocking)
Non-ACI
1/10G Access and 10/40G Aggregation
48 ports 10G SFP+ & 4 ports 40G QSFP+
48 ports 1/10G-T & 4 ports 40G QSFP+
(non blocking)
ACI Access Ready
Nexus 9508
•
•
•
•
•
13 RU high
30Tbps fabric today
Up to 288p 40G & 1,152p 10G
Headroom for 100G densities
(connectors, power)
Supervisors w/ quad core CPU and default 64GB SSD
40G Fabric Spine
36 ports 40G QSFP+
(Non Blocking)
ACI Spine
Cisco Confidential
19
CHASSIS ARCHITECTURE – DENSITY
17.5 in
• Maximum three chassis per rack
30 in
– Assuming 18KW per rack
• Up to 3,456 10G line rate ports
per rack
• Up to 864 40G line rate ports per
rack
• Designed for at least 2.5x speed
increase in next gen ASICs
13 RU
Front View
Cisco Confidential
20
CHASSIS DESIGN – COMPONENTS
Nexus 9508 Front View
Nexus 9508 Rear View
8 Line Card Slots
Max 3.84 Tbps/Slot duplex
3 Fan Trays
3 or 6 Fabric Modules
(behind fan trays)
Redundant
Supervisor Engines
Redundant System
Controller Cards
Designed for:
 Power Efficiency
 Cooling Efficiency
 Reliability
 Future Scale
Cisco Confidential
3000W AC Power Supplies
2+0, 2+1, 2+2 Redundancy
Support up to 8 Power supports
No Mid-plane for
LC to FM connectivity
21
CHASSIS – POWER SUPPLIES UNITS (PSU)
• Single 20A input at 220V
• Support for range of international cabling options
• 92%+ Efficiency
• Range of PS configurations
3000W AC PSU
• Minimum 1 PSU, Maximum 8 PSU
• (2) PSU for fully loaded chassis
• N+1 redundancy
• N+N grid redundancy
• 2x head room for future port densities,
bandwidth, and optics
80 Plus Platinum is equivalent to Climate
Saver/ Green Grid Platinum rating
Cisco Confidential
22
CHASSIS – SUPERVISOR MODULES
• Redundant half-width supervisor
engine
• Sandy Bridge, Quad Core, 1.8GHz
• 16GB Memory
• RAM upgradable to 64GB
• 64 GB SSD (default)
• Common for 4, 8 and 16 slot chassis
• Performance/ Scale Focused
• Range of Management Interfaces
Console Port
(2) USB Ports
Management Port
External Clock Input
(Precision Time Protocol)
Cisco Confidential
23
CHASSIS – SYSTEM CONTROLLERS
• Redundant half-width system controller
•
Offload supervisor from switch “control
plane” tasks
•
Increased System Resiliency
•
Increased Scale
• Common for 4, 8 and 16 slot chassis
• Performance/ Scale focused
•
Dual Core ARM Processor, 1.3GHz
•
Central Point of Chassis Control
•
Ethernet Out-of-Band Channel (EOBC)
switch between Supervisors and Line cards
•
Ethernet Protocol Channel (EPC) switch
1Gbps switch for Intra-node Data Plane
communication (Protocol Packets)
Cisco Confidential
•
Manages / Monitors
• Power Supplies via SMB (System Management
Bus)
• Fan Trays
24
NEXUS 9500 – CONTROL PLANE – COMMUNICATIONS
• The Nexus 9500 chassis has two communication channels connected through the SGMII
(1Gbps) switches on System Controller Modules
•
•
EOBC (Ethernet Out of Band Channel)
EPC (Ethernet Protocol Channel)
• No dedicated direct path between I/O Modules and Supervisor Module
EOBC
EPC
1G
1G
EOBC
Switch
1G
EPC
Switch
1G
EOBC
I/O Modules
Cisco Confidential
EOBC
Supervisor
System Controller
1G
EPC
Fabric Cards
NFE
ALE
NFE
25
HARDWARE
NEXUS 9500 – CONTROL PLANE – EOBC
• Ethernet Out of Band Channel (EOBC) inter-connects all modules together through
SGMII (1Gbs) switch that resides on System Controller (SC).
• The EOBC serves as normal control path.
• It also replaces the traditional System Management Bus (SMB) to simplify the system
design.
SUP-A
FC-1
FC-1
NFE
TR2-1
FC-1
NFE
FC-2
FC-1
NFE
TR2-1
FC-2
NFE
FC-3
FC-1
NFE1
TR2-1
FC-3
NFE
SUP-B
EOBC Switch on SC
EOBC
LC-1-ALE
LC-2-ALE
LC-3-ALE
LC-8-ALE
LC-1-NFE
LC-2 NFE
LC-3 NFE
LC-8- NFE
e1/1
Cisco Confidential
48
e2/1
48
e3/1
48
e8/1
48
26
NEXUS 9500 – CONTROL PLANE – EPC
•
The Ethernet Protocol Channel (EPC) handles protocol packets between the Supervisor and Line
Cards.
•
Unlike EOBC, the EPC only connects supervisors and fabric modules through SGMII (1Gbs) switch.
•
There is no dedicated direct-path between the Line Cards and the Supervisor modules.
•
To send protocol packets to the Supervisors, the Line Cards utilize HiGig2 links to transfer packets to
Fabric Modules first. Fabric Modules terminate those packets and re-direct it via EPC to Supervisor.
SUP-A
Supervisor
SUP-B
EPC Switch on System Controller
EPC Switch
FC-1
FC-1
NFE-1
TR2-1
Fabric Module
FC-1
NFE-2
FC-2
FC-1
NFE-1
TR2-1
FC-2
NFE-2
FC-3
FC-1
NFE-1
TR2-1
FC-3
NFE-2
Hi-Gig2 Links
Line Cards
LC-1 ALE
LC-2-ALE
LC-3-ALE
LC-8-ALE
LC-1 NFE
LC-2-NFE
LC-3-NFE
LC-8-NFE
e1/1
Cisco Confidential
48
e2/1
48
e3/1
48
e8/1
48
27
HARDWARE
8-SLOT MODULAR CHASSIS AIR FLOW
• Chassis is complete Front-to-Back
Airflow
• Airflow direction is NOT Reversible
• Fan Trays are fully redundant
Front
View
• Fan Trays must be removed in order to
service Fabric Modules
• Designed for speed increase in multiple
next gen ASICs
Rear
View
Fan Tray
Removed
Fabric Modules
Cisco Confidential
28
FABRIC MODULES AND FAN TRAYS
• Up to 6 Fabric Modules
• 3 Fan Trays
• (3) dual fans per tray
Different cost points for 1/10G
access and 40G aggregation
• Dynamic speed control driven
by temperature sensors
Flexibility for future generation
of fabric modules
• Straight Airflow across Line
Cards and Fabric Modules
Quad Core ARM CPU 1.3 GHz
for Supervisor offload
• N+1 Redundancy per
Hot Swappable
Tray
All Modules Forward Traffic
• Smooth degradation
during replacement
Fabric
Module
Cisco Confidential
Fan
Tray
29
FABRIC MODULE – DATA PLANE SCALING FOR 8-SLOT CHASSIS
• A Fabric Module in an 8-Slot Chassis can provide up to 320Gbps to each Line
Card slot.
• With 6 Fabric Modules, each Line Card slot can have up to 1.92Tbps forwarding
bandwidth in both directions.
Fabric 1
NFE
Fabric 2
NFE
NFE
320 Gbps
(8x 40Gbps)
NFE
320 Gbps
(8x 40Gbps)
Fabric 3
NFE
Fabric 4
NFE
320 Gbps
(8x 40Gbps)
NFE
NFE
320 Gbps
(8x 40Gbps)
Fabric 5
NFE
Fabric 6
NFE
320 Gbps
(8x 40Gbps)
NFE
NFE
320 Gbps
(8x 40Gbps)
320 Gbps
640 Gbps
960 Gbps
1.28 Tbps
Line Card Slot
Cisco Confidential
1.60 Tbps
1.92 Tbps
30
LINE CARDS – OVERVIEW
40G Aggregation
36 ports 40G QSFP+
(Non Blocking)
NXOS Only
1/10G Access and 10/40G Aggregation
48 ports 10G SFP+ & 4 ports 40G QSFP+
48 ports 1/10G-T & 4 ports 40G QSFP+
(non blocking)
ACI Ready
36 ports 40G QSFP+ ((1.5:1 oversubscribed)
ACI Access Ready
40G Fabric Spine
36 ports 40G QSFP+
(Non Blocking)
ACI Only
Cisco Confidential
31
FIXED SWITCH PLATFORM – NEXUS 9300
Nexus 9396PQ
Uplink Module
• 48 port 10G SFP+ & 12 port 40G QSFP+
• 2 RU
• FAN1
• 100-240V (650W AC)
•
•
•
12 port 40G QSFP+
Additional 40MB buffer
Full VXLAN Bridging & Routing Capability
Nexus 93128TX
Nexus 9300 - Common
•
•
•
•
•
•
•
96 port 1/10G-T & 8 port 40G QSFP+
3 RU
FAN2
100-120V (800W AC), 200-240V (1200W AC)
Cisco Confidential
Redundant FAN (3) and Power Supply (2)
Front-to-back and Back-to-Front airflow
Dual or Quad Core CPU with default 64GB SDD
32
OPTICAL INNOVATION – REMOVING 40G BARRIERS
Challenge
• 40G Optics are significant portion of CAPEX
• 40G Optics require new cabling
Solution
• Re-use existing 10G MMF cabling infrastructure
• Re-use patch cables (same LC connector)
• Price comparable to 10G optics
Cisco 40G SR-BiDi QSFP
• QSFP pluggable, MSA compliant
• Dual LC Connector
• Support for 100m on OM3 and 125m+ on OM4
• Transmit/Receive on 2 wavelengths at 20G each
Available end of CY13 and supported across all Cisco QSFP ports
Cisco Confidential
33
OPTICS SUPPORT ON THE NEXUS 9000 SERIES
• All optical interfaces are pluggable (MPO)
• 10G SFP Transceivers – SR, LR
• 10G Cables – Passive Copper, Active Optical
• 10G Fabric Extender Transceiver (FET)
• 40G QSFP Transceivers – SR4, CSR4, BiDi, LR4
• 40G Cables – Passive Copper, Active Optical
• 1G Transceivers – SM, MM, GLC-T
Cisco Confidential
34
THANK YOU
Cisco Confidential
35