Transcript iSCSI Technical Preso

IP Storage Networking
FCIP/iSCSI
Steve Tegeler
Storage Networking Team
Northwest Territory
425/468-0836
[email protected]
iSCSI Overview
© 2004 Cisco Systems, Inc. All rights reserved.
1
Agenda
• Storage Networking Technology Review
• IP Storage Networking
• FCIP
Write Acceleration, Compression, IPSec, SAN Extension Tuner
• FCIP Wizard
• iSCSI
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
2
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
3
The Old Storage Environment
Clients
• Direct Attached Storage (DAS)
• Storage is captive ‘behind’
the server
• Server CPU must handle
user I/O requests, but also:
User-database inquiries
User file/print serving
Data-integrity checking
Communication with
other devices
• Data access is file system
and platform dependant
• Costly to scale; complex
to manage
IP Network
Servers
Win2k
Linux
Win2k
FC
Linux
Unix
FC
SCSI
Direct-Attached Storage (DAS)
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
4
The SCSI I/O Channel
• SCSI is the dominant protocol
used to communicate between
servers and storage devices
in open system
• SCSI I/O channel is a
half-duplex pipe for
SCSI CDBs and data
• Parallel bus evolution
Bus width: 8, 16 bits
Bus speed: 5–80 Mhz
Throughput: 5–320 MBps
Devices/bus: 2–16 devices
Cable length: 1.5m–25m
Applications
Raw
Block Device
SCSI Generic
TCP/IP
Stack
NIC
Driver
Ethernet
NIC
• A network approach can scale
the I/O channel in many areas
(length, devices, speed)
Ethernet
SCSI CDB: SCSI Command Descriptor Block Used to Relay
SCSI Commands, Parameters, and Status between SCSI
Initiators and SCSI Targets; Typically 6, 10, or 12 Byte Block
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
File System
Adapter Driver
SCSI Adapter
SCSI
Initiator
Half-Duplex
SCSI
I/O Channel
SCSI
SCSI
Target
5
Networking the I/O Channel
• Transport must not jeopardize
SCSI payload (security, integrity,
latency)
• Two primary transports to choose
from today: Fibre Channel and IP
Host System
Initiator
SCSI
Networked
I/O Channel
• Same SCSI protocol (SCSI-3)
carried over a network transport
layer via serial implementation
Channel
Controller
Network
• A networked I/O channel allows
for multiple improvements:
Distance limitations greatly increased
High number of addressable devices
Target and LUNs
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
6
Fibre Channel Networking
• Very common method for networking SCSI
• Fibre Channel provides high-speed
transport for SCSI payload
• Fibre Channel SAN overcomes many
shortcomings of DAS including:
Addressing for up to 16-million nodes
(24 bits)
Host System
Initiator
SCSI
Fibre
Channel HBA
Loop (shared) and Fabric (switched) transport
Speeds of 100 or 200 MBps (1 or 2 Gbps)
Fibre Channel
Fabric
Distance of up to 10km (without extenders)
Can utilize CWDM or DWDM for over 10km
Support for multiple protocols
• Combines best attributes of a
channel and a network
Target
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
7
IP: An Alternate I/O Transport
Host System
• Viable transport for I/O traffic
• Not necessarily for long-haul I/O only
• Similar characteristics to Fibre Channel:
Initiator
SCSI
Addressing for close to 4 billion nodes (IPv4)
Primarily a switched transport (with routing)
Ethernet speeds of 10/100 Mbps or 1/10 Gbps
or various WAN speeds
Support for multiple high-level protocols
IP “Channel
Adapter”
IP Network
• Cost and manageability advantages with IP
• IP knowledge base widespread in industry
Target
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
8
IP Storage Networking
• IP storage networking provides solution to carry
storage traffic within IP
• Uses TCP: a reliable transport for delivery
• Applicable to local data center and long-haul applications
• Two primary protocols:
iSCSI—Internet-SCSI—used to transport SCSI CDBs and data
within TCP/IP connections
IP
TCP
iSCSI
SCSI
Data
FCIP—Fibre-Channel-over-IP—used to transport Fibre Channel frames
within TCP/IP connections—any FC frame—not just SCSI
IP TCP
OPT-2053
9761_05_2004_c2
FCIP
FC
© 2004 Cisco Systems, Inc. All rights reserved.
SCSI
Data
9
FCIP – Extending your FC SAN
OPT-2053
9761_05_2004_c2
©
© 2004,
2004 Cisco
Cisco Systems,
Systems, Inc.
Inc. All
All rights
rights reserved.
reserved.
10
Fibre Channel over IP (FCIP)
Point to Point
FCIP – Fibre Channel over Internet Protocol
The encapsulation of Fibre Channel frames into IP packets and tunneling through an
existing TCP/IP network infrastructure, in order to connect geographically distant
islands
Ethernet Catalyst
Switches & Routers
Optical Extension
Metro DWDM
& CWDM
E-port
E-port
LAN/MAN/WAN
SAN
FCIP Tunnel Session
IPS
SAN
IPS
Sync or Async
Replication
FC Disk
OPT-2053
9761_05_2004_c2
FCIP tunnels can be thought of as
ISL’s with Latency
© 2004 Cisco Systems, Inc. All rights reserved.
FC Disk
11
iSCSI
OPT-2053
9761_05_2004_c2
©
© 2004,
2004 Cisco
Cisco Systems,
Systems, Inc.
Inc. All
All rights
rights reserved.
reserved.
12
Audience Poll
• Who has a FC Network today?
• Who has deployed iSCSI,
• Array based, or gateway based?
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
13
What is iSCSI?
• A SCSI transport protocol that operates over TCP/IP
Encapsulates SCSI CDBs (operational commands: e.g. read
or write) and data into TCP/IP byte streams
Allows IP hosts to access IP-based SCSI targets (either natively
or via iSCSI to FC Gateways)
• Standards status
RFC 3720 on iSCSI
Collection of RFCs describing iSCSI
RFC 3347—iSCSI Requirements
RFC 3721—iSCSI Naming and Discover
RFC 3723—iSCSI Security
• Broad industry support
Server vendors now publishing own supported iSCSI drivers
Native iSCSI storage arrays now appearing
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
14
Recap SCSI Architectural Model Transports
SCSI Applications (File Systems, Databases)
SCSI
Device-Type
Commands
SCSI
Generic
Commands
SCSI
Transport
Protocols
SCSI Block Commands
SCSI Stream
Commands
SCSI Commands, Data, and Status
Parallel
SCSI Transport
FCP
SCSI over FC
OPT-2053
9761_05_2004_c2
iSCSI
SCSI over TCP/IP
TCP
Layer 3
Network
Transport
Layer 2
Network
Other SCSI Commands
IP
Parallel SCSI
Interfaces
© 2004 Cisco Systems, Inc. All rights reserved.
Fibre Channel
Ethernet, PPP, HDLC…
15
Why - iSCSI vs. FC
It’s all about the $$$
iSCSI-enabled Hosts
iSCSI
•
Leverage IP infrastructure for storage connectivity
•
Low-cost complement to FC SAN, provides
additional resource consolidation
iSCSI
iSCSI
1000BaseT NIC + GigE Port: Roughly $100 + $300 = $400
FC HBA+ FC Port: Roughly $1000 + $1000 = $2000
•
Secure connectivity via CHAP-based authentication
•
Transparent iSCSI routing gives iSCSI hosts a pWWN
Uses controller-based LUN masking or MDS-based virtual
targets for resource provisioning
Uses zoning for device connectivity
•
iSCSI driver (free) works with any Ethernet NIC
TOE only necessary with processor-bound servers
•
iSCSI is an industry-supported IETF standard
•
Many O/S vendors providing iSCSI initiator (MS, HP,
Novell, Linux), others provided by Cisco
Catalyst
Ethernet
Switch
IP
Network
Cisco MDS
9000 with
IP Services
Module
FC Disk Array
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
FC Servers
FC Tape Library
16
iSCSI for Storage Consolidation
iSCSI-Enabled
Hosts (Initiators)
• IP access to open
systems iSCSI and
Fibre Channel storage
iSCSI
iSCSI
iSCSI
• iSCSI driver is loaded onto
hosts on Ethernet network
• Able to consolidate servers
via iSCSI onto existing
storage arrays
iSCSI
Array
(Target)
• Able to build Ethernet-based
SANs using iSCSI arrays
• Storage assigned on a LUN-byLUN basis at iSCSI router
Logical Unit Number (LUN): A Field within SCSI
Containing up to 64 Bits that Identifies the Logically
Addressable Unit within a Target SCSI Device
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
iSCSI
IP
Network
iSCSI
Gateway
FC
Fabric
FC HBAAttached
Host
(Initiator)
Storage
Pool (Target)
17
iSCSI for Remote Block Access
• Block access to remote
storage over IP
iSCSI-Enabled
Host iSCSI
• Application must tolerate
latency for long distances
• Metro Ethernet services
offer lower-latency
transport alternative
• Remote backup over
IP WAN
• Centralized management
from centralized storage
Site A
IP
WAN
Site B
Remote
Mirrors
iSCSI
Devic
e
FC
Fabric
Storage
Pool
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
18
How - iSCSI Architecture: Software Driver
Host
Applications
File System
iSCSI GW Device
Block Device
iSCSI Host Driver
SCSI Generic
iSCSI GW Module
SCSI Driver
TCP/IP Driver
FC HBA
GigE NIC
iSCSI
TCP/IP
Stack
NIC
Driver
IP Network
NIC
Fibre Channel
iSCSI Path
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
Adapter Driver
SCSI Adapter
Conventional
SCSI Path
19
OS Support
• Many operating systems supported via
Cisco drivers and/or from OS vendor
Cisco provides full-driver suite
Solaris 2.6 (EOL),7,8,9
Linux-based on 2.6 kernel
Win 2000 with SP2 or later
Windows XP Pro
iSCSI
WinNT 4.0 with SP6A
Software
Driver
HP/UX 10.2, 11.0
AIX 4.3.3, 5.1, 5.2
OS vendors support native iSCSI drivers
Windows *native* Win 2000, XP,
2003 support
HP *native* HP/UX 11i support
IBM *native* AIX 5.x support
Novell Netware *native* support
Solaris 10 (March 2005)
Linux (RedHat Suse)
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
Applications
File System
Block Device
SCSI Generic
iSCSI
TCP/IP Stack
NIC Driver
NIC Adapter
Adapter
Driver
SCSI
Adapter
20
iSCSI HBAs and TCP
Offload Engines (TOEs)
• Offloads TCP and,
optionally, iSCSI
processing into
hardware
• Relieves host
CPU from:
Applications
File System
Block Device
SCSI Generic
TCP/IP
Stack
iSCSI
Driver
NIC
Driver
TOE
Driver
TCP processing—16-bit
checksum per packet
iSCSI—optional 32-bit
header and data digests
(CRC32C)
OPT-2053
9761_05_2004_c2
TCP/IP
Stack
HBA
Driver
Adapter
Driver
iSCSI
SCSI Adapter
TCP/IP
Stack
TCP
iSCSI
Offload and TCP
Offload
© 2004 Cisco Systems, Inc. All rights reserved.
21
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
22
Example performance impact on CPU util %
FC vs. iSCSI TOE vs. iSCSI SW Driver
iSCSI SW Driver
iSCSI TOE
CPU %
FC HBA
?
Throughput MB/s
OPT-2053
9761_05_2004_c2
15-35MB/s
Inflection point
determined by
system resources
(CPU/Memory)
© 2004 Cisco Systems, Inc. All rights reserved.
23
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
24
iSCSI Naming
• Initiator and target require iSCSI names
Name is location independent
iSCSI node name = SCSI device name of iSCSI device
Associated with iSCSI nodes, NOT adapters
Up to 255-byte displayable/human readable string
(UTF-8 encoding)
Use SLP (Service Location Protocol) V2, iSNS, or query
target for names (SendTargets)
• Two iSCSI name types:
iqn—iSCSI qualified name
eui—Extended Unique Identifier (IEEE EUI-64—
also used for FC WWNs)
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
25
iSCSI Name Structure
Type
iqn
–
Type
Date
–
–
Unique String
–
Organization
Naming Authority
Subgroup Naming Authority or
String Defined by Organization Naming Authority
iqn.1987-05.com.cisco.1234abcdef987601267da232.betty
iqn.2001-04.com.acme.storage.tape.sys1.xyz
Date = yyyy-mm When
Domain Acquired
eui
–
Type
Reversed Domain Name
EUI-64 Identifier (ASCII Encoded Hexadecimal)
eui.02004567a425678d
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
26
iSCSI Connectivity
pWWN – P5
nWWN – N5
FCID –XXXX05
iSCSI
HBA
HBA
FC
iqn.host-1
IP-10.1.1.2
iSCSI
IP
Network
IP-10.1.1.1
Fibre Channel
Fabric
HBA
HBA
iqn.host-2
IP-10.1.1.3
pWWN – P6
nWWN –N6
FCID – XXXX06
iSCSI
HBA
HBA
iqn.host-3
IP-10.1.1.4
• iSCSI Initiator knows IP and IQN
• FC Target knows WWN and FCID
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
27
iSCSI Mapping to a WWN
iSCSI
pWWN – P2
nWWN- N2
FCID XXXX02
pWWN – P5
nWWN – N5
FCID –XXXX05
HBA
HBA
FC
iqn.host-1
IP-10.1.1.2
pWWN – P3
nWWN – N3
FCID XXXX03
IP-10.1.1.1
iSCSI
Fibre Channel
Fabric
HBA
HBA
iqn.host-2
IP-10.1.1.3
iSCSI
HBA
HBA
pWWN – P4
nWWN- N4
FCID XXXX04
pWWN – P6
nWWN –N6
FCID – XXXX06
iqn.host-3
IP-10.1.1.4
• Each iSCSI Initiator gets a unique WWN and FCID
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
28
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
29
Considerations when building an iSCSI Fabric
iSCSI
Clients
• iSCSI fabric topology
iSCSI
Ethernet fabric topology
iSCSI
iSCSI
iSCSI
iSCSI
iSCSI
iSCSI
iSCSI
iSCSI
iSCSI
• iSCSI fabric scalability
Trunking
Port channeling
• iSCSI fabric availability
VRRP
E
N
D
Scalability
Availability T
Security O
Manageability E
• iSCSI fabric security
IPS
IPS
N
D
Authentication and binding
• iSCSI fabric manageability
iSCSI identity and management
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
Shared
Storage Pool
30
Dedicated IP Storage Network
• Separate logical
IP network but not
necessarily separate
physical network
• Can use a VLAN of
existing Ethernet
network
• Recommend use of
dedicated NIC on
host for iSCSI
• Minimized potential for
bandwidth contention
OPT-2053
9761_05_2004_c2
Front-Side IP Network
Clients
iSCSI
iSCSI
Catalyst
Switches
iSCSI
iSCSI-Enabled
Hosts
Dedicated
IP Storage
Network
iSCSI
Routers
FC
Fabric
Storage
Pool
© 2004 Cisco Systems, Inc. All rights reserved.
iSCSI
FC-Attached
Hosts with HBAs
31
IP Network Security Techniques
• Firewall
Standalone or intelligent firewall service module
Allow well-known TCP port 3260 for iSCSI
• IPSec VPN
VPN tunnel for iSCSI remote access
• Access Control List (ACL)
• VLAN and PVLAN
Subinterface implementation on iSCSI
Separated VLAN for iSCSI
• Port security
Allow, block, or restrain access to Ethernet based on
MAC address
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
33
What is iSNS?
Internet Storage Name Service (iSNS) is a name registration
service for IP storage devices:
Analogous to FCNS and DNS
Provides centralized management capabilities
iSNS supports:
Target device discovery
Discovery Domains (similar to zones)
Authentication
State change notification
Supports iSCSI and iFCP
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
34
What is iSNS? (cont.)
iSNS server
iSCSI
iSCSI
iSNS
IP
iSCSI
IP
iSCSI
iSCSI
iSCSI
IP
iSCSI
iSCSI
iSCSI
iSCSI
iFCP
Gateway
IP
IP
FC
iSCSI
FC
iSCSI
FC
FC
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
35
Agenda
• Storage Networking Technology Review
• iSCSI and IP Storage Networking
What, Why, How
• Performance
• iSCSI Terminology and Topology
• Design considerations when deploying iSCSI
• Summary
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
36
Summary
• Leverages the existing IP infrastructure
Hence the intelligence, capacity, and best practice design
can be leveraged in the iscsi infrastructure
• Complementary to FC yet represents a low-cost
transport choice
• Midrange applications connectivity
• Midrange server connectivity with blade server
integration as new system candidate
• Potential long-distance SAN transport
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
37
Reference Materials
• http://www.t10.org/
• http://www.t11.org/index.htm
• http://www.ietf.org/rfc.html on RFC 3720
• http://www.cisco.com/en/US/partner/products/hw/ps
4159/index.html
• http://www.lightreading.com/webinar_archive_home
.asp?webinar_id=27003
OPT-2053
9761_05_2004_c2
© 2004 Cisco Systems, Inc. All rights reserved.
38