Transcript Demartek iSCSI Storage May 2007

iSCSI Storage
Dennis Martin
Demartek
May 2007 RMWTUG Meetings
Introduction
♦ Dennis Martin, President, Demartek
♦ 27 years in I.T.
–
–
–
–
–
–
Software Engineer
Software Engineering Manager
Data Center Manager
Marketing
Industry Analyst
Mainframe, UNIX, Windows
♦ Founded RMWNTUG in Denver in 1994
♦ Microsoft Storage MVP 2005, 2006, 2007
© 2007 Demartek
2
Pre-Presentation Survey
♦ How many are using iSCSI storage today?
♦ How many are planning to use iSCSI storage?
♦ How many have heard of iSCSI but want to know
more?
© 2007 Demartek
3
iSCSI Storage – Agenda
♦
♦
♦
♦
♦
♦
♦
♦
♦
♦
Storage: DAS, NAS and SAN
iSCSI Concepts
Network Technology Items
Storage Technology Items
Solutions Available Today
Microsoft iSCSI Best Practices
Performance
Futures
Recommendations
Reference links
© 2007 Demartek
4
Direct Access Storage (DAS)
♦ Storage is directly connected to CPU
♦ Short distance between CPU and storage (inches or
small number of meters)
♦ Storage is “owned” by one host computer only
♦ Limited number of storage devices can be
connected
♦ Common interfaces are ATA/IDE, SCSI, SATA, SAS
© 2007 Demartek
5
DAS Diagram
Inside the Host Computer Cabinet
CPU
Memory
Storage Interface
Storage Device
© 2007 Demartek
Due to cable length limitations, the storage devices often reside in the same
cabinet as the CPU, or in a separate enclosure physically near the CPU cabinet.
Host CPU performs “block” I/O directly to device.
© 2007 Demartek
6
Network Attached Storage (NAS)
♦ One host server shares its resources with other
clients on the network
♦ Clients make requests by filename
♦ NAS server has its own storage, clients only see
“share” names
♦ Large number of “shares” can be created
♦ NAS server can be a long distance from clients
♦ Common protocols are NFS and CIFS
© 2007 Demartek
7
NAS Diagram
Network Attached Storage (NAS) Diagram
Network clients
Local Area Network (LAN)
NAS Server
© 2007 Demartek
NAS Private Storage
© 2007 Demartek
Network clients request files from file
server (NAS device) over the LAN.
NAS device receives file requests from
network clients and translates those
requests to “block” I/O commands to its
own private storage. It then formats its
data into a “file” format and responds to
the network clients.
8
Storage Area Network (SAN)
♦ Infrastructure allows computers to appear to have
direct connection to storage
♦ Storage can be great distance from computers
requiring access
♦ Clients make “block” I/O requests
♦ Can be millions of storage devices in a SAN
♦ Two technologies: Fibre Channel and Ethernet
© 2007 Demartek
9
SAN Diagram
Storage Area Network (SAN) Diagram
Network clients
Local Area Network (LAN)
Application Servers
Storage Area Network (SAN)
Storage traffic for SAN
storage generally flows
over a separate network
from the LAN traffic.
Storage Devices
© 2007 Demartek
© 2007 Demartek
10
SAN Components
♦
♦
♦
♦
Server
SAN Interface Card
SAN Switch
Storage Device (with Interface)
© 2007 Demartek
11
SAN Component Diagram
Storage Area Network (SAN) Component Diagram
Application Server
Switch
Interface card
Storage Device
© 2007 Demartek
© 2007 Demartek
12
Unified Storage
♦ Combination of NAS and SAN into single,
integrated storage solution
♦ Often includes:
– Fibre Channel SAN technology
– iSCSI SAN technology
– NAS technology
© 2007 Demartek
13
iSCSI SAN Components
iSCSI SAN Components
Application Server
Ethernet Switch
Ethernet adapter
Ethernet cabling
© 2007 Demartek
iSCSI Target
Storage Device
For iSCSI today, Ethernet components should be 1-Gigabit or better.
Cabling and interfaces can be copper or fiber-optic.
Ethernet adapters can be on the motherboard or in a PCI/PCI-X/PCI-E slot.
© 2007 Demartek
14
iSCSI Initiators
♦ Same basic function as SCSI initiators
♦ Available in software or hardware adapters
♦ Microsoft provides free software initiator
– Free download for : Windows Server 2003, Windows XP
Pro SP1, Windows 2000 SP3
– Included in Control Panel in Windows Vista, Longhorn
Server
– Available in 32-bit and 64-bit
♦ Third-parties sell iSCSI Host Bus Adapters (HBA)
♦ Initiators have iSCSI Qualified Name (IQN)
© 2007 Demartek
15
iSCSI Targets
♦ Same basic function as SCSI targets
♦ Available in software or hardware
♦ Wide variety of hardware targets available today
that can fit price, capacity or performance needs
– SATA disk arrays
– SAS disk arrays
– Fibre Channel disk arrays
♦ Targets have iSCSI Qualified Name (IQN)
© 2007 Demartek
16
Microsoft iSCSI Initiator Install
3
1
2
4
MPIO not
checked by
default
© 2007 Demartek
17
Microsoft iSCSI Initiator
Two target portals
visible to this initiator
Three targets (volumes)
visible to this initiator
Notice the IQNs
© 2007 Demartek
18
Multi-path I/O (MPIO)
♦ MPIO is supported for iSCSI
♦ Same MPIO as Fiber Channel
♦ Allows multiple network
connections from the same
host server to function as data
paths to the target
♦ Different load balance policies
are available
© 2007 Demartek
19
iSCSI Security
♦ iSCSI security should be considered in addition to
other network security that may be implemented
♦ Challenge Handshake Authentication Protocol
(CHAP)
– Predefined shared secret between initiators and targets
– Only allows logon if the other party knows the secret
– Can be one-way or mutual authentication
♦ IPSec
– If enabled, all data transfer packets are encrypted and
authenticated
© 2007 Demartek
20
Network Technology Items
♦
♦
♦
♦
♦
♦
♦
These affect iSCSI AND your regular network
Server-class NICs
Scalable Networking Pack
Receive-side Scaling
TCP Offload Engines (TOE)
Switches
Jumbo Frames
© 2007 Demartek
21
Server-class NICs
♦ There’s a big difference between a low-cost
desktop NIC and a server-class NIC
– Server-class NICs typically have some degree of offload
function such as TCP Checksum Offload, Large Send
Offload and other similar functions
– Server-class NICs have equal or better performance
with lower CPU utilization on the server
– Server-class NICs are more expensive than low-cost
NICs
© 2007 Demartek
22
Scalable Networking Pack
♦ Microsoft Scalable Networking Pack (SNP)
– Free download for Windows 2003 Server and Windows
XP 64-bit
– Included in Windows Server 2003 R2 SP2
♦ Provides network acceleration and compatibility
with hardware-based offload technologies
– Receive-side Scaling
– TCP Offload Engines
– NetDMA
♦ Improves all network traffic, not just iSCSI
© 2007 Demartek
23
Receive-side Scaling
* Also known as “Scalable I/O” on Linux systems
♦ Especially important with multi-core and multiprocessor systems
♦ Without SNP and Receive-side Scaling, all
incoming network traffic interrupts are routed to
exactly one core
♦ With SNP and Receive-side Scaling, incoming
network traffic interrupts are distributed across the
cores and processors
♦ Requires a NIC that supports Receive-side Scaling
♦ Provides very good performance
© 2007 Demartek
24
Receive-side Scaling
© 2007 Demartek
25
TCP Offload Engine (TOE)
♦
♦
♦
♦
TCP Offload NIC handles all TCP/IP processing
Reduces CPU utilization on host server
Provides overall best performance
More expensive than server-class NICs with
Receive-side Scaling
© 2007 Demartek
26
CPU Utilization Graphs
Percent CPU Utilization
Sequential Write
Sequential Read
100
100
90
NIC-LOW
80
NIC-SVR
% CPU Utilization
% CPU Utilization
NIC-TOE
70
Queue depth = 4
Paths = 2
60
50
40
50
40
30
20
20
10
10
1K
2K
4K
8K
16K
32K
64K
128K
256K
512K
1M
Queue depth = 4
Paths = 2
60
30
0
512B
NIC-SVR
80
NIC-TOE
70
NIC-LOW
90
0
512B
1K
2K
4K
8K
16K
32K
64K
128K
256K
512K
1M
Block Size
Block Size
Affect of NIC type on CPU Utilization
© 2007 Demartek
27
Switches, Jumbo Frames
♦ Switches used for iSCSI should be Gigabit or better
♦ When jumbo frames are enabled, must be
enabled in the switch and all NICs connected to
that switch, including iSCSI initiators and targets
– Make sure all components are using the same jumbo
frame size
© 2007 Demartek
28
Storage Technology Items
♦
♦
♦
♦
♦
RAID type
Stripe sizes
Disk drive type
MPIO in the target
Caching
© 2007 Demartek
29
RAID Definitions
♦ RAID – Redundant Array of Inexpensive Disks
– RAID 0: Interleaving or “striping” data across two or
more disks
– RAID 1: Disk mirroring – same data written on two
different disks
– RAID 5: Data striping with parity across multiple disks
– RAID 6: Data striping with double parity across multiple
disks
– RAID 1+0 or RAID 10: combine RAID 1 and RAID 0
• Two variations: stripe first or mirror first
© 2007 Demartek
30
RAID Types
Data: “ABCD”
RAID 0
AC
RAID 1
BD
RAID 10
AC
ABCD ABCD
BD
AC
BD
RAID 5
A D
B
C
P1
RAID 6
A D
B
C
© 2007 Demartek
P1
P2
31
Stripe Sizes
♦ Size of data block that is striped across disk drives
in a RAID group
♦ I/O of larger than the stripe size causes the target
to perform extra work to complete the I/O
© 2007 Demartek
32
Disk Drive Type
♦ Enterprise class (SAS, SCSI, Fibre Channel)
– Designed to be operated 7 x 24 x 365
– Dual internal processors (data transfer, physical motion)
♦ Desktop class (SATA, ATA/IDE)
– Largest capacities, best price/capacity
♦ Notebook class
– Low power consumption, slower
♦ Consumer class
– Very low power consumption and weight
© 2007 Demartek
33
Drive Type Chart
Device
Enterprise
Desktop
Notebook
Consumer
Seek time
3 – 5 ms
8 – 10 ms
10 – 12 ms
~12 ms
Xfer rate (MB/s)*
70 – 120
60 – 80
30 – 60
10 – 30
RPM (K)
10, 15
5.4, 7.2, 10
4.2, 5.4, 7.2
3.6, 4.2
Capacities
Large
Very large
Medium
Small
Processors
2
1
1
1
Cmd. Queuing
TCQ
NCQ
-
-
Power need
High
Medium
Low
Very low
5 years
3 – 5 years
1 – 3 years
1 year
Warranty
* Maximum device transfer rate from drive surface to buffer. Sustained rates are lower. This is not the same
as the interface transfer rate.
© 2007 Demartek
34
Caching
♦ Different iSCSI target solutions use different
caching
– Some are based on Windows Server platform
– Some are based on Linux platform
– Some use proprietary platform
© 2007 Demartek
35
Solutions Available Today
♦ Most of the major disk storage vendors provide
iSCSI solutions
♦ Several iSCSI-only vendors are producing very
good solutions
♦ Wide variety of price, capacity, performance and
advanced feature selection is available
♦ Generally speaking, iSCSI storage vendors have
done a good job of making tasks such as LUN
provisioning very easy to accomplish
© 2007 Demartek
36
Microsoft iSCSI Best Practices
♦ Use true server-class NIC
♦ Separate iSCSI traffic from all other network
traffic
♦ Use enterprise-class managed switches
© 2007 Demartek
37
Performance
♦ I have personally used iSCSI target solutions that:
– Achieve greater than 30,000 IOPs (with two paths)
– Achieve greater than 200 MB/sec (with two paths)
♦ You can have great performance with iSCSI
© 2007 Demartek
38
Futures
♦ 10-Gigabit Ethernet
– Currently used primarily in clustering and network
trunking applications
– Expect prices to drop, especially as PCI-Express
becomes the dominant bus type in servers
♦ Storage Management Software
– Expect iSCSI target vendors to make more information
available to centralized storage management software
solutions
♦ Microsoft iSCSI Deployment Paper – soon!
© 2007 Demartek
39
Recommendations
♦ Make sure you have Scalable Networking Pack
installed and use either Receive-side Scaling
capable NICs or TCP Offload Engines
♦ Take advantage of MPIO
© 2007 Demartek
40
References
♦ Microsoft Storage www.microsoft.com/storage/
♦ Microsoft iSCSI Storage
www.microsoft.com/WindowsServer2003/technologies/storage/iscs
i/default.mspx
♦ Microsoft Storage Technical Articles and White Papers
www.microsoft.com/windowsserversystem/storage/indextecharticle.
mspx
♦ Microsoft Scalable Networking Pack
www.microsoft.com/technet/network/snp/default.mspx
♦ Microsoft Exchange Solution Reviewed Program – Storage
technet.microsoft.com/en-us/exchange/bb412165.aspx
© 2007 Demartek
41
Post-Presentation Survey
♦ How many would be interested in a day-long,
vendor-neutral, storage class?
♦ What topics would be interesting to you in such a
class?
© 2007 Demartek
42
Thank You
Dennis Martin
Demartek
(303) 940-7575
Dennis @ demartek.com
www.demartek.com
© 2007 Demartek
43