Transcript No Slide Title

Archive Task Team (ATT)
Disk Storage
Stuart Doescher, USGS
(Ken Gacke)
WGISS-18
September 2004
Beijing, China
Storage Technologies
• Online Storage
– Direct Attach Storage (DAS)
• Storage directly attached to server
– Network Attach Storage (NAS)
• TCP/IP access to storage typically with CIFS and NFS access
– Storage Area Network (SAN)
• Dedicated high speed network connecting storage devices
• Storage devices disassociated from server
Storage Technologies
• Direct Attach Online Storage
– Disk is direct attached to single server
– System Configuration
• SCSI or Fibre Channel
– RAID Fibre Channel devices are typically SAN
ready
• Just a Bunch of Disk (JBOD)
• Redundant Array Independent Disk (RAID)
– High Performance on the local server
– Manageability
• Simple Configuration
• Resource reallocation requires physical move of
controllers and disk
Storage Technologies
Direct Attached
100Mb Network (FTP/NFS)
Host A
Host B
Host C
File System
File System
100MB FC
File System
Storage Technologies
• Direct Attach Online Storage
– Advantages
• High performance on local server
• Good for image processing and database applications
– Disadvantages
• Data sharing limited to slower network performance
• Difficult to reallocate resources to other servers
Storage Technologies
• NAS Online Storage
– Disk attached on server accessible over TCP/IP
Network
– System Configuration
• Fibre Channel RAID Configurations
• Switched Network Environment
– Performance
• Network Switches and/or dedicated network topologies
– Reliability
• NAS Server performs a single function thereby reducing
faults
• RAID, Mirror, Snapshot capabilities
– Easy to Manage
Storage Technologies
1Gb Network (NFS/CIFS)
Host A
Network Attached
Host B
Host C
NAS
Server
File System
File System
File System
Share Files
Storage Technologies
• Network Attach Online Storage
– Advantages
• Easy to share files among servers
– Network Storage support NFS and CIFS
– Servers can use existing network
infrastructure
• Good for small file sharing such as office
automation
• Availability of fault protection such as snapshot
and mirroring
– Disadvantages
• Slower performance due to TCP/IP overhead
• Increases network load
• Backup/Restore to tape may be difficult and/or
slow
• Does not integrate with nearline storage
Storage Technologies
• SAN Online Storage
– Disk attached within Fabric Network
– System Configuration
• Fibre Channel
• RAID Configurations
– Scalable High Performance
– High Reliability with redundant paths
– Manageability
• Configuration becomes more complex
• Logical reallocation of resources
Storage Technologies
Redundancy SAN Configuration
100Mb Network
Host A
Fibre
Switch
Host B
Fibre
Switch
Host C
(DMF)
Storage Technologies
• SAN Online Storage Architecture
– Disk Farm
• Multiple servers share large disk farm
• Server mounts unique file systems
– Clustered File Systems
• Multiple servers share a single file system
• Software Required – Vendor solutions include
– SGI CXFS
– ADIC StorNext File System
– Tivoli SANErgy
Storage Technologies
Disk Farm SAN Configuration
100Mb Network
Host A
Host B
Host C
Fibre
Switch
Logical
reallocation
of disk
Storage Technologies
• SAN Risks
– Cost is higher than DAS/NAS
– Technology Maturity
•
•
Solutions are typically vendor specific
Application software dependencies
– Infrastructure Support
•
•
•
Complexity of Architecture
Management of SAN Resources
Sharing of storage resources across multiple
Programs/Projects
Storage Technologies
• SAN Benefits
– Administration flexibility
• Logically move disk space among servers
• Large capacity drives can be sliced into smaller
file systems
• Scales better than direct attach
• Integrate within nearline configuration
– Data Reliability
• Storage disassociated from the server
• Fault Tolerant with Redundant Paths
– Increase Resource Utilization
• Reduce the number of FTP network transfers
• Logically allocate space among servers
Storage Technologies
SAN with Nearline Configuration
1Gb Network
Host A
Fibre
Switch
Clustered
File System
Host B
CXFS
Tape Library
Host C
DMF/CXFS
CR1 SAN/Nearline Architecture
Ethernet
DMF Server
Product Distribution
Disk Cache
/dmf/edc 68GB
/dmf/doqq 547GB
/dmf/guo 50GB
/dmf/pds 223GB
/dmf/pdsc 1100GB
Tape Drives
8x9840
2x9940B
Future Seamless/Silo
Architecture
Ethernet
CIFS
Mount
Data Servers
FTP (lxs37)
Web/Extract
DMF
PDS
TP9400 TP9300S
3TB
Tape Library
8x9840
3x9940B
Storage Technologies
• Multi Tiered Storage Vision
– Online
• Supported Configurations
– DAS – Local processing such as image
processing
– NAS – Data sharing such as office automation
– SAN – Production processing such as product
generation
• Data accessed frequently
– Nearline
• Integrated within SAN
• Scalable for large datasets and less frequently
accessed data
• Multiple Copies and/or Offsite Storage
Storage Technologies
• SAN – Final Thoughts
– SAN Technology Maturity
• SAN solution should be from a single vendor
– Program/Project SAN solution benefits
+ Decrease storage requirements
+ Increase performance
+ Increase reliability
+ Increase flexibility of resource allocations
- Increase cost (hardware/software)
- Increase configuration complexity
Storage Architecture Decisions
• Optimized by individual program and program
manager, not the enterprise.
• Requirements Factors
–
–
–
–
–
Reliability – Data Preservation
Performance – Data Access
Cost – $/GB, Engineering Support, O&M
Scalability – Data Growth, Multi-mission, etc.
Compatibility with current Architecture
• Evaluated and recommended through
engineering white papers and weighted decision
matrices.
Bulk RAID Weighted Matrix
Selecton Criteria
Initial Cost
Support Cost
Vendor Support
EDC Experience
Performance
Reliability
Manageability
Scalability
SAN Ready
Upgradeable
Weighted Score
Initial Cost
Support Cost
Vendor Support
EDC Experience
Performance
Reliability
Manageability
Scalability
SAN Ready
Upgradeable
Total Weighted Score
RW #
10
10
2
0
5
1
5
1
1
1
Nexsan
Ataboy2
10
10
5
6
6
6
7
5
8
5
CE
SATA
10
9
3
5
6
3
4
5
0
3
EMC
SATA
7
5
9
6
7
7
7
7
8
9
STK
B220
9
8
9
7
7
7
9
7
8
9
STK
D240
6
6
9
8
9
9
9
8
8
8
Nexsan
Ataboy2
100
100
10
0
30
6
35
5
8
5
299
CE
SATA
100
90
6
0
30
3
20
5
0
3
257
EMC
SATA
70
50
18
0
35
7
35
7
8
9
239
STK
B220
90
80
18
0
35
7
45
7
8
9
299
STK
D240
60
60
18
0
45
9
45
8
8
8
261
High Performance RAID Weighted
Matrix
Selecton Criteria
Initial Cost
Support Cost
Vendor Support
EDC Experience
Performance
Reliability
Manageability
Scalability
SAN Ready
Upgradeable
Weighted Score
Initial Cost
Support Cost
Vendor Support
EDC Experience
Performance
Reliability
Manageability
Scalability
SAN Ready
Upgradeable
Total Weighted Score
RW #
9
9
8
6
8
9
8
6
4
4
EMC
CX300
8
4
9
7
8
9
7
7
8
9
EMC
CX500
5
4
9
7
8
9
7
8
8
9
STK
D240
6
6
9
8
9
9
9
8
8
8
STK
D220
7
6
9
8
8
9
9
7
8
8
Ciprico
FibreSt
4
5
8
7
6
6
6
7
6
7
Adaptec NexSan
SANbloc Ataboy
10
10
10
10
7
5
5
6
7
6
7
6
7
7
7
5
8
8
7
5
EMC
CX300
72
36
72
42
64
81
56
42
32
36
533
EMC
CX500
45
36
72
42
64
81
56
48
32
36
512
STK
D240
54
54
72
48
72
81
72
48
32
32
565
STK
D220
63
54
72
48
64
81
72
42
32
32
560
Ciprico
FibreSt
36
45
64
42
48
54
48
42
24
28
431
Adaptec NexSan
SANbloc Ataboy
90
90
90
90
56
40
30
36
56
48
63
54
56
56
42
30
32
32
28
20
543
496
Disk & Tape Systems
Additional Reference materials
•
•
•
•
0403 Storage Matrix.doc
040419 SAN Forum.ppt
0407 storage overview.ppt
EDC CR1 Storage Architecuture.ppt