Transcript TRIUMF Site Report - HEPiX Services at CASPUR

TRIUMF Site Report
HEPiX/HEPNT
NIKHEF, Amsterdam
May 19-23/2003
Corrie Kost
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
Overview
•
•
•
•
•
Move to new facility
Printing moves to CUPS
Print/Copy/Scan-to-Email Integration
WestGrid components at TRIUMF
C4-Testbed Proposal
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
Move to ISAC II Building
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
New Facilities in ISAC II
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
REVISED LAN
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
Current LAN
ISAC II
Cyclotron Controls
ISAC II Experiments
Pet Group
INTERNET
ISAC II Offices
ISAC Controls
UNIVERSITY
Data Center
UBC
ISAC I Offices
Commercial
Spare
BCNET
Gigapop
e2e Lightpath
CWDM
Edu/Research
CWDM
ISAC I Experiments
WestGRID
Cyclotron Operations
ISAC I
PASSPORT
8600
Meson &
Proton Halls
Main Office
Electronics Devel
Trailer Gg
Main Office 2nd Floor
Proton Exp Hall
Proton Annex
Gerber Room
Musr Facility
Detector Facility
Trailer Complex
Meson Hall
Main Office
Complex
Chemistry Annex
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
Move to New Facility
• Weekend of March 29/2003 – 2 Passport 8600’s
• Failure of air conditioner March 29 pm
• Web/file server disk problems
–
–
–
–
–
–
Old system failed – replaced
New system (disk) failed during backup
Diskcopy does not work for EXT3
DD gave ~37,000 lost&found entries
Murphy’s law
Mirrored on spare linux system – last gasp
• Legacy clean-up / move to racks / change servers
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
Common Unix Printing System
CUPS
Components
• CUPS
• Foomatic (ppd)
• gs
• Samba
PRINT SERVER A (printcap)
PRINT SERVER B (CUPS)
Stage 1 : Separate Windows and Linux Print Servers
Stage 2 : Use SAMBA to use a single Linux Print Server
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
PRINT/COPY/SCAN(EMAIL)
Integration
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
WestGrid – UBC/TRIUMF
• 1000 Cpu’s
• 10 TBytes Disk
• 75 Tbytes Tape
SUN
HP
IBM
INTERDYNAMIX
DELL
ATAPA
LINUXNETWORKX RACKSAVER
And the winner is……
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
IBM
WestGrid Solution
For TRIUMF/UBC
• 1008 3.06GHz Xeons
• 14 Blades (28 CPU)/Chassis
• 533 MHz FSB
• 2GB memory /node
• 40GB drive/node
• 36 Chassis in 6 racks
• ~ 20Kwatts power/rack
• ~ 400,000 Btu/hr (33 tons)
• 40% less heat/power than 1U
• ~ 1500 lbs/rack
• 3.2 TeraFlop (peak)
• 1.2-1.4Tflops (linpack)
• 4*6*6 =144 Gige into 232 port
Foundry Switch – max 756 nodes
• 286Mbits/sec unblocked
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
CA*net 4 Testbed Project
Proposed Testbed Network Topology
7+ Nodes hosts a 1.5 Terabyte Linux file server, each
with 2 GbE optical interfaces, and a 10/100/1000
copper ethernet to provide LAN management.
Netherlight
OC192
(10Gbps)
Starlight
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003
CA*net 4 Testbed Project
Software Packages
RedHat Linux 7.3 or 8.0
Globus Toolkit 2.2
OpenAFS
Sun Grid Engine
Sun J2EE SDK
iperf
Tsunami
bbFTP
Web 100
Net 100
GNU Zebra 0.93a
Software Matrix
Disk Server











Router

TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003




CA*net 4 Testbed Project
•
•
•
•
Project time contributions ~ 24 months
International in scope
Request persistent testbed
End-to-end lightpaths by user – to 10Gbps
– Quality of service
– Guaranteed bandwidth
– Stable network characteristics
• Test long-distance transfer protocols
– 10Gbit Ethernet, Remote Direct Memory Access
RDMA/IP, FibreChannel/IP, serial SCSI, HyperSCSI etc
TRIUMF Site Report – HEPiX/HEPNT – NIKHEF, Amsterdam, May 19-23/2003