Transcript Seaborg - MSU Computer Science

Seaborg
Cerise Wuthrich
CMPS 5433
Seaborg

Manufactured by IBM
 Distributed Memory Parallel Supercomputer
 Based on IBM’s SP RS/6000 Architecture
Seaborg
 Used
by National Energy Research
Scientific Computing Center (funded by
Department of Energy) at Berkeley Lab
 Named for Glenn Seaborg – Nobel
laureate chemist who discovered 10
atomic elements, including plutonium
IBM SP RS/6000 Architecture
SP –
Scalable
Power
Parallel
 RS – RISC
System
 Composed
of nodes

Nodes
 416
nodes with 16 processors/node
 380
compute nodes
 20 nodes used for disk storage
 6 login nodes
 2 network nodes
 8 spares
Front and back view of nodes
Node Architecture
 16
IBM Power3
processors per
node
 Between 16 – 64
GB Memory per
node
 2 network
adapters per
node
Processors

IBM Power3 processors each running at 375 MHz
 Power – Performance Optimized With Enhanced
RISC
 PowerPC processors are RISC-based symmetric
multiprocessors (every processor is functionally
identical) with 64-bit addressability
 Connected to L2 cache by bus running at 250 MHz
 Dynamic Branch Prediction
 Instruction prefetching
 FP units are fully pipelined
 4 FLOP/cycle x 375 MHz = 1500 Million or 1.5
GFLOPS/sec
Power PC 3 processor
32 KB
8 MB
64KB
Power3 Processor
15 million transistors
Interconnection Network
 Nodes
connected with high bandwidth,
low latency IBM SP2 switch
 Can be connected in various topologies
depending on number of nodes
 Each switchboard has up to 32 links
 16
links to nodes
 16 links to other
switchboards
Interconnection Network
Star Topology
used for up to
80 nodes and
still guarantee
4 independent
shortest paths
Interconnection Network
Intermediate
switchboards
must be added
for 81-256
nodes
Interconnection Network
 The
combination of HW and SW of the
switch system is known as the CSS –
Communication SubSystem
 Network is highly available
Latency in the network
 Within
nodes,
latency is 9
microseconds
 Between nodes,
using Message
Passing Interface,
the latency is 17
microseconds
Scalability
 Architecture
can handle from 1 – 512
nodes
 The current version of Seaborg (2003)
is twice the size of the original (2001)
Memory
 Within
each node, between 16 & 64 GB
of shared memory
 Between nodes, there is distributed
memory
 Parallel programs can be run using
distributed memory message passing,
shared memory threading or a
combination
I/O
 20
nodes run the distributed parallel I/O
system called GPFS – General Parallel
File System
 44 Terabytes of disk space
 Each node runs its own copy of AIX –
IBM’s Unix-based OS
Production Status/Cost
 $33
Million for the first version put into
operation in June 2001
 At the time, it was the 2nd most powerful
computer in the world and the most
powerful one for unclassified research
 In 2003, number of nodes was doubled
Customers
 2100
researchers at national labs and
universities across the country
 Restricted to Department of Energy
funded massively parallel processing
projects
 Located at the National Energy
Research Computing Center
Applications

Massively Parallel Scientific Research
 Gasoline Combustion Simulation
 Fusion Energy Research
 Climate Modeling
 Materials Science
 Computational Biology
 Particle Simulations
 Plasma Acceleration
 Large Scale Simulation of Atomic Structures
Interesting Features

In 2004, 2.4 times
as many requests
as resources
available
 Uses POE (Parallel
Operating
Environment) and
LoadLeveler to
schedule jobs
Survey Results – Why do you
use Seaborg?





Need massively parallel
computer
High speed
Achieves level of
numerical accuracy
Can run several
simulations in parallel
Easy to connect using
ssh

Fastest and most
efficient computer
available for my
research
 Long queue times
are great
 Large enough
memory for my
needs
Survey Results – How could
Seaborg be improved?

I think too many nodes are scheduled for
many jobs. Scaling is not good in many
cases.
 “..Virtually impossible to do interactive work”
 “Debuggers are terrible.”
 “Compilers and debuggers are a step down
from the Cray.”
 Giving preference to high concurrency jobs
makes smaller jobs wait
Questions