Transcript Overview of the Program Slides

SIAC 2000 Program
1
SIAC 2000 at a Glance
AM
Lunch
PM
Mon
NOW
HPC
Tue
PVMMPI
Clusters
Wed
Condor
HPVM
Sun
Dinner
Condor
Globus Clusters
HPVM
2
SIAC 2000 Materials: Book 1
• David Spector, “Building Linux
Clusters: Scaling Linux for
Scientific and Enterprise
Applications,” August 2000
• CD-ROM RedHat Linux, PVM, …
• O'Reilly & Associates, ISBN:
1565926250
3
SIAC 2000 Materials: Book 2
• Rajkumar Buyya (Editor), “High
Performance Cluster Computing:
Programming and Applications”
Volume 2, June 1999
• Prentice Hall; ISBN: 0130137855
4
SIAC 2000 Materials: Book 3
• Selection of papers
– Linux clusters
– Beowulf HOWTO
– PVM, MPI
• FAQs
• Handouts from speakers
5
Introduction to
Cluster Computing
Prabhaker Mateti
Wright State University
Dayton, Ohio
6
Overview
• High performance computing
• High throughput computing
• NOW, HPC, and HTC
• Parallel algorithms
• Software technologies
7
“High Performance” Computing
• CPU clock frequency
• Parallel computers
• Alternate technologies
– Optical
– Bio
– Molecular
8
“Parallel” Computing
• Traditional supercomputers
– SIMD, MIMD, pipelines
– Tightly coupled shared memory
– Bus level connections
– Expensive to buy and to maintain
• Cooperating networks of
computers
9
“NOW” Computing
• Workstation
• Network
• Operating System
• Cooperation
• Distributed (Application) Programs
10
Traditional Supercomputers
• Very high starting cost
– Expensive hardware
– Expensive software
• High maintenance
• Expensive to upgrade
11
Traditional Supercomputers
No one is predicting their demise,
but …
12
Computational Grids
are the future
13
Computational Grids
“Grids are persistent environments
that enable software applications
to integrate instruments, displays,
computational and information
resources that are managed by
diverse organizations in
widespread locations.”
14
Computational Grids
• Individual nodes can be
supercomputers, or NOW
• High availability
• Accommodate peak usage
• LAN : Internet :: NOW : Grid
15
Globus: A Computational Grid
• Lee Liming, Argonne
• Monday,
Aug 21, 2000,
1:00 – 5:30 PM
16
“NOW” Computing
• Workstation
• Network
• Operating System
• Cooperation
• Distributed+Parallel Programs
17
Workstation? PC? Mac?
18
“Workstation Operating System”
• Authenticated users
• Protection of resources
• Multiple processes
• Preemptive scheduling
• Virtual Memory
• Hierarchical file systems
• Network centric
19
Network
• Ethernet
– 10 Mbps obsolete
– 100 Mbps common
– 1000 Mbps desired
• Protocols
– TCP/IP
20
Cooperation
• Workstations are “personal”
• Others use slows you down
•…
• Willing to share
• Willing to trust
21
Distributed Programs
• Spatially distributed programs
– A part here, a part there, …
– Parallel
– Synergy
• Temporally distributed programs
– Compute half today, half tomorrow
– Combine the results at the end
• Migratory programs
– Have computation, will travel
22
Technological Bases of
Distributed+Parallel Programs
• Spatially distributed programs
– Message passing
• Temporally distributed programs
– Shared memory
• Migratory programs
– Serialization of data and programs
23
Distributed Shared Memory
• “Simultaneous” read/write access
by spatially distributed processors
• Abstraction layer of an
implementation built from
message passing primitives
• Semantics not so clean
24
Technological Bases for
Migratory programs
• Same CPU architecture
– X86, PowerPC, MIPS, SPARC, …, JVM
• Same OS + environment
• Be able to “checkpoint”
– suspend, and
– then resume computation
– without loss of progress
25
Development of
Distributed+Parallel Programs
• New code + algorithms
• Old programs rewritten in new
languages that have distributed
and parallel primitives
• Parallelize legacy code
26
New Programming Languages
• With distributed and parallel
primitives
• Functional languages
• Logic languages
• Data flow languages
27
Parallel Programming Languages
• based on the shared-memory
model
• based on the distributed-memory
model
• parallel object-oriented languages
• parallel functional programming
languages
• concurrent logic languages
28
PVM, and MPI
• Message passing primitives
• Can be embedded in many existing
programming languages
• Architecturally portable
• Open-sourced implementations
29
PVM, and MPI
• Prabhaker Mateti, WSU
• Tuesday
Aug 22, 2000
8:30 – 12:00
30
OpenMP
• Distributed shared memory API
• Implementations: Real soon now
• http://www.openmp.org/
31
SPMD
• Single program, multiple data
• Contrast with SIMD
• Same program runs on multiple
nodes
• May or may not be lock-step
• Nodes may be of different speeds
• Barrier synchronization
32
Condor
• Cooperating workstations
• Migratory programs
– Checkpointing
– Remote IO
• Resource matching
33
Condor
• Prabhaker Mateti, WSU
• Wednesday
Aug 22, 2000
8:30 – 12:00
34
Clusters of Workstations
• Inexpensive alternative to
traditional supercomputers
• High availability
– Lower down time
– Easier access
• Development platform with
production runs on traditional
supercomputers
35
Linux Clusters
• Kumaran Kalyanasundaram, SGI
• Monday Aug 21, 2000
6:30 – 9:00 PM
• Tuesday Aug 22, 2000
1:00 – 5:30 PM
36
HPVM Clusters
• Mario Lauria
• Wednesday
Aug 23, 2000
1:00 – 4:00 PM
37