Transcript The IC Wall Collaboration between Computer science + Physics

Parallel Programming
Henri Bal
([email protected])
Vrije Universiteit
Faculty of Sciences
Amsterdam
Overview
•
•
•
•
What is parallel programming?
Why do we need parallel programming?
Organization of this course
Practicum Parallel Programming
ICWall
•A large display suitable for:
Education (classroom)
– Modern scientific applications that
need to visualize large data-sets
– 3D applications
–
ICWall: an "off-the-shelf" video wall
•High-end graphics is very expensive
•Use tiled display with many projectors
•Use standard (inexpensive) components
–
9 PCs, graphic and network cards, 16 projectors
•Similar idea as cluster computing,
but applied to graphics
•Current ICWall is strongly simplified version
–1 PC with 2 projectors
Parallel Programming
• Sequential programming
Single thread of control
• Parallel programming
Multiple threads of control
• Why parallel programming?
Eases programming? Not really.
Performance? Yes!
Famous quote
• “Parallel programming may do something to revive
the pioneering spirit in programming,
which seems to be degenerating into
a rather dull and routine occupation’’
• S. Gill, Computer Journal, 1958
Why do we need parallel processing?
• Many applications need much faster machines
• Sequential machines are reaching their speed limits
– Moore’s Law (1975): Circuit complexity doubles every 18 months
– Using many transistors effectively to increase performance becomes critical problem
• Memory becomes a bottleneck
– DRAM access times improve only 10% per year
– Caches more and more important
Parallel processing
• Use multiple processors to solve large problems fast
– Also gives more cache memory and aggregate memory bandwidth
• Microprocessors are getting cheaper and cheaper
• Cheap multiprocessors and multicore CPUs
bring parallel processing to the desktop!
History
•
•
•
•
•
•
•
1950s: first ideas (see Gill’s quote)
1967 first parallel computer (ILLIAC IV)
1970s programming methods, experimental machines
1980s: parallel languages (SR, Linda, Orca), commercial supercomputers
1990s: software standardization (MPI), clusters, large-scale machines (Blue Gene)
2000s: grid computing: combining resources world-wide (Globus)
Now: much research on multi-cores, Graphical Processor Units (GPUs), IBM Cell, ….
Large-scale parallel machines
• Many parallel machines exist
• See http://www.top500.org
• Current #1: RoadRunner Cell cluster
131072
with 122400 cores
• #2 is IBM BlueGene/L
with 212992 cores
Challenging Applications
•
•
•
•
•
•
Modeling ozone layer, climate, ocean
Quantum chemistry
Protein folding
General: computational science
Aircraft modeling
Handling use volumes of data from scientific instruments
– Lofar (astronomy)
– LHC (CERN, high-energy physics)
• Computer chess
• Analyzing multimedia content
• Generating movies
Application areas
•
•
•
•
Engineering and design
Scientific applications
Commercial applications (e.g., transactions, databases)
Embedded systems (e.g., cars)
• This class focuses on scientific applications
About this Course
Goal: Study how to write programs that run in parallel on a large number of machines.
Focus on programming methods, languages, applications
Focus on distributed-memory (message passing) machines, not (yet) on multi-cores,
special purpose (GPU, Cell) machines
Prerequisites:
Some knowledge about sequential languages
Little knowledge about networking and operating systems
Aspects of Parallel Computing
Algorithms and applications
Programming methods, languages, and environments
Parallel machines and architectures
Course Outline
• Introduction in algorithms and applications
• Parallel machines and architectures
Overview of parallel machines, trends in top-500
Cluster computers, BlueGene
• Programming methods, languages, and environments
Message passing (SR, MPI, Java)
Higher-level language: HPF
• Applications
N-body problems, search algorithms, bioinformatics
• Grid computing
Multimedia content analysis on Grids (guest lecture Frank Seinstra)
Course Information
Examination
Written exam based on:
- Reader: available electronically from Blackboard
- Lectures
More information (and slides):
http://www.cs.vu.nl/~bal/college08.html
Practicum Parallel Programming
Separate practicum (6 ECTS)
• Implement ASP + SOR algorithms in C/MPI
• Implement Capture Go search algorithm in Java/RMI
• Test and measure the programs on our DAS cluster
More information
Register by email to [email protected] (Niels Drost) now
Starts after this course (mid October)
See http://www.cs.vu.nl/pp-cursus/ for a complete description