Introduction to Parallel Computing
George Karypis
Analytical Modeling of Parallel Algorithms
Sources of Overhead in Parallel Programs
The total time spent by a parallel system is usually higher than that spent by a serial system to solve the same problem.
Introduction to Parallel Computing
George Karypis
Basic Communication Operations
Outline
Importance of Collective Communication Operations One-to-All Broadcast All-to-One Reduction All-to-All Broadcast & Reduction All-Reduce & Prefix-Sum Scatter and Gathe
Introduction to Parallel Computing
George Karypis Principles of Parallel Algorithm Design
Outline
Overview of some Serial Algorithms Parallel Algorithm vs Parallel Formulation Elements of a Parallel Algorithm/Formulation Common Decomposition Methods
concu
Introduction to Parallel Computing
George Karypis Parallel Programming Platforms
Elements of a Parallel Computer
Hardware
Multiple Processors Multiple Memories Interconnection Network
System Software
Parallel Operating System Programming Constructs to Exp
Introduction to parallel Computing
P. B. Sunil Kumar Department of Physics IIT Madras, Chennai 600036 www.physics.iitm.ac.in/~sunil
Friday 19 November 2010
What is high performance computing ?
Large volume numerical calculations : Set up facilites
to run
`F o~ebws
v
W yxjC `z !
s
k s `EC
v
wb
!
q
b~
F
sGi
`f 5
s
` G iisl 4
kisshWibsPi
ikjkbFfv5hmmfq(fWg|bFbfw(v#s#
q t h u vs t h u u v h
p#bisiwRhubFbwsmRtiukiu`mWvb FwhPitg
x t d t
Chapter 1
Preliminaries
1.1. Introduction
We give some necessary background. The rst section gives the notational
conventions to be used in the book. In the second section, we give some
elementary results and denitions from linear algebra and probability.