PARALLEL PROCESSING
The quest for higher-performance digital computers seems
unending. In the past two
decades, the performance of microprocessors has enjoyed
an exponential growth. The growth
of microprocessor speed/performance by a factor of 2 every
18
ROADBLOCKS TO PARALLEL
PROCESSING
Over the years, the enthusiasm of parallel computer
designers and researchers has been
counteracted by many objections and cautionary
statements. The most important of these are
listed in this section [Quin87]. The list b
TYPES OF PARALLELISM: A
TAXONOMY
Parallel computers can be divided into two main categories
of control flow and data
flow. Control-flow parallel computers are essentially based
on the same principles as the
sequential or von Neumann computer, except that
PARALLEL PROCESSING UPS AND
DOWNS
L. F. Richardson, a British meteorologist, was the first
person to attempt to forecast the
weather using numerical computations. He started to
formulate his method during the First
World War while serving in the army ambu
SIMPLE COMPUTATIONS
In this section, we define five fundamental building-block
computations:
1. Semigroup (reduction, fan-in) computation
2. Parallel prefix computation
3. Packet routing
4. Broadcasting, and its more general version, multicasting
5. Sorti
SOME SIMPLE ARCHITECTURES
In this section, we define four simple parallel
architectures:
1. Linear array of processors
2. Binary tree of processors
3. Two-dimensional mesh of processors
4. Multiple processors with shared variables
Linear Array. Figure 2.2
ALGORITHMS FOR A 2D MESH
In all of the 2D mesh algorithms presented in this section,
we use the linear-array
algorithms of Section 2.3 as building blocks. This leads to
simple algorithms, but not
necessarily the most efficient ones. Mesh-based
architectur
ALGORITHMS FOR A BINARY
TREE
In algorithms for a binary tree of processors, we will
assume that the data elements are
initially held by the leaf processors only. The nonleaf
(inner) processors participate in the
computation, but do not hold data elements
Packet Routing
To send a packet of information from Processor i to
Processor j on
a linear array, we simply attach a routing tag with the
value j i to it. The sign of a routing
tag determines the direction in which it should move (+ =
right, = left) while
ALGORITHMS FOR A LINEAR
ARRAY
Semigroup Computation. Let us consider first a special
case of semigroup computation,
namely, that of maximum finding. Each of the p processors
holds a value initially and
our goal is for every processor to know the largest o