PARALLEL PROCESSING
The quest for higher-performance digital computers seems
unending. In the past two
decades, the performance of microprocessors has enjoyed
an exponential growth. The growth
of micr
ROADBLOCKS TO PARALLEL
PROCESSING
Over the years, the enthusiasm of parallel computer
designers and researchers has been
counteracted by many objections and cautionary
statements. The most important o
TYPES OF PARALLELISM: A
TAXONOMY
Parallel computers can be divided into two main categories
of control flow and data
flow. Control-flow parallel computers are essentially based
on the same principles
PARALLEL PROCESSING UPS AND
DOWNS
L. F. Richardson, a British meteorologist, was the first
person to attempt to forecast the
weather using numerical computations. He started to
formulate his method du
SIMPLE COMPUTATIONS
In this section, we define five fundamental building-block
computations:
1. Semigroup (reduction, fan-in) computation
2. Parallel prefix computation
3. Packet routing
4. Broadcasti
SOME SIMPLE ARCHITECTURES
In this section, we define four simple parallel
architectures:
1. Linear array of processors
2. Binary tree of processors
3. Two-dimensional mesh of processors
4. Multiple pr
ALGORITHMS FOR A 2D MESH
In all of the 2D mesh algorithms presented in this section,
we use the linear-array
algorithms of Section 2.3 as building blocks. This leads to
simple algorithms, but not
nece
ALGORITHMS FOR A BINARY
TREE
In algorithms for a binary tree of processors, we will
assume that the data elements are
initially held by the leaf processors only. The nonleaf
(inner) processors partici
Packet Routing
To send a packet of information from Processor i to
Processor j on
a linear array, we simply attach a routing tag with the
value j i to it. The sign of a routing
tag determines the dire
ALGORITHMS FOR A LINEAR
ARRAY
Semigroup Computation. Let us consider first a special
case of semigroup computation,
namely, that of maximum finding. Each of the p processors
holds a value initially an