DISTRIBUTED-MEMORY OR GRAPH
MODELS
Given the internal processor and memory structures in
each node, a distributed-memory
architecture is characterized primarily by the network
used to interconnect the nodes (Fig. 4.5).
This network is usually represented
DATA BROADCASTING
Because data broadcasting is a fundamental building-block
computation in parallel
processing, we devote this section to discussing how it can
be accomplished on the various
PRAM submodels. Semigroup computation, parallel prefix
computati
PRAM SUBMODELS AND
ASSUMPTIONS
As mentioned in Section 4.4, the PRAM model prescribes
the concurrent operation of
p processors (in SIMD or MIMD mode) on data that are
accessible to all of them in an m-word
shared memory. In the synchronous SIMD or SPMD
ve
CIRCUIT MODEL AND PHYSICAL
REALIZATIONS
In a sense, the only sure way to predict the performance of
a parallel architecture on a
given set of problems is to actually build the machine and
run the programs on it. Because
this is often impossible or very co
SEMIGROUP OR FAN-IN
COMPUTATION
Semigroup computation was defined in Section 2.1 based
on associative binary operator
. This computation is trivial for a CRCW PRAM of the
reduction variety if the reduction
operator happens to be . For example, computing t
PARALLEL PREFIX COMPUTATION
Just as was the case for a linear array (Section 2.3),
parallel prefix computation consists
of the first phase of the semigroup computation. We see in
Fig. 5.4 that as we find the
semigroup computation result in S[p 1], all par
A PARALLEL SELECTION
ALGORITHM
If a parallel computation model supports fast sorting, then
the n-input parallel selection
problem can be solved through sorting. This is the case,
e.g., for the CRCW PRAM-S or
summation submodel with p = n processors as sho
SEQUENTIAL RANK-BASED
SELECTION
Rank-based selection is the problem of finding a (the) kth
smallest element in a sequence
S = x0 , x 1 , . . . , x n -1 whose elements belong to a linear
order. Median, maximum, and
minimum finding are special cases of this
MATRIX MULTIPLICATION
In this section, we discuss PRAM matrix multiplication
algorithms as representative
examples of the class of numerical problems. Matrix
multiplication is quite important in its
own right and is also used as a building block in many
o
RANKING THE ELEMENTS OF A LINKED
LIST
Our next example computation is important not only
because it is a very useful building
block in many applications, but also in view of the fact that
it demonstrates how a problem
that seems hopelessly sequential can