Unformatted text preview: AN APPROXIMATE MINIMUM DEGREE ORDERING ALGORITHM
TIMOTHY A. DAVIS , PATRICK AMESTOYy , AND IAIN S. DUFFz Computer and Information Sciences Dept., University of Florida,
Technical Report TR94039, December, 1994 (revised July 1995).
Abstract. An Approximate Minimum Degree ordering algorithm (AMD) for preordering a symmetric sparse matrix prior to numerical factorization is presented. We use techniques based on
the quotient graph for matrix factorization that allow us to obtain computationally cheap bounds
for the minimum degree. We show that these bounds are often equal to the actual degree. The
resulting algorithm is typically much faster than previous minimum degree ordering algorithms, and
produces results that are comparable in quality with the best orderings from other minimum degree
algorithms. Keywords: approximate minimum degree ordering algorithm, quotient graph,
sparse matrices, graph algorithms, ordering algorithms
AMS classi cations: 65F50, 65F05.
1. Introduction. When solving large sparse symmetric linear systems of the
form Ax = b, it is common to precede the numerical factorization by a symmetric
reordering. This reordering is chosen so that pivoting down the diagonal in order on
the resulting permuted matrix PAPT = LLT produces much less llin and work than
computing the factors of A by pivoting down the diagonal in the original order. This
reordering is computed using only information on the matrix structure without taking
account of numerical values and so may not be stable for general matrices. However,
if the matrix A is positivede nite 21], a Cholesky factorization can safely be used.
This technique of preceding the numerical factorization with a symbolic analysis can
also be extended to unsymmetric systems although the numerical factorization phase
must allow for subsequent numerical pivoting 1, 2, 16]. The goal of the preordering
is to nd a permutation matrix P so that the subsequent factorization has the least
llin. Unfortunately, this problem is NPcomplete 31], so heuristics are used.
The minimum degree ordering algorithm is one of the most widely used heuristics,
since it produces factors with relatively low llin on a wide range of matrices. Because
of this, the algorithm has received much attention over the past three decades. The
algorithm is a symmetric analogue of Markowitz' method 26] and was rst proposed
by Tinney and Walker 30] as algorithm S2. Rose 27, 28] developed a graph theoretical model of Tinney and Walker's algorithm and renamed it the minimum degree
algorithm, since it performs its pivot selection by selecting from a graph a node of
minimum degree. Later implementations have dramatically improved the time and
memory requirements of Tinney and Walker's method, while maintaining the basic
idea of selecting a node or set of nodes of minimum degree. These improvements have
reduced the memory complexity so that the algorithm can operate within the storage
of the original matrix, and have reduced the amount of work needed to keep track of
the degrees of nodes in the graph (which is the most computationally intensive part
Computer and Information Sciences Department University of Florida, Gainesville, Florida,
USA. phone: (904) 3921481, email: [email protected] .edu. Support for this project was provided by
the National Science Foundation (ASC9111263 and DMS9223088). Portions of this work were
supported by a postdoctoral grant from CERFACS.
y ENSEEIHTIRIT, Toulouse, France. email: [email protected]
z Rutherford Appleton Laboratory, Chilton, Didcot, Oxon. 0X11 0QX England, and European Center for Research and Advanced Training in Scienti c Computation (CERFACS), Toulouse,
France. email: [email protected]
1 2 P. AMESTOY, T. A. DAVIS, , AND I. S. DUFF of the algorithm). This work includes that of Du and Reid 10, 13, 14, 15]; George
and McIntyre 23]; Eisenstat, Gursky, Schultz, and Sherman 17, 18]; George and Liu
19, 20, 21, 22]; and Liu 25]. More recently, several researchers have relaxed this
heuristic by computing upper bounds on the degrees, rather than the exact degrees,
and selecting a node of minimum upper bound on the degree. This work includes
that of Gilbert, Moler, and Schreiber 24], and Davis and Du 7, 8]. Davis and Du
use degree bounds in the unsymmetricpattern multifrontal method (UMFPACK), an
unsymmetric Markowitzstyle algorithm. In this paper, we describe an approximate
minimum degree ordering algorithm based on the symmetric analogue of the degree
bounds used in UMFPACK.
Section 2 presents the original minimum degree algorithm of Tinney and Walker
in the context of the graph model of Rose. Section 3 discusses the quotient graph
(or element graph) model and the use of that model to reduce the time taken by
the algorithm. In this context, we present our notation for the quotient graph, and
present a small example matrix and its graphs. We then use the notation to describe
our approximate degree bounds in Section 4. The Approximate Minimum Degree
(AMD) algorithm and its time complexity is presented in Section 5. In Section 6,
we rst analyse the performance and accuracy of our approximate degree bounds on
a set of test matrices from a wide range of disciplines. The AMD algorithm is then
compared with other established codes that compute minimum degree orderings.
2. Elimination graphs. The nonzero pattern of a symmetric nbyn matrix,
A, can be represented by a graph G0 = (V 0; E0), with nodes V 0 = f1; :::; ng and
edges E 0. An edge (i; j ) is in E 0 if and only if aij 6= 0. Since A is symmetric, G0 is
undirected.
The elimination graph, Gk = (V k ; E k ), describes the nonzero pattern of the submatrix still to be factorized after the rst k pivots have been chosen. It is undirected,
since the matrix remains symmetric as it is factorized. At step k, the graph Gk depends on Gk 1 and the selection of the kth pivot. To nd Gk , the kth pivot node p
is selected from V k 1. Edges are added to E k 1 to make the nodes adjacent to p in
Gk 1 a clique (a fully connected subgraph). This addition of edges ( llin) means that
we cannot know the storage requirements in advance. The edges added correspond to
llin caused by the kth step of factorization. A llin is a nonzero entry Lij , where
(PAPT )ij is zero. The pivot node p and its incident edges are then removed from the
graph Gk 1 to yield the graph Gk . Let AdjG (i) denote the set of nodes adjacent to
i in the graph Gk . Throughout this paper, we will use the superscript k to denote a
graph, set, or other structure obtained after the rst k pivots have been chosen. For
simplicity, we will drop the superscript when the context is clear.
The minimum degree algorithm selects node p as the kth pivot such that the
degree of p, tp jAdjG 1 (p)j, is minimized (where j:::j denotes the size of a set or the
number of nonzeros in a matrix). The minimum degree algorithm is a nonoptimal
greedy heuristic for reducing the number of new edges ( llins) introduced during the
factorization. We have already noted that the optimal solution is NPcomplete 31].
By minimizing the degree, the algorithm minimizes the upper bound on the llin
caused by the kth pivot. Selecting p as pivot creates at most (t2 tp )=2 new edges in
p
G.
3. Quotient graphs. In contrast to the elimination graph, the quotient graph
models the factorization of A using an amount of storage that never exceeds the
storage for the original graph, G0 21]. The quotient graph is also referred to as the
k k APPROXIMATE MINIMUM DEGREE 3 generalized element model 13, 14, 15, 29]. An important component of a quotient
graph is a clique. It is a particularly economic structure since a clique is represented
by a list of its members rather than by a list of all the edges in the clique. Following
the generalized element model, we refer to nodes removed from the elimination graph
as elements (George and Liu refer to them as eliminated nodes). We use the term
variable to refer to uneliminated nodes.
The quotient graph, G k = (V k ; V k ; E k ; Ek ), implicitly represents the elimination
graph Gk , where G 0 = G0. For clarity, we drop the superscript k in the following. The
nodes in G consist of variables (the set V ), and elements (the set V ). The edges are
divided into two sets: edges between variables E V V , and between variables and
elements E V V . There are no edges between elements since they are removed
by element absorption. The sets V 0 and E 0 are empty.
We use the following set notation (A, E , and L) to describe the quotient graph
model and our approximate degree bounds. Let Ai be the set of variables adjacent to
variable i in G , and let Ei be the set of elements adjacent to variable i in G (we refer
to Ei as element list i). That is, if i is a variable in V , then
Ai fj : (i; j ) 2 E g V ; Ei fe : (i; e) 2 E g V ;
and
AdjG (i) Ai Ei V V :
The set Ai refers to a subset of the nonzero entries in row i of the original matrix A
(thus the notation A). That is, A0 fj : aij 6= 0g, and Ak Ak 1, for 1 k n.
i
i
i
Let Le denote the set of variables adjacent to element e in G . That is, if e is an
element in V , then we de ne
Le AdjG (e) = fi : (i; e) 2 E g V :
The edges E and E in the quotient graph are represented explicitly as the sets Ai
and Ei for each variable in G , and the sets Le for each element in G . We will use A, E ,
and L to denote three sets containing all Ai , Ei , and Le, respectively, for all variables
i and all elements e. George and Liu 21] show that the quotient graph takes no more
storage than the original graph (jAk j + jE k j + jLkj jA0 j for all k).
The quotient graph G and the elimination graph G are closely related. If i is a
variable in G, it is also a variable in G , and
(3.1) AdjG (i) = Ai ! e2Ei Le n fig; where the \n" is the standard set subtraction operator.
When variable p is selected as the kth pivot, element p is formed (variable p is removed from V and added to V ). The set Lp = AdjG (p) is found using Equation (3.1).
The set Lp represents a permuted nonzero pattern of the kth column of L (thus the
notation L). If i 2 Lp, where p is the kth pivot, and variable i will become the mth
pivot (for some m > k), then the entry Lmk will be nonzero. Equation (3.1) implies
that Le n fpg Lp for all elements e adjacent to variable p. This means that all variables adjacent to an element e 2 Ep are adjacent to the element p and these elements 4 P. AMESTOY, T. A. DAVIS, , AND I. S. DUFF
Fig. 3.1. Elimination graph, quotient graph, and matrix for rst three steps. G 0 G 2 1 G 2 5 2 G
5 5 9 7 3 6 9 7 10 8 4 1 10 8 3 9 7 3 10 3 8 4 6 4 5
9 7 10 6 8 6
4 (a) Elimination graph 0 1 2 2 2 5 3 2 5 2 5 5 9 7 3 6 9 7 3 6 9 7 3 6 9 7 3 6 10 8 4 1 10 8 4 1 10 8 4 1 10 8 4 1 (b) Quotient graph 1 1
2 1
2 3 3
4 2
3 4
5 1
2 3
4 5
6 6
7 7
8 9
10 6
7 8
9 5
6 7
8 4
5 8
9 10 9
10 10 (c) Factors and active submatrix e 2 Ep are no longer needed. They are absorbed into the new element p and deleted
15], and reference to them is replaced by reference to the new element p. The new
element p is added to the element lists, Ei, for all variables i adjacent to element p.
Absorbed elements, e 2 Ep , are removed from all element lists. The sets Ap and Ep ,
and Le for all e in Ep , are deleted. Finally, any entry j in Ai , where both i and j are
in Lp , is redundant and is deleted. The set Ai is thus disjoint with any set Le for
e 2 Ei . In other words, Ak is the pattern of those entries in row i of A that are not
i
modi ed by steps 1 through k of the Cholesky factorization of PAPT . The net result
is that the new graph G takes the same, or less, storage than before the kth pivot was
selected. 3.1. Quotient graph example. We illustrate the sequence of elimination graphs
and quotient graphs of a 10by10 sparse matrix in Figures 3.1 and 3.2. The example
is ordered so that a minimum degree algorithm recommends pivoting down the diagonal in the natural order (that is, the permutation matrix is the identity). In Figures
3.1 and 3.2, variables and elements are shown as thinlined and heavylined circles,
respectively. In the matrices in these gures, diagonal entries are numbered, unmodi ed original nonzero entries (entries in A) are shown as a solid squares. The solid
squares in row i form the set Ai . The variables in current unabsorbed elements (sets
Le) are indicated by solid circles in the columns of L corresponding to the unabsorbed 5 APPROXIMATE MINIMUM DEGREE
Fig. 3.2. Elimination graph, quotient graph, and matrix for steps 4 to 7. G 4 G 5 G 6 G7 5
9 7 10 8 9 7 10 6 8 9 7 9 10 6 8 10 8 (a) Elimination graph 2 4 5
9 7 8 5 3 10 9 6 5 9 4 8 9 6
4 7 10 7 10 6 8 9 7 10 6 8 (b) Quotient graph 1 1
2 1
2 3 3
4 2
3 4
5 1
2 3
4 5
6 6
7 7
8 9
10 6
7 8
9 5
6 7
8 4
5 8
9 10 9
10 10 (c) Factors and active submatrix elements. The solid circles in row i form the set Ei . Entries that do not correspond
to edges in the quotient graph are shown as an . Figure 3.1 shows the elimination
graph, quotient graph, and the matrix prior to elimination (in the left column) and
after the rst three steps (from left to right). Figure 3.2 continues the example for
the next four steps.
Consider the transformation of the graph G 2 to the graph G 3 . Variable 3 is
selected as pivot. We have L3 = A3 = f5; 6; 7g (a simple case of Equation (3.1)). The
new element 3 represents the pairwise adjacency of variables 5, 6, and 7. The explicit
edge (5,7) is now redundant, and is deleted from A5 and A7 .
Also consider the transformation of the graph G 4 to the graph G 5 . Variable 5 is
selected as pivot. The set A5 is empty and E5 = f2; 3g. Following Equation (3.1), L5 = (A5 L2 L3 ) n f5g
= (; f5; 6; 9g f5; 6; 7g) n f5g
= f6; 7; 9g;
which is the pattern of column 5 of L (excluding the diagonal). Since the new element
5 implies that variables 6, 7, and, 9 are pairwise adjacent, elements 2 and 3 do not
add any information to the graph. They are removed, having been \absorbed" into
element 5. Additionally, the edge (7, 9) is redundant, and is removed from A7 and 6 P. AMESTOY, T. A. DAVIS, , AND I. S. DUFF A9 . In G 4, we have A6 = ;
E6 = f2; 3; 4g
A7 = f9; 10g
E7 = f3; 4g :
A9 = f7; 8; 10g E9 = f2g
After these transformations, we have in G 5 ,
A6 = ;
E6 = f4; 5g
A7 = f10g
E7 = f4; 5g ;
A9 = f8; 10g E9 = f5g
and the new element in G 5 ,
L5 = f6; 7; 9g:
3.2. Indistinguishable variables and external degree. Two variables i and
j are indistinguishable in G if AdjG (i) fig = AdjG (j ) fj g. They will have the same degree until one is selected as pivot. If i is selected, then j can be selected
next without causing any additional llin. Selecting i and j together is called mass
elimination 23]. Variables i and j are replaced in G by a supervariable containing
both i and j , labeled by its principal variable (i, say) 13, 14, 15]. Variables that
are not supervariables are called simple variables. In practice, new supervariables are
constructed at step k only if both i and j are in Lp (where p is the pivot selected at
step k). In addition, rather than checking the graph G for indistinguishability, we use
the quotient graph G so that two variables i and j are found to be indistinguishable
if AdjG (i) fig = AdjG (j ) fj g. This comparison is faster than determining if
two variables are indistinguishable in G, but may miss some identi cations because,
although indistinguishability in G implies indistinguishability in G, the reverse is not
true.
We denote the set of simple variables in the supervariable with principal variable
i as i, and de ne i = fig if i is a simple variable. When p is selected as pivot at the
kth step, all variables in p are eliminated. The use of supervariables greatly reduces
the number of degree computations performed, which is the most costly part of the
algorithm. Nonprincipal variables and their incident edges are removed from the
quotient graph data structure when they are detected. The set notation A and L refers
either to a set of supervariables or to the variables represented by the supervariables,
depending on the context. In degree computations and when used in representing
elimination graphs, the sets refer to variables; otherwise they refer to supervariables.
In Figure 3.2, detected supervariables are circled by dashed lines. Nonprincipal
variables are left inside the dashed supervariables. These are, however, removed from
the quotient graph. The last quotient graph in Figure 3.2 represents the selection of
pivots 7, 8, and 9, and thus the right column of the gure depicts G7, G 9 , and the
matrix after the ninth pivot step.
The external degree di ti jij + 1 of a principal variable i is
(3.2) di = jAdjG (i) n ij = jAi n ij + ! e2Ei Le n i ; since the set Ai is disjoint from any set Le for e 2 Ei . At most (d2 di)=2 llins occur
i
if all variables in i are selected as pivots. We refer to ti as the true degree of variable i.
Selecting the pivot with minimum external degree tends to produce a better ordering
than selecting the pivot with minimum true degree 25] (also see Section 6.2). APPROXIMATE MINIMUM DEGREE 7 Algorithm 1 (Minimum degree algorithm, based on quotient graph)
V = f1:::ng
V =;
for i = 1 to n do
Ai = fj : aij = 0 and i = j g
6
6
Ei = ;
di = jAi j
i = fig
end for
k=1
while k n do
mass elimination:
select variable p 2 V that minimizes dp
Lp = Ap Se2E Le n p
for each i 2 Lp do
remove redundant entries:
Ai = (Ai n Lp ) n p
element absorption:
Ei = (Ei n Ep ) fpg
compute externalSdegree:
di = jAi n ij + e2E Le n i
end for
supervariable detection, pairs found via hash function:
for each pair i and j 2 Lp do
if i and j are indistinguishable then
remove the supervariable j:
i=i j
di = di jjj
V = V n fj g
Aj = ;
Ej = ;
end if
end for
p i convert variable p to element p: V = (V fpg) n Ep
V = V n fpg
Ap = ;
Ep = ;
k = k + jpj end while 3.3. Quotientgraphbased minimum degree algorithm. A minimum degree algorithm based on the quotient graph is shown in Algorithm 1. It includes
element absorption, mass elimination, supervariables, and external degrees. Supervariable detection is simpli ed by computing a hash function on each variable, so that
not all pairs of variables need be compared 3]. Algorithm 1 does not include two 8 P. AMESTOY, T. A. DAVIS, , AND I. S. DUFF important features of Liu's Multiple Minimum Degree algorithm (MMD): incomplete
update 17, 18] and multiple elimination 25]. With multiple elimination, an independent set of pivots with minimum degree is selected before any degrees are updated. If
a variable is adjacent to two or more pivot elements, its degree is computed only once.
A variable j is outmatched if AdjG (i) AdjG (j ). With incomplete degree update,
the degree update of the outmatched variable j is avoided until variable i is selected
as pivot. These two features further reduce the amount of work needed for the degree
computation in MMD. We will discuss their relationship to the AMD algorithm in
the next section.
The time taken to compute di using Equation (3.2) by a quotientgraphbased
minimum degree algorithm is
(jAi j + (3.3) X e2Ei jLej); which is (jAdjG (i)j) if all variables are simple.1 This degree computation is the
most costly part of the minimum degree algorithm. When supervariables are present,
the time taken is in the best case proportional to the degree of the variable in the
\compressed" elimination graph, where all nonprincipal variables and their incident
edges are removed.
4. Approximate degree. Having now discussed the data structures and the
standard minimum degree implementations, we now consider our approximation for
the minimum degree and indicate its lower complexity.
We assume that p is the kth pivot, and that we compute the bounds only for
supervariables i 2 Lp. Rather than computing the exact external degree, di , our
Approximate Minimum Degree algorithm (AMD) computes an upper bound 7, 8],
k (4.1) 8 n k;
9
> k1
>
< di + jLp n ij;
=
k
X
:
di = min >
jAi n ij + jLp n ij +
jLe n Lp j >
:
;
e2E nfpg
i The rst two terms (n k, the size of ...
View
Full Document
 Spring '08
 Staff
 Graph Theory, minimum degree

Click to edit the document details