{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

REP-1994-160 - AN APPROXIMATE MINIMUM DEGREE ORDERING...

Info icon This preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: AN APPROXIMATE MINIMUM DEGREE ORDERING ALGORITHM TIMOTHY A. DAVIS , PATRICK AMESTOYy , AND IAIN S. DUFFz Computer and Information Sciences Dept., University of Florida, Technical Report TR-94-039, December, 1994 (revised July 1995). Abstract. An Approximate Minimum Degree ordering algorithm (AMD) for preordering a symmetric sparse matrix prior to numerical factorization is presented. We use techniques based on the quotient graph for matrix factorization that allow us to obtain computationally cheap bounds for the minimum degree. We show that these bounds are often equal to the actual degree. The resulting algorithm is typically much faster than previous minimum degree ordering algorithms, and produces results that are comparable in quality with the best orderings from other minimum degree algorithms. Keywords: approximate minimum degree ordering algorithm, quotient graph, sparse matrices, graph algorithms, ordering algorithms AMS classi cations: 65F50, 65F05. 1. Introduction. When solving large sparse symmetric linear systems of the form Ax = b, it is common to precede the numerical factorization by a symmetric reordering. This reordering is chosen so that pivoting down the diagonal in order on the resulting permuted matrix PAPT = LLT produces much less ll-in and work than computing the factors of A by pivoting down the diagonal in the original order. This reordering is computed using only information on the matrix structure without taking account of numerical values and so may not be stable for general matrices. However, if the matrix A is positive-de nite 21], a Cholesky factorization can safely be used. This technique of preceding the numerical factorization with a symbolic analysis can also be extended to unsymmetric systems although the numerical factorization phase must allow for subsequent numerical pivoting 1, 2, 16]. The goal of the preordering is to nd a permutation matrix P so that the subsequent factorization has the least ll-in. Unfortunately, this problem is NP-complete 31], so heuristics are used. The minimum degree ordering algorithm is one of the most widely used heuristics, since it produces factors with relatively low ll-in on a wide range of matrices. Because of this, the algorithm has received much attention over the past three decades. The algorithm is a symmetric analogue of Markowitz' method 26] and was rst proposed by Tinney and Walker 30] as algorithm S2. Rose 27, 28] developed a graph theoretical model of Tinney and Walker's algorithm and renamed it the minimum degree algorithm, since it performs its pivot selection by selecting from a graph a node of minimum degree. Later implementations have dramatically improved the time and memory requirements of Tinney and Walker's method, while maintaining the basic idea of selecting a node or set of nodes of minimum degree. These improvements have reduced the memory complexity so that the algorithm can operate within the storage of the original matrix, and have reduced the amount of work needed to keep track of the degrees of nodes in the graph (which is the most computationally intensive part Computer and Information Sciences Department University of Florida, Gainesville, Florida, USA. phone: (904) 392-1481, email: [email protected] .edu. Support for this project was provided by the National Science Foundation (ASC-9111263 and DMS-9223088). Portions of this work were supported by a post-doctoral grant from CERFACS. y ENSEEIHT-IRIT, Toulouse, France. email: [email protected] z Rutherford Appleton Laboratory, Chilton, Didcot, Oxon. 0X11 0QX England, and European Center for Research and Advanced Training in Scienti c Computation (CERFACS), Toulouse, France. email: [email protected] 1 2 P. AMESTOY, T. A. DAVIS, , AND I. S. DUFF of the algorithm). This work includes that of Du and Reid 10, 13, 14, 15]; George and McIntyre 23]; Eisenstat, Gursky, Schultz, and Sherman 17, 18]; George and Liu 19, 20, 21, 22]; and Liu 25]. More recently, several researchers have relaxed this heuristic by computing upper bounds on the degrees, rather than the exact degrees, and selecting a node of minimum upper bound on the degree. This work includes that of Gilbert, Moler, and Schreiber 24], and Davis and Du 7, 8]. Davis and Du use degree bounds in the unsymmetric-pattern multifrontal method (UMFPACK), an unsymmetric Markowitz-style algorithm. In this paper, we describe an approximate minimum degree ordering algorithm based on the symmetric analogue of the degree bounds used in UMFPACK. Section 2 presents the original minimum degree algorithm of Tinney and Walker in the context of the graph model of Rose. Section 3 discusses the quotient graph (or element graph) model and the use of that model to reduce the time taken by the algorithm. In this context, we present our notation for the quotient graph, and present a small example matrix and its graphs. We then use the notation to describe our approximate degree bounds in Section 4. The Approximate Minimum Degree (AMD) algorithm and its time complexity is presented in Section 5. In Section 6, we rst analyse the performance and accuracy of our approximate degree bounds on a set of test matrices from a wide range of disciplines. The AMD algorithm is then compared with other established codes that compute minimum degree orderings. 2. Elimination graphs. The nonzero pattern of a symmetric n-by-n matrix, A, can be represented by a graph G0 = (V 0; E0), with nodes V 0 = f1; :::; ng and edges E 0. An edge (i; j ) is in E 0 if and only if aij 6= 0. Since A is symmetric, G0 is undirected. The elimination graph, Gk = (V k ; E k ), describes the nonzero pattern of the submatrix still to be factorized after the rst k pivots have been chosen. It is undirected, since the matrix remains symmetric as it is factorized. At step k, the graph Gk depends on Gk 1 and the selection of the kth pivot. To nd Gk , the kth pivot node p is selected from V k 1. Edges are added to E k 1 to make the nodes adjacent to p in Gk 1 a clique (a fully connected subgraph). This addition of edges ( ll-in) means that we cannot know the storage requirements in advance. The edges added correspond to ll-in caused by the kth step of factorization. A ll-in is a nonzero entry Lij , where (PAPT )ij is zero. The pivot node p and its incident edges are then removed from the graph Gk 1 to yield the graph Gk . Let AdjG (i) denote the set of nodes adjacent to i in the graph Gk . Throughout this paper, we will use the superscript k to denote a graph, set, or other structure obtained after the rst k pivots have been chosen. For simplicity, we will drop the superscript when the context is clear. The minimum degree algorithm selects node p as the kth pivot such that the degree of p, tp jAdjG 1 (p)j, is minimized (where j:::j denotes the size of a set or the number of nonzeros in a matrix). The minimum degree algorithm is a non-optimal greedy heuristic for reducing the number of new edges ( ll-ins) introduced during the factorization. We have already noted that the optimal solution is NP-complete 31]. By minimizing the degree, the algorithm minimizes the upper bound on the ll-in caused by the kth pivot. Selecting p as pivot creates at most (t2 tp )=2 new edges in p G. 3. Quotient graphs. In contrast to the elimination graph, the quotient graph models the factorization of A using an amount of storage that never exceeds the storage for the original graph, G0 21]. The quotient graph is also referred to as the k k APPROXIMATE MINIMUM DEGREE 3 generalized element model 13, 14, 15, 29]. An important component of a quotient graph is a clique. It is a particularly economic structure since a clique is represented by a list of its members rather than by a list of all the edges in the clique. Following the generalized element model, we refer to nodes removed from the elimination graph as elements (George and Liu refer to them as eliminated nodes). We use the term variable to refer to uneliminated nodes. The quotient graph, G k = (V k ; V k ; E k ; Ek ), implicitly represents the elimination graph Gk , where G 0 = G0. For clarity, we drop the superscript k in the following. The nodes in G consist of variables (the set V ), and elements (the set V ). The edges are divided into two sets: edges between variables E V V , and between variables and elements E V V . There are no edges between elements since they are removed by element absorption. The sets V 0 and E 0 are empty. We use the following set notation (A, E , and L) to describe the quotient graph model and our approximate degree bounds. Let Ai be the set of variables adjacent to variable i in G , and let Ei be the set of elements adjacent to variable i in G (we refer to Ei as element list i). That is, if i is a variable in V , then Ai fj : (i; j ) 2 E g V ; Ei fe : (i; e) 2 E g V ; and AdjG (i) Ai Ei V V : The set Ai refers to a subset of the nonzero entries in row i of the original matrix A (thus the notation A). That is, A0 fj : aij 6= 0g, and Ak Ak 1, for 1 k n. i i i Let Le denote the set of variables adjacent to element e in G . That is, if e is an element in V , then we de ne Le AdjG (e) = fi : (i; e) 2 E g V : The edges E and E in the quotient graph are represented explicitly as the sets Ai and Ei for each variable in G , and the sets Le for each element in G . We will use A, E , and L to denote three sets containing all Ai , Ei , and Le, respectively, for all variables i and all elements e. George and Liu 21] show that the quotient graph takes no more storage than the original graph (jAk j + jE k j + jLkj jA0 j for all k). The quotient graph G and the elimination graph G are closely related. If i is a variable in G, it is also a variable in G , and (3.1) AdjG (i) = Ai ! e2Ei Le n fig; where the \n" is the standard set subtraction operator. When variable p is selected as the kth pivot, element p is formed (variable p is removed from V and added to V ). The set Lp = AdjG (p) is found using Equation (3.1). The set Lp represents a permuted nonzero pattern of the kth column of L (thus the notation L). If i 2 Lp, where p is the kth pivot, and variable i will become the mth pivot (for some m > k), then the entry Lmk will be nonzero. Equation (3.1) implies that Le n fpg Lp for all elements e adjacent to variable p. This means that all variables adjacent to an element e 2 Ep are adjacent to the element p and these elements 4 P. AMESTOY, T. A. DAVIS, , AND I. S. DUFF Fig. 3.1. Elimination graph, quotient graph, and matrix for rst three steps. G 0 G 2 1 G 2 5 2 G 5 5 9 7 3 6 9 7 10 8 4 1 10 8 3 9 7 3 10 3 8 4 6 4 5 9 7 10 6 8 6 4 (a) Elimination graph 0 1 2 2 2 5 3 2 5 2 5 5 9 7 3 6 9 7 3 6 9 7 3 6 9 7 3 6 10 8 4 1 10 8 4 1 10 8 4 1 10 8 4 1 (b) Quotient graph 1 1 2 1 2 3 3 4 2 3 4 5 1 2 3 4 5 6 6 7 7 8 9 10 6 7 8 9 5 6 7 8 4 5 8 9 10 9 10 10 (c) Factors and active submatrix e 2 Ep are no longer needed. They are absorbed into the new element p and deleted 15], and reference to them is replaced by reference to the new element p. The new element p is added to the element lists, Ei, for all variables i adjacent to element p. Absorbed elements, e 2 Ep , are removed from all element lists. The sets Ap and Ep , and Le for all e in Ep , are deleted. Finally, any entry j in Ai , where both i and j are in Lp , is redundant and is deleted. The set Ai is thus disjoint with any set Le for e 2 Ei . In other words, Ak is the pattern of those entries in row i of A that are not i modi ed by steps 1 through k of the Cholesky factorization of PAPT . The net result is that the new graph G takes the same, or less, storage than before the kth pivot was selected. 3.1. Quotient graph example. We illustrate the sequence of elimination graphs and quotient graphs of a 10-by-10 sparse matrix in Figures 3.1 and 3.2. The example is ordered so that a minimum degree algorithm recommends pivoting down the diagonal in the natural order (that is, the permutation matrix is the identity). In Figures 3.1 and 3.2, variables and elements are shown as thin-lined and heavy-lined circles, respectively. In the matrices in these gures, diagonal entries are numbered, unmodi ed original nonzero entries (entries in A) are shown as a solid squares. The solid squares in row i form the set Ai . The variables in current unabsorbed elements (sets Le) are indicated by solid circles in the columns of L corresponding to the unabsorbed 5 APPROXIMATE MINIMUM DEGREE Fig. 3.2. Elimination graph, quotient graph, and matrix for steps 4 to 7. G 4 G 5 G 6 G7 5 9 7 10 8 9 7 10 6 8 9 7 9 10 6 8 10 8 (a) Elimination graph 2 4 5 9 7 8 5 3 10 9 6 5 9 4 8 9 6 4 7 10 7 10 6 8 9 7 10 6 8 (b) Quotient graph 1 1 2 1 2 3 3 4 2 3 4 5 1 2 3 4 5 6 6 7 7 8 9 10 6 7 8 9 5 6 7 8 4 5 8 9 10 9 10 10 (c) Factors and active submatrix elements. The solid circles in row i form the set Ei . Entries that do not correspond to edges in the quotient graph are shown as an . Figure 3.1 shows the elimination graph, quotient graph, and the matrix prior to elimination (in the left column) and after the rst three steps (from left to right). Figure 3.2 continues the example for the next four steps. Consider the transformation of the graph G 2 to the graph G 3 . Variable 3 is selected as pivot. We have L3 = A3 = f5; 6; 7g (a simple case of Equation (3.1)). The new element 3 represents the pairwise adjacency of variables 5, 6, and 7. The explicit edge (5,7) is now redundant, and is deleted from A5 and A7 . Also consider the transformation of the graph G 4 to the graph G 5 . Variable 5 is selected as pivot. The set A5 is empty and E5 = f2; 3g. Following Equation (3.1), L5 = (A5 L2 L3 ) n f5g = (; f5; 6; 9g f5; 6; 7g) n f5g = f6; 7; 9g; which is the pattern of column 5 of L (excluding the diagonal). Since the new element 5 implies that variables 6, 7, and, 9 are pairwise adjacent, elements 2 and 3 do not add any information to the graph. They are removed, having been \absorbed" into element 5. Additionally, the edge (7, 9) is redundant, and is removed from A7 and 6 P. AMESTOY, T. A. DAVIS, , AND I. S. DUFF A9 . In G 4, we have A6 = ; E6 = f2; 3; 4g A7 = f9; 10g E7 = f3; 4g : A9 = f7; 8; 10g E9 = f2g After these transformations, we have in G 5 , A6 = ; E6 = f4; 5g A7 = f10g E7 = f4; 5g ; A9 = f8; 10g E9 = f5g and the new element in G 5 , L5 = f6; 7; 9g: 3.2. Indistinguishable variables and external degree. Two variables i and j are indistinguishable in G if AdjG (i) fig = AdjG (j ) fj g. They will have the same degree until one is selected as pivot. If i is selected, then j can be selected next without causing any additional ll-in. Selecting i and j together is called mass elimination 23]. Variables i and j are replaced in G by a supervariable containing both i and j , labeled by its principal variable (i, say) 13, 14, 15]. Variables that are not supervariables are called simple variables. In practice, new supervariables are constructed at step k only if both i and j are in Lp (where p is the pivot selected at step k). In addition, rather than checking the graph G for indistinguishability, we use the quotient graph G so that two variables i and j are found to be indistinguishable if AdjG (i) fig = AdjG (j ) fj g. This comparison is faster than determining if two variables are indistinguishable in G, but may miss some identi cations because, although indistinguishability in G implies indistinguishability in G, the reverse is not true. We denote the set of simple variables in the supervariable with principal variable i as i, and de ne i = fig if i is a simple variable. When p is selected as pivot at the kth step, all variables in p are eliminated. The use of supervariables greatly reduces the number of degree computations performed, which is the most costly part of the algorithm. Non-principal variables and their incident edges are removed from the quotient graph data structure when they are detected. The set notation A and L refers either to a set of supervariables or to the variables represented by the supervariables, depending on the context. In degree computations and when used in representing elimination graphs, the sets refer to variables; otherwise they refer to supervariables. In Figure 3.2, detected supervariables are circled by dashed lines. Non-principal variables are left inside the dashed supervariables. These are, however, removed from the quotient graph. The last quotient graph in Figure 3.2 represents the selection of pivots 7, 8, and 9, and thus the right column of the gure depicts G7, G 9 , and the matrix after the ninth pivot step. The external degree di ti jij + 1 of a principal variable i is (3.2) di = jAdjG (i) n ij = jAi n ij + ! e2Ei Le n i ; since the set Ai is disjoint from any set Le for e 2 Ei . At most (d2 di)=2 ll-ins occur i if all variables in i are selected as pivots. We refer to ti as the true degree of variable i. Selecting the pivot with minimum external degree tends to produce a better ordering than selecting the pivot with minimum true degree 25] (also see Section 6.2). APPROXIMATE MINIMUM DEGREE 7 Algorithm 1 (Minimum degree algorithm, based on quotient graph) V = f1:::ng V =; for i = 1 to n do Ai = fj : aij = 0 and i = j g 6 6 Ei = ; di = jAi j i = fig end for k=1 while k n do mass elimination: select variable p 2 V that minimizes dp Lp = Ap Se2E Le n p for each i 2 Lp do remove redundant entries: Ai = (Ai n Lp ) n p element absorption: Ei = (Ei n Ep ) fpg compute externalSdegree: di = jAi n ij + e2E Le n i end for supervariable detection, pairs found via hash function: for each pair i and j 2 Lp do if i and j are indistinguishable then remove the supervariable j: i=i j di = di jjj V = V n fj g Aj = ; Ej = ; end if end for p i convert variable p to element p: V = (V fpg) n Ep V = V n fpg Ap = ; Ep = ; k = k + jpj end while 3.3. Quotient-graph-based minimum degree algorithm. A minimum degree algorithm based on the quotient graph is shown in Algorithm 1. It includes element absorption, mass elimination, supervariables, and external degrees. Supervariable detection is simpli ed by computing a hash function on each variable, so that not all pairs of variables need be compared 3]. Algorithm 1 does not include two 8 P. AMESTOY, T. A. DAVIS, , AND I. S. DUFF important features of Liu's Multiple Minimum Degree algorithm (MMD): incomplete update 17, 18] and multiple elimination 25]. With multiple elimination, an independent set of pivots with minimum degree is selected before any degrees are updated. If a variable is adjacent to two or more pivot elements, its degree is computed only once. A variable j is outmatched if AdjG (i) AdjG (j ). With incomplete degree update, the degree update of the outmatched variable j is avoided until variable i is selected as pivot. These two features further reduce the amount of work needed for the degree computation in MMD. We will discuss their relationship to the AMD algorithm in the next section. The time taken to compute di using Equation (3.2) by a quotient-graph-based minimum degree algorithm is (jAi j + (3.3) X e2Ei jLej); which is (jAdjG (i)j) if all variables are simple.1 This degree computation is the most costly part of the minimum degree algorithm. When supervariables are present, the time taken is in the best case proportional to the degree of the variable in the \compressed" elimination graph, where all non-principal variables and their incident edges are removed. 4. Approximate degree. Having now discussed the data structures and the standard minimum degree implementations, we now consider our approximation for the minimum degree and indicate its lower complexity. We assume that p is the kth pivot, and that we compute the bounds only for supervariables i 2 Lp. Rather than computing the exact external degree, di , our Approximate Minimum Degree algorithm (AMD) computes an upper bound 7, 8], k (4.1) 8 n k; 9 > k1 > < di + jLp n ij; = k X : di = min > jAi n ij + jLp n ij + jLe n Lp j > : ; e2E nfpg i The rst two terms (n k, the size of ...
View Full Document

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern