12
October Meeting 6, 2005
Solving Recurrence Relations
Recurrence relations are perhaps the most important tool in the analysis of algorithms. We have encountered several methods that can sometimes be used to solve such relations, such as guessing the solution and proving it by induction and developing the relation into a sum for which we find a closed form expression. We now describe a new method to solve recurrence relations and use it to settle the remaining open question in the analysis of Fibonacci heaps. Annihilation of sequences. Suppose we are given an infinite sequence of numbers, A = a0 , a1 , a2 , . . . . We can multiply with a constant, shift to the left and add another sequence: kA = LA = A+B = ka0 , ka1 , ka2 , . . . , a 1 , a2 , a3 , . . . , a 0 + b 0 , a1 + b 1 , a2 + b 2 , . . . .
Multiple operators. Instead of just one, we can apply several operators to a sequence. We may multiply with two constants, k( A) = (k )A, multiply and shift, L(kA) = k(LA), and shift twice, L(LA) = L2 A. For example, (L - k)(L - ) annihilates all sequences of the form ck i + d i , where we assume k = . Indeed, L - k annihilates ck i and leaves behind ( - k)d i , which is annihilated by L - . Furthermore, (L - k)(L - ) annihilates no other sequences. More generally, we have FACT. (L - k1 )(L - k2 ) . . . (L - kn ) annihilates all sei i i quences of the form c1 k1 + c2 k2 + . . . + cn kn . What if k = ? To answer this question consider (L - k)2 ik i = (L - k) (i + 1)k i+1 - ik i+1 = (L - k) k i+1 = 0.
As an example, consider the sequence of powers of two, ai = 2i . Multiplying with 2 and shifting to the left give the same result. Therefore, LA - 2A = 0, 0, 0, . . . .
More generally, we have FACT. (L - k)n annihilates all sequences of the form p(i)k i , with p(i) a polynomial of degree n - 1. Since operators annihilate only certain types of sequences, we can determine the sequence if we know the annihilating operator. The general method works in five steps: 1. Write down the annihilator for the recurrence. 2. Factor the annihilator. 3. Determine what sequence each factor annihilates. 4. Put the sequences together. 5. Solve for the constants of the solution by using initial conditions.
We write LA - 2A = (L - 2)A and think of L - 2 as an operator that annihilates the sequence of powers of 2. In general, L - k annihilates any sequence of the form ck i . What does L - k do to other sequences A = c i , when = k? (L - k)A = c , c 2 , c 3 , . . . - ck, ck , ck 2 , . . . = ( - k) c, c , c 2 , . . . = ( - k)A.
We see that the operator L - k annihilates only one type of sequence and multiplies other similar sequences by a constant.
38
Fibonacci numbers. We put the method to a test by considering the Fibonacci numbers defined recursively as follows: F0 F1 Fj = 0, = 1, = Fj-1 + Fj-2 , for j 2.
Maximum degree. Recall that D(n) is the maximum possible degree of any one node in a Fibonacci heap of size n. We need two easy facts about the kind of trees that arise in Fibonacci heaps in order to show that D(n) is at most logarithmic in n. Let be a node of degree j, and let 1 , 2 , . . . , j be its children ordered by the time they were linked to . D EGREE L EMMA . The degree of i is at least i - 2. P ROOF. Recall that nodes are linked only during the deletemin operation. Right before the linking happens (and one nodes becomes a child of the other) the two nodes are roots and have the same degree. It follows that the degree of i was at least i - 1 at the time it was linked to . The degree of i might have been even higher because it is possible that lost some of the older children after i had been linked. After being linked, i may have lost at most one of its children, for else it would have been cut. Its degree is therefore at least i - 2, as claimed. S IZE L EMMA . The number of descendents of (including ) is at least Fj+2 . P ROOF. Let sj be the minimum number of descendents a node of degree j can have. We have s0 = 1 and s1 = 2. For larger j, we get sj from sj-1 by adding the size of a minimum tree with root degree j-2, which is sj-2 . Hence sj = sj-1 + sj-2 , which is the same recurrence relation that defines the Fibonacci numbers. The initial values are shifted two positions so we get sj = Fj+2 , as claimed. Consider a Fibonacci heap with n nodes and let be a node with maximum degree D = D(n). The Size Lemma implies n FD+2 . The Fibonacci number with index D + 2 is roughly D+2 / 5. Because D+2 < 5, we have n 1 D+2 - 1. 5
Writing a few of the initial numbers, we get the sequence 0, 1, 1, 2, 3, 5, 8, . . . . We notice that L2 - L - 1 annihilates the sequence because (L2 - L - 1) Fj = L 2 Fj - L F j - F j = =
Fj+2 - Fj+1 - Fj 0.
If we factor the operator into its roots, we get L2 - L - 1 = (L - )(L - ), where = 1- = = 1+ 5 = 1.618 . . . , 2 1- 5 = - 0.618 . . . . 2
The first root is known as golden the ratio because it represents the aspect ratio of a rectangular piece of paper from which we may remove a square to leave a smaller rectangular piece of the same ratio: : 1 = 1 : - 1. Thus we know that (L - )(L - ) annihilates Fj and this means that the j-th Fibonacci number is of the form Fj = cj + c j . We get the constant factors from the initial conditions: F0 F1 = 0 = c + c, = 1 = c + c .
Solving the two linear equations in two unknowns, we get c = 1/ 5 and c = -1/ 5. This implies that Fj = 1 5 1+ 5 2
j
1 - 5
1- 5 2
j
.
After rearranging the terms and taking the logarithm to the base , we get D log 5(n + 1) - 2.
From this viewpoint, it seems surprising that Fj turns out to be an integer for all j. Note that || > 1 and || < 1. It follows that for growing exponent j, j goes to infinity and j goes to zero. This implies that Fj is approximately j / 5, and that this approximation becomes more and more accurate as j grows.
Recall that log x = log2 x/ log2 and use the calculator to verify that log2 = 0.694 . . . > 0.5 and log 5 = 1.672 . . . < 2. Hence D log2 (n + 1) + log 5 - 2 log2 < 2 log2 (n + 1).
39
Non-homogeneous terms. We now return to the annihilation method for solving recurrence relations and consider aj = aj-1 + aj-2 + 1.
The running time is described by the solution to the recurrence T (1) = 1, T (n) = 2T (n/2) + n. We have no way to work with terms like T (n/2) yet. However, we can transform the recurrence into a more manageable form. Defining n = 2i and ti = T (2i ) we get t0 ti = 1, = 2ti-1 + 2i .
This is similar to the recurrence that defines Fibonacci numbers and describes the minimum number of nodes in an AVL tree, also known as height-balanced tree. It is defined by the requirement that the height of the two subtrees of a node differ by at most 1. The smallest tree of height j thus consists of the root, a subtree of height j - 1 and another subtree of height j - 2. We refer to the terms involving ai as the homogeneous terms of the relation and the others as the non-homogeneous terms. We know that L2 - L - 1 annihilates the homogeneous part, aj = aj-1 + aj-2 . If we apply it to the entire relation we get (L2 - L - 1) aj = = aj+2 - aj+1 - aj 1, 1, . . . .
The homogeneous part is annihilated by L - 2. Similarly, non-homogeneous part is annihilated by L - 2. Hence, (L - 2)2 annihilates the entire relation and we get ti = (ci + c)2i . Expressed in the original notation we thus have T (n) = (c log2 n + c)n = O(n log n). This result is of course no surprise and reconfirms what we learned earlier about sorting. The Master Theorem. It is sometimes more convenient to look up the solution to a recurrence relation than playing with different techniques to see whether any one can make it to yield. Such a cookbook method for recurrence relations of the form T (n) = aT (n/b) + f (n) is provided by the following theorem. He...