recurrences

# recurrences - Introduction to the Theory of Computation...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Introduction to the Theory of Computation AZADEH FARZAN SPRING 2010 Tuesday, January 26, 2010 FUNCTIONS DEFINED BY INDUCTION Tuesday, January 26, 2010 RECURSIVELY DEFINED FUNCTIONS 76 CHAPTER 3. FUNCTIONS DEFINED BY IN Examples. Let f : N → N be ￿ • for which we need to ﬁrst0compute f (0), which is given = 0 if n explicitly. f (n) = f (n − 1) + 2n − 1 if n > 0 Now working forward, we get: • f (0) = 0 • f (1) = f (0) + 2 · 1 − 1 = 0 + 2 − 1 = 1 • f (2) = f (1) + 2 · 2 − 1 = 1 + 4 − 1 = 4 • f (3) = f (2) + 2 · 3 − 1 = 4 + 6 − 1 = 9 • f (4) = f (3) + 2 · 4 − 1 = 9 + 8 − 1 = 16 GivenCan you guess what f (n) is? though the function f simply computes the s these calculations, it looks as argument! We prove that this is, in fact, the case. Tuesday, January 26, 2010 Proposition 3.1 For any n ∈ N, f (n) = n2 . RIB U LET’S PROVE IT! Claim. Let f : N → N be ￿ 0 f (n) = f (n − 1) + 2n − 1 if n = 0 if n > 0, then for all n ∈ N, f (n) = n2 . In general, for proving properties of a recursively deﬁned Proof: We induction is the natural tool to use. object, use induction. Basis: f (0) = 0. Induction step: suppose f (i) = i: f (i + 1) = = i2 + 2(i + 1) − 1 = i2 + 2i + 1 = (i + 1)2 . f (i) + 2(i + 1) − 1 Tuesday, January 26, 2010 BAD RECURSIVE DEFINITIONS Not every recursive equation deﬁnes a function! h(n) = h(n − 1) + 5, for all n > 0 ￿ ￿ 0, h￿ (n − 2) + 5, 0, h￿￿ (￿n/2￿) + 5, if n = 0 if n > 0 h (n) = ￿ h (n) = ￿￿ if n = 0 if n > 0 Tuesday, January 26, 2010 ANOTHER EXAMPLE How about: f ( n) = ￿ ￿ 0 f ￿ (n + 1) − 2n − 1 if n = 0 if n ≥ 0 But, is the same as f ￿ (n) = f ￿ (n + 1) − 2n − 1, if n ≥ 0 f (n) = f (n − 1) + 2n − 1, if n > 0. ￿ ￿ Tuesday, January 26, 2010 A PRINCIPLE Principle of function deﬁnition by recursion. Let b ∈ Z, and g : N × Z → Z be a function. Then there is a unique function f : N → Z that satisﬁes the following equation: ￿ b if n = 0 f ( n) = g (n, f (n − 1)) if n > 0. Tuesday, January 26, 2010 FIBONACCI SEQUENCE Now, let’s look at an example based on complete induction. F (2) = F (3) = F (4) = F (5) = F (6) = Example. Let F : N → N be given by: 0 1 F (n) = F (n − 1) + F (n − 2) F (1) + F (0) = 1 + 0 = 1 F (2) + F (1) = 1 + 1 = 2 F (3) + F (2) = 2 + 1 = 3 F (4) + F (3) = 3 + 2 = 5 F (5) + F (4) = 5 + 3 = 8 if n = 0 if n = 1 if n > 1. The Fibonacci Sequence is the sequence F (0), F (1), . . ., i.e. 0, 0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, . . . Tuesday, January 26, 2010 FIBONACCI CLOSED FORM Theorem. For any n ∈ N: ˆ φn − φn F ( n) = √ , 5 where 1+ 5 φ= 2 √ and ˆ = 1 − 5. φ 2 √ Aside: φ is the famous golden ration, the only solution to a+b a = = φ. a b Tuesday, January 26, 2010 PRINCIPLE GENERALIZED Principle of function deﬁnition by complete recursion: Let k, ￿ be positive integers, b0 , b1 , . . . , bk−1 be arbitrary integers. Also, let h1 , h2 , . . . , h￿ : N → N be functions such that hi (n) < n for each i, 1 ≤ i ≤ ￿ and each n ≥ k , and g : N × Z￿ → Z be a function. Then there is a unique function f : N → Z that satisﬁes the equation: ￿ bn if 0 ≤ n < k f (n) = g (n, f (h1 (n)), f (h2 (n)), . . . , f (h￿ (n))) if n ≥ k. Tuesday, January 26, 2010 TIME COMPLEXITY OF MERGE SORT Time to sort an array with 1 element T (n) = ￿ c T (￿n/2￿) + T (￿n/2￿) + dn if n = 1 if n > 1 Time to sort the ﬁrst half Time to sort the second half Time to merge the results This is a proper recursive function deﬁnition because: Lemma. For any integer n > 1, 1 ≤ ￿n/2￿ ≤ ￿n/2￿ ≤ n. Tuesday, January 26, 2010 SOLVING THE RECURRENCE If we assume n is a power of 2, then we can write: ￿ c if n = 1 T ( n) = 2T (n/2) + dn if n > 1. To guess the solution, we use a technique called repeated substitution: T (n) = = = = = = = 2T (n/2) + dn ￿ ￿ 2 2 2T (n/2 ) + dn/2 + dn 23 T (n/23 ) + 3dn ... 2i T (n/2i ) + idn 22 T (n/22 ) + 2dn ￿ ￿ 2 3 2 2 2T (n/2 ) + dn/2 + 2dn Tuesday, January 26, 2010 LET’S PROVE IT Lemma. If n is a power of 2, for any natural number i ≤ log2 n, the function T satisﬁes T (n) = 2 T (n/2 ) + idn. i i Tuesday, January 26, 2010 CLOSED FORM FORUMULA Theorem. If n is a power of 2, then T (n) = cn + dn log2 n. Proof. Let i = log2 n in the previous Lemma: T (n) = 2log2 n T (n/2log2 n ) + (log2 n)dn = nT (1) + dn log2 n = cn + dn log2 n. Lemma. The function T is non-decreasing, i.e. for all m, n ∈ N, if m < n, then T (m) ≤ T (n). Tuesday, January 26, 2010 WHAT ABOUT NON-POWERS OF 2 Theorem. There is a constant κ > 0 (whose value depends on c and d), such that for all n ≥ 2: Proof. If n is not a power of 2, let n be the smallest power of 2, ˆ ˆ ˆ larger than n, i.e. n = 2￿log2 n￿ . Thus n < n < n. ˆ 2 T (n) ≤ T (ˆ ) n = cn + dn log2 n ˆ ˆ ˆ ≤ c(2n) + d(2n) log2 (2n) = 2cn + 2dn + 2dn log2 n ≤ 2cn log2 n + 2dn log2 n + 2dn log2 n = κn log2 n where k = 2c + 4d. Tuesday, January 26, 2010 T (n) ≤ κn log2 n. DIVIDE AND CONQUER RECURRENCES Tuesday, January 26, 2010 SOME TERMINOLOGY an instance of a problem a solution of an instance ≡ ≡ a legitimate input of it a valid output for it Each instance has an (integer) size. The deﬁnition of size depends on the problem. Example. For MergeSort: an instance a solution the size of an instance is is is an array a sorted array its length Tuesday, January 26, 2010 DIVIDE AND CONQUER STRATEGY To solve large instance of the problem of size n, we (a) Divide the instance into a instance of size roughly n/b. (b) Recursively solve each of the b smaller instance. (c) Combine the solutions to smaller instances into one for the original large one. If the instance is small enough then we solve it directly using an ad hoc algorithm (this is usually trivial). Example. In MergeSort, a = 2 and b = 2, i.e. we divide an input (array) of size n into 2 arrays of size n/2. Tuesday, January 26, 2010 DIVIDE AND CONQUER RUNNING Assuming that dividing and combining (steps (a) and (c)) take time dn￿ on an instance of size n, we can write: T (n) = ￿ where a1 , a2 ∈ N such that a = a1 + a2 ≥ 1, b ∈ N and b > 1, and c, d, ￿ ∈ R+ . c￿ ￿ ￿n￿ a1 T ￿ n ￿ + a2 T ￿ b ￿ + dn￿ b if 1 ≤ n < b if n ≥ b Note. To make life easier, we assume all instance of size < b take time c. Example. In MergeSort: a1 = a2 = 1, b = 2, and ￿ = 1. Tuesday, January 26, 2010 SOLVING THE RECURRENCE It is not possible to obtain simple closed-form formulas. We will instead ﬁnd tight upper and lower bounds. Let’s consider only values of n that are powers of b. T (n) = ￿ The answer depends on how a compares to b￿ . c ￿￿ aT n + dn￿ b if 1 ≤ n < b if n ≥ b Tuesday, January 26, 2010 UPPER BOUND Theorem. There is a constant κ ≥ 0 (that depends on a, b, c, d, ￿) so that, for all integers n ≥ b that are powers of b: κn ￿ if a < b￿ κn￿ logb n if a = b￿ T ( n) ≤ κnlogb a if a > b￿ . Tuesday, January 26, 2010 + dn￿ T (n) = aT ￿ b￿ ￿ ￿ n ￿￿ ￿ n = a aT 2 + d + dn￿ b b ￿n￿ ￿a￿ = a2 T 2 + ￿ dn￿ + dn￿ ￿ b￿ ￿ b ￿ ￿ ￿ ￿ ￿ n n￿ a 2 = a aT 3 + d 2 + ￿ dn￿ + dn￿ b b b ￿ n ￿ ￿ a ￿2 ￿a￿ 3 ￿ ￿ ￿ = a T 3 + ￿ dn + ￿ dn + dn b b b = ... i−1 ￿n￿ ￿ ￿ a ￿j = ai T i + dn￿ b b￿ j =0 T ( n) = a Tuesday, January 26, 2010 ￿n￿ PROOF logb n T (1)+dn ￿ logb n−1 ￿ ￿ j =0 b￿ ￿j a = cn logb a +dn ￿ logb n−1 ￿ ￿ j =0 b￿ ￿j a PROOF logb n−1 ￿ j =0 α= j ￿ logb n 1−αlogb n 1−α if α ￿= 1 if α = 1 T ( n) = ￿ cn + dn cn￿ + dn￿ logb n logb a ￿ ￿ 1−αlogb 1−α ￿ n if a ￿= b￿ (i.e., α ￿= 1) if a = b￿ (i.e., α = 1) Tuesday, January 26, 2010 UPPER BOUND Theorem. There is a constant κ ≥ 0 (that depends on a, b, c, d, ￿) so that, for all integers n ≥ b that are powers of b: κn ￿ if a < b￿ κn￿ logb n if a = b￿ T ( n) ≤ κnlogb a if a > b￿ . The theorem extends (as in the case of MergeSort to arbitrary values of n (only with a diﬀerent constant, say λ, instead of κ). Tuesday, January 26, 2010 LOWER BOUND Theorem. There is a constant λ￿ > 0 (that depends on a, b, c, d, ￿) so that, for all integers n ≥ b: ￿￿ if a < b￿ and d > 0 λn λ￿ n￿ logb n if a = b￿ and d > 0 T (n) ≥ ￿ logb a λn if a > b￿ and d = 0. Note. λ￿ is strictly positive or else the theorem is trivial for λ￿ = 0. Tuesday, January 26, 2010 ANOTHER RECURRENCE Consider the following recurrence, where the dividing and combining time (steps (a) and (c)) is not of the form dn￿ : ￿ 1￿ if n = 1 ￿ T (n) = 2T ￿ n ￿ + ￿log2 n￿ if n > 1 2 We can upper-bound this recurrence by the following one: ￿ 1￿ if n = 1 ￿√ T ( n) = 2T ￿ n ￿ + n if n > 1 2 Note that √ n = n1/2 , so we can use our Theorem. Tuesday, January 26, 2010 ...
View Full Document

{[ snackBarMessage ]}

Ask a homework question - tutors are online