This preview shows page 1. Sign up to view the full content.
Unformatted text preview: Introduction to the Theory of Computation
AZADEH FARZAN SPRING 2010 Tuesday, January 26, 2010 FUNCTIONS DEFINED BY INDUCTION Tuesday, January 26, 2010 RECURSIVELY DEFINED FUNCTIONS
76 CHAPTER 3. FUNCTIONS DEFINED BY IN Examples. Let f : N → N be • for which we need to ﬁrst0compute f (0), which is given = 0 if n explicitly. f (n) = f (n − 1) + 2n − 1 if n > 0 Now working forward, we get: • f (0) = 0 • f (1) = f (0) + 2 · 1 − 1 = 0 + 2 − 1 = 1 • f (2) = f (1) + 2 · 2 − 1 = 1 + 4 − 1 = 4 • f (3) = f (2) + 2 · 3 − 1 = 4 + 6 − 1 = 9 • f (4) = f (3) + 2 · 4 − 1 = 9 + 8 − 1 = 16 GivenCan you guess what f (n) is? though the function f simply computes the s these calculations, it looks as argument! We prove that this is, in fact, the case.
Tuesday, January 26, 2010 Proposition 3.1 For any n ∈ N, f (n) = n2 . RIB U LET’S PROVE IT!
Claim. Let f : N → N be 0 f (n) = f (n − 1) + 2n − 1 if n = 0 if n > 0, then for all n ∈ N, f (n) = n2 . In general, for proving properties of a recursively deﬁned Proof: We induction is the natural tool to use. object, use induction. Basis: f (0) = 0. Induction step: suppose f (i) = i: f (i + 1) = = i2 + 2(i + 1) − 1 = i2 + 2i + 1 = (i + 1)2 . f (i) + 2(i + 1) − 1 Tuesday, January 26, 2010 BAD RECURSIVE DEFINITIONS
Not every recursive equation deﬁnes a function! h(n) = h(n − 1) + 5, for all n > 0
0, h (n − 2) + 5, 0, h (n/2) + 5, if n = 0 if n > 0 h (n) =
h (n) =
if n = 0 if n > 0 Tuesday, January 26, 2010 ANOTHER EXAMPLE
How about: f ( n) =
0 f (n + 1) − 2n − 1 if n = 0 if n ≥ 0 But, is the same as f (n) = f (n + 1) − 2n − 1, if n ≥ 0 f (n) = f (n − 1) + 2n − 1, if n > 0.
Tuesday, January 26, 2010 A PRINCIPLE
Principle of function deﬁnition by recursion. Let b ∈ Z, and g : N × Z → Z be a function. Then there is a unique function f : N → Z that satisﬁes the following equation: b if n = 0 f ( n) = g (n, f (n − 1)) if n > 0. Tuesday, January 26, 2010 FIBONACCI SEQUENCE
Now, let’s look at an example based on complete induction. F (2) = F (3) = F (4) = F (5) = F (6) = Example. Let F : N → N be given by: 0 1 F (n) = F (n − 1) + F (n − 2)
F (1) + F (0) = 1 + 0 = 1 F (2) + F (1) = 1 + 1 = 2 F (3) + F (2) = 2 + 1 = 3 F (4) + F (3) = 3 + 2 = 5 F (5) + F (4) = 5 + 3 = 8 if n = 0 if n = 1 if n > 1. The Fibonacci Sequence is the sequence F (0), F (1), . . ., i.e. 0, 0, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, . . .
Tuesday, January 26, 2010 FIBONACCI CLOSED FORM
Theorem. For any n ∈ N: ˆ φn − φn F ( n) = √ , 5 where 1+ 5 φ= 2 √ and ˆ = 1 − 5. φ 2 √ Aside: φ is the famous golden ration, the only solution to a+b a = = φ. a b Tuesday, January 26, 2010 PRINCIPLE GENERALIZED
Principle of function deﬁnition by complete recursion: Let k, be positive integers, b0 , b1 , . . . , bk−1 be arbitrary integers. Also, let h1 , h2 , . . . , h : N → N be functions such that hi (n) < n for each i, 1 ≤ i ≤ and each n ≥ k , and g : N × Z → Z be a function. Then there is a unique function f : N → Z that satisﬁes the equation: bn if 0 ≤ n < k f (n) = g (n, f (h1 (n)), f (h2 (n)), . . . , f (h (n))) if n ≥ k. Tuesday, January 26, 2010 TIME COMPLEXITY OF MERGE SORT
Time to sort an array with 1 element
T (n) = c T (n/2) + T (n/2) + dn if n = 1 if n > 1 Time to sort the ﬁrst half Time to sort the second half Time to merge the results
This is a proper recursive function deﬁnition because: Lemma. For any integer n > 1, 1 ≤ n/2 ≤ n/2 ≤ n. Tuesday, January 26, 2010 SOLVING THE RECURRENCE
If we assume n is a power of 2, then we can write: c if n = 1 T ( n) = 2T (n/2) + dn if n > 1. To guess the solution, we use a technique called repeated substitution:
T (n) = = = = = = = 2T (n/2) + dn 2 2 2T (n/2 ) + dn/2 + dn 23 T (n/23 ) + 3dn ... 2i T (n/2i ) + idn 22 T (n/22 ) + 2dn 2 3 2 2 2T (n/2 ) + dn/2 + 2dn Tuesday, January 26, 2010 LET’S PROVE IT
Lemma. If n is a power of 2, for any natural number i ≤ log2 n, the function T satisﬁes T (n) = 2 T (n/2 ) + idn.
i i Tuesday, January 26, 2010 CLOSED FORM FORUMULA
Theorem. If n is a power of 2, then T (n) = cn + dn log2 n. Proof. Let i = log2 n in the previous Lemma: T (n) = 2log2 n T (n/2log2 n ) + (log2 n)dn = nT (1) + dn log2 n = cn + dn log2 n.
Lemma. The function T is nondecreasing, i.e. for all m, n ∈ N, if m < n, then T (m) ≤ T (n). Tuesday, January 26, 2010 WHAT ABOUT NONPOWERS OF 2
Theorem. There is a constant κ > 0 (whose value depends on c and d), such that for all n ≥ 2: Proof. If n is not a power of 2, let n be the smallest power of 2, ˆ ˆ ˆ larger than n, i.e. n = 2log2 n . Thus n < n < n. ˆ 2 T (n) ≤ T (ˆ ) n = cn + dn log2 n ˆ ˆ ˆ ≤ c(2n) + d(2n) log2 (2n) = 2cn + 2dn + 2dn log2 n ≤ 2cn log2 n + 2dn log2 n + 2dn log2 n = κn log2 n where k = 2c + 4d.
Tuesday, January 26, 2010 T (n) ≤ κn log2 n. DIVIDE AND CONQUER RECURRENCES Tuesday, January 26, 2010 SOME TERMINOLOGY
an instance of a problem a solution of an instance ≡ ≡ a legitimate input of it a valid output for it Each instance has an (integer) size. The deﬁnition of size depends on the problem. Example. For MergeSort:
an instance a solution the size of an instance is is is an array a sorted array its length Tuesday, January 26, 2010 DIVIDE AND CONQUER STRATEGY
To solve large instance of the problem of size n, we (a) Divide the instance into a instance of size roughly n/b. (b) Recursively solve each of the b smaller instance. (c) Combine the solutions to smaller instances into one for the original large one.
If the instance is small enough then we solve it directly using an ad hoc algorithm (this is usually trivial). Example. In MergeSort, a = 2 and b = 2, i.e. we divide an input (array) of size n into 2 arrays of size n/2. Tuesday, January 26, 2010 DIVIDE AND CONQUER RUNNING
Assuming that dividing and combining (steps (a) and (c)) take time dn on an instance of size n, we can write: T (n) = where a1 , a2 ∈ N such that a = a1 + a2 ≥ 1, b ∈ N and b > 1, and c, d, ∈ R+ . c n a1 T n + a2 T b + dn b if 1 ≤ n < b if n ≥ b Note. To make life easier, we assume all instance of size < b take time c. Example. In MergeSort: a1 = a2 = 1, b = 2, and = 1. Tuesday, January 26, 2010 SOLVING THE RECURRENCE
It is not possible to obtain simple closedform formulas. We will instead ﬁnd tight upper and lower bounds. Let’s consider only values of n that are powers of b. T (n) = The answer depends on how a compares to b . c aT n + dn b if 1 ≤ n < b if n ≥ b Tuesday, January 26, 2010 UPPER BOUND
Theorem. There is a constant κ ≥ 0 (that depends on a, b, c, d, ) so that, for all integers n ≥ b that are powers of b: κn if a < b κn logb n if a = b T ( n) ≤ κnlogb a if a > b . Tuesday, January 26, 2010 + dn T (n) = aT b n n = a aT 2 + d + dn b b n a = a2 T 2 + dn + dn b b n n a 2 = a aT 3 + d 2 + dn + dn b b b n a 2 a 3 = a T 3 + dn + dn + dn b b b = ... i−1 n a j = ai T i + dn b b j =0
T ( n) = a
Tuesday, January 26, 2010 n PROOF logb n T (1)+dn logb n−1
j =0 b j a = cn logb a +dn logb n−1
j =0 b j a PROOF
logb n−1
j =0 α=
j logb n 1−αlogb n 1−α if α = 1 if α = 1 T ( n) = cn + dn cn + dn logb n
logb a 1−αlogb 1−α n if a = b (i.e., α = 1) if a = b (i.e., α = 1) Tuesday, January 26, 2010 UPPER BOUND
Theorem. There is a constant κ ≥ 0 (that depends on a, b, c, d, ) so that, for all integers n ≥ b that are powers of b: κn if a < b κn logb n if a = b T ( n) ≤ κnlogb a if a > b . The theorem extends (as in the case of MergeSort to arbitrary values of n (only with a diﬀerent constant, say λ, instead of κ). Tuesday, January 26, 2010 LOWER BOUND
Theorem. There is a constant λ > 0 (that depends on a, b, c, d, ) so that, for all integers n ≥ b: if a < b and d > 0 λn λ n logb n if a = b and d > 0 T (n) ≥ logb a λn if a > b and d = 0. Note. λ is strictly positive or else the theorem is trivial for λ = 0. Tuesday, January 26, 2010 ANOTHER RECURRENCE
Consider the following recurrence, where the dividing and combining time (steps (a) and (c)) is not of the form dn : 1 if n = 1 T (n) = 2T n + log2 n if n > 1 2
We can upperbound this recurrence by the following one: 1 if n = 1 √ T ( n) = 2T n + n if n > 1 2
Note that √ n = n1/2 , so we can use our Theorem. Tuesday, January 26, 2010 ...
View Full
Document
 Spring '10
 FarzanAzadeh

Click to edit the document details