This preview shows page 1. Sign up to view the full content.
Unformatted text preview: 4/19/11 Goals for Today • Be able to state the principle of induc+on Induc+on Iden+fy its rela+onship to recursion State how it is diﬀerent from recursion • Be able to understand induc+ve proofs Iden+fy the reason why induc+on is necessary Follow most important steps of the proof Lecture 22 Spring 2011 • Be able to construct simple induc+ve proofs More of this to come next lecture, discussion 1 Overview 2 How do we know this is true? Merge Sort /** /** * Merge 2 subarrays of x, using y as temp * Sorts the Comparable array x between lo */ * (inclusive) and hi (exclusive), recursively and private void merge(T x, int lo, int mid, int hi, * in O(n log n) +me Or that this is true? T y) { */ int i = lo; // subarray pointers int j = mid; private void mergeSort(T x, int lo, int hi, T y) { int k = lo; // des+na+on pointer // base case • Recursion A programming/algorithm strategy Solves a problem by reducing it to simpler or smaller instance(s) of the same problem if (hi <= lo + 1) return; // nothing to do • Induc5on A mathema5cal proof technique Proves statements about natural numbers 0,1,2,... (or more generally, induc+vely deﬁned objects) • Closely related, but diﬀerent // at least 2 elements // split and recursively sort int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } while (i < mid && j < hi) { y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : x[i++]; } // one of the subarrays is empty // copy remaining elements from the other System.arraycopy(x, i, y, k, mid
i); System.arraycopy(x, j, y, k, hi
j); // now copy everything back to original array System.arraycopy(y, lo, x, lo, hi
lo); } 3 Is this s+ll true? Merge Sort /** * Sorts the Comparable array x between lo * (inclusive) and hi (exclusive), recursively and * in O(n log n) +me How about this? */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do while (i < mid && j < hi) { y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : x[i++]; // at least 2 elements } // split and recursively sort Merge Sort /** * Sorts the Comparable array x between lo * (inclusive) and hi (exclusive), recursively and * in O(n log n) +me */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do // at least 2 elements // split and recursively sort int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } // one of the subarrays is empty // copy remaining elements from the other System.arraycopy(x, i, y, k, mid
i); System.arraycopy(x, j, y, k, hi
j); // now copy everything back to original array System.arraycopy(y, lo, x, lo, hi
lo); } 5 Recursion: • The strategy you used to perform the sor+ng • Result is algorithm/program Guides the Process /** * Merge 2 subarrays of x, using y as temp */ private void merge(T x, int lo, int mid, int hi, T y) { int i = lo; // subarray pointers int j = mid; int k = lo; // des+na+on pointer lo+1; int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } 4 Induc5on: • How you show that the the program actually sorts • Also, how you show it has O(n log n) performance • Result is a proof/argument 6 1 4/19/11 Simpler Example: Sum of Integers What are We Proving? Examples • Our claim must be a property of the natural numbers • We can describe a func+on in diﬀerent ways • S(n) = “the sum of the integers from 0 to n” S(0) = 0, …, S(3) = 0+1+2+3 = 6, … • P(n): The number n is even is a statement with variable n Write as P(n) allows (numeric) values to be subs+tuted for n • Itera5ve Deﬁni5on S(n) = 0+1+ ... + n = Σn i i=0 • P(n): MergeSort sorts any given array • P(n): On any given array of length n, MergeSort ﬁnishes in less than c (n log n) steps • For each number n, P(n) is either true or false • Are S(n) and SC(n) the same func+on? • P(n): S(n) = SC(n) • P(n): MergeSort sorts any given array of length n P(0), P(1), P(2), … • Closed form characteriza5on SC(n) = n(n+1)/2 • P(n): Number n is even or odd 7 8 Are These Func+ons the Same? • Are the same if same inputs give same outputs • Property P(n): S(n) = SC(n) • Let’s formulate S(n) in yet another way: S(n) = 0 + 1 + 2 + … + n
1 + n this is S(n
1) • Test some values and see if work S(0) = 0, S(1) = 0+1 = 1, S(2) = 0+1+2 = 3, S(3) = 0+1+2+3 = 6, Recursive Deﬁni+on • This gives us a recursive deﬁni+on: SC(0) = 0(1/2) = 0 ✓ SC(1) = 1(2/2) = 1 ✓ SC(2) = 2(3/2) = 3 ✓ SC(3) = 3(4/2) = 6 ✓ SR(0) = 0 SR (n) = SR (n
1) + n, n > 0 Base Case Recursive Case • Example: SR (4) = SR (3) + 4 = SR (2) + 3 + 4 = SR (1) + 2+ 3 + 4= SR (0) + 1 + 2 + 3 + 4 = 0 + 1 + 2 + 3 + 4 • This approach will never be complete, as
there are infinitely many n to check
9 An Intermediate Problem 10 Induc+on over Natural Numbers Goal: Prove property P(n) holds for n ≥ 0 • Are these func+ons the same? Recursive deﬁni5on: • SR(0) = 0 • SR (n) = SR (n
1) + n, n > 0 Closed form characteriza5on: • SC(n) = n(n+1)/2 1. Base Step: Show that P(0) is true Induc+ve Hypothesis 2. Induc+ve Step: Assume P(k) true for an unspeciﬁed integer k Use assump+on to show that P(k+1) is true Conclusion: Because we could have picked any k, we conclude P(n) holds for all integers n ≥ 0 • Property P(n): SR(n) = SC(n) 11 12 2 4/19/11 Dominoes 0 1 2 3 A Beser Argument • Argument: 4 (Base Step) Domino 0 falls because we push it over (Induc+ve Hypothesis) Assume domino k falls over (Induc+ve Step) Because domino k’s length is larger than the spacing, it will knock over domino k+1 (Conclusion) Because we could have picked any domino to be the kth one, the dominoes will fall over • Assume equally spaced dominos, where spacing between dominos is less than domino length. • Want to argue that all dominoes fall: Domino 0 falls because we push it over Domino 0 hits domino 1, therefore domino 1 falls Domino 1 hits domino 2, therefore domino 2 falls Domino 2 hits domino 3, therefore domino 3 falls ...
Repe++ve argument. Requires one sentence per domino. • This is an induc+ve argument Much more compact than example from last slide Works for an arbitrary number of dominoes! • What is a beser way to make this argument? 13 14 SR(n) = SC(n) for all n? Proof (by Induc+on) • Recall: P(0
) P(1) P(k) P(2) • SR(0) = 0, SR (n) = SR (n
1) + n, n > 0 • SC(n) = n(n+1)/2 P(k+1) •
•
•
• • Property P(n): SR(n) = SC(n) • Base Step: Prove P(0) using the deﬁni+on • Induc+ve Hypothesis (IH): Property P(n): SR(n) = SC(n) Base Step: SR(0) = 0 and SC(0) = 0, both by deﬁni+on Induc+ve Hypothesis: Assume SR(k) = SC(k) Induc+ve Step: SR(k+1) = SR(k) + (k+1) Deﬁni+on of SR(k+1) Assume that P(k) holds for unspeciﬁed k • Induc+ve Step: Prove that P(k+1) is true using IH and the deﬁni+on = SC(k) + (k+1) = k(k+1)/2 + (k+1) = [k(k+1)+2(k+1)]/2 = (k+1)(k+2)/2
= SC(k+1)
• Conclusion: SR(n) = SC(n) for all n ≥ 0 Induc+ve Hypothesis Deﬁni+on of SC(k) Algebra Deﬁni+on of SC(k+1) 15 Our Original Problem • S(n) = “the sum of the integers from 0 to n” S(0) = 0, …, S(3) = 0+1+2+3 = 6, … Finishing the Proof • Can just show that S(n) = SR(n) For some, this is a convincing argument: S(n) = 0 + 1 + 2 + … + n
1 + n • Itera5ve Deﬁni5on n
S(n) = 0+1+ ... + n = Σi=0 i this is S(n
1) Can also do another induc+ve proof • Or could have worked it into our original proof • Closed form characteriza5on SC(n) = n(n+1)/2 • Property P(n): S(n) = SC(n) 16 Old P(n): S(n) = SC(n) New P(n): S(n) = SR(n) = SC(n) Did we show this? 17 Implies “Recursive Go
Between” 18 3 4/19/11 A Complete Argument • Recall: A Complete Argument • Recall: • S(n) = 0 + 1 + … + n • SR(0) = 0, SR (n) = SR (n
1) + n, n > 0 • SC(0) = n(n+1)/2 •
•
•
• • S(n) = 0 + 1 + … + n • SR(0) = 0, SR (n) = SR (n
1) + n, n > 0 • SC(0) = n(n+1)/2 Property P(n): S(n) = SR(n) = SC(n) Base Step: S(0) = 0 and SR(0) = 0 and SR(0) = 0, all by deﬁni+on Induc+ve Hypothesis: Assume S(k) = SR(k) = SC(k) Induc+ve Step: First prove S(k+1) = SR(k+1) S(k+1) = 0 + 1 + … + k + (k+1) Deﬁni+on of S(k+1) = S(k) + (k+1) = SR(k) + (k+1)
= SR(k+1)
Deﬁni+on of S(k) Induc+ve Hypothesis Deﬁni+on of SR(k+1) • Property P(n): S(n) = SR(n) = SC(n) • Induc+ve Step (Con+nued): Now prove SR(k+1) = SC(k+1) SR(k+1) = SR(k) + (k+1) Deﬁni+on of SR(k+1) = SC(k) + (k+1) = k(k+1)/2 + (k+1) = [k(k+1)+2(k+1)]/2 = (k+1)(k+2)/2
= SC(k+1)
Induc+ve Hypothesis Deﬁni+on of SC(k) Algebra Deﬁni+on of SC(k+1) • Conclusion: S(n) = SR(n) = SC(n) for all n ≥ 0 19 Induc+on Requires Recursion • Either a recursive algorithm is provided 20 Example With No (Ini+al) Recursion • Claim: Can make any amount of postage above 8¢ with some combina+on of 3¢ and 5¢ stamps Induc+on used to prove property of algorithm Example: Correctness of MergeSort 5¢ 3¢ • Or you must construct a recursive algorithm May not be an actual program; could be a recursive func+on, or abstract process Example: Our “recursive go
between” for S(n), SC(n) Ouen call this the “induc+ve” strategy • Remember Algorithm or strategy: recursion Proof argument: induc+on • Property P(n): You can make n¢ of postage from some combina+on of 3¢ and 5¢ stamps • Induc+on: Prove that it can be done • Recursion: A strategy that computes the number of 3¢, 5¢ stamps needed Recursion to be used in a proof only 21 22 e “Induc+ve” Strategy Recursiv
• Given: n¢ of postage • Returns: amount of 3¢ and amount of 5¢ stamps A Note on Base Step P(0) if (n == 8) { return one 3¢, one 5¢ } else { Compute answer for (n
1)¢ Result is p 3¢ stamps, q 5¢ stamps if (q > 0) { // If there is a 5¢ stamp, replace with two 3¢ ones return p+2 3¢ stamps, q
1 5¢ stamps } else { // If no 5¢ stamp, must be at least three 3¢ ones return p
3 3¢ stamps, q+2 5¢ stamps } } P(1
) P(2) P(3) P(4) P(k) P(k+1) • Some+mes want to show a property is true for integers ≥ b • Intui5on: Knock over domino b, and dominoes in front get knocked over Not interested in 0, 1, ..., (b
1) • In general, the base step in induc+on does not have to be 0 • If base step is some integer b Induc+on proves the proposi+on for n = b, b+1, b+2, ... Does not say anything about n = 0, 1, ..., b
1 23 24 4 4/19/11 Induc+on: Base Step • Given: n¢ of postage n 8 • = Returns: amount of 3¢ and amount of 5¢ stamps Induc+on: Induc+ve Step • Given: n¢ of postage n • (k+1) = Returns: amount of 3¢ and amount of 5¢ stamps Base Step: 3¢+5¢ = 8¢ if (n == 8) { return one 3¢, one 5¢ } else { Compute answer for (n
1)¢ Result is p 3¢ stamps, q 5¢ stamps if (q > 0) { // If there is a 5¢ stamp, replace with two 3¢ ones return p+2 3¢ stamps, q
1 5¢ stamps } else { // If no 5¢ stamp, must be at least three 3¢ ones return p
3 3¢ stamps, q+2 5¢ stamps } } if (n == 8) { return one 3¢, one 5¢ IH: (3p)¢+(5q)¢ = k¢ } else { Compute answer for (n
1)¢ Induc+ve Step: Result is p 3¢ stamps, q 5¢ stamps (3p+6)¢+(5q
5)¢ = (3p)¢+(5q)¢+1¢ = (k+1)¢ if (q > 0) { // If there is a 5¢ stamp, replace w ith t wo 3¢ ones return p+2 3¢ stamps, q
1 5¢ stamps } else { // If no 5¢ stamp, must be at least three 3¢ ones return p
3 3¢ stamps, q+2 5¢ stamps Induc+ve Step: } (3p
9)¢+(5q+10)¢ = (3p)¢+(5q)¢+1¢ } = (k+1)¢ 25 Cleaning it Up: Induc+ve Proof 26 e Alternate “Induc+ve” Strategy Recursiv
• Given: n¢ of postage • Returns: amount of 3¢ and amount of 5¢ stamps • Claim: You can make any amount of postage above 8¢ with some combina+on of 3¢ and 5¢ stamps • Base Step: It is true for 8¢, because 8 = 3 + 5 • Induc+ve Hypothesis: Suppose true for some k ≥ 8 • Induc+ve Step: If we used a 5¢ stamp to make k, we replace it by two 3¢ stamps. This gives k+1 If did not use a 5¢ stamp to make k, we must have used at least three 3¢ stamps. Replace three 3¢ stamps by two 5¢ stamps. This gives k+1. • Conclusion: Any amount of postage above 8¢ can be made with some combina+on of 3¢ and 5¢ stamps if (n == 8) { return one 3¢, one 5¢ stamp } else if (n == 9) { return three 3¢ stamps } else if (n == 10) { return two 5¢ stamps } else { Compute answer for (n
3)¢ Result is p 3¢ stamps, q 5¢ stamps return p+1 3¢ stamps, q 5¢ stamps } 27 Strong Induc+on 28 Strong Induc+on: Base Step • Given: n¢ of postage • Weak induc5on n = • , 9Returns: amount of 3¢ and amount of 5¢ stamps 8 , 10 P(0): Show that property P is true for 0 P(k) => P(k+1): Show that if property P is true for k, it is true for k+1 Conclude that P(n) holds for all n Base Step (part 1): 3¢+5¢ = 8¢ if (n == 8) { return one 3¢, one 5¢ stamp } else if (n == 9) { Base Step (part 2): 3¢+3¢+3¢ = 9¢ return three 3¢ stamps } else if (n == 10) { Base Step (part 3): 5¢+5¢ = 10¢ return two 5¢ stamps } else { Compute answer for (n
3)¢ Result is p 3¢ stamps, q 5¢ stamps return p+1 3¢ stamps, q 5¢ stamps } • Strong induc5on P(0), …, P(m): Show property P is true for 0 to m P(0) and P(1) and ... and P(k) => P(k+1): Show that if P is true for numbers less than or equal to k, then it is true for k+1 Conclude that P(n) holds for all n • Both proof techniques are equally powerful 29 30 5 4/19/11 Strong Induc+on: Induc+ve Step • Given: n¢ of postage k+1 •n =Returns: amount of 3¢ and amount of 5¢ stamps if (n == 8) { Strong Induc+on Hypothesis: return one 3¢, one 5¢ stamp Strategy works for any amount of } else if (n == 9) { postage m, where 8 ≤ m ≤ k return three 3¢ stamps } else if (n == 10) { return two 5¢ stamps } else { SIH: (3p)¢+(5q)¢ = (k
2)¢ Compute answer for (n
3)¢ Result is p 3¢ stamps, q 5¢ stamps return p+1 3¢ stamps, q 5¢ stamps Induc+ve Step: } (3p+3)¢+(5q)¢ = (k+1)¢ Clean Up: Strong Induc+ve Proof • Claim: You can make any amount of postage above 8¢ with some combina+on of 3¢ and 5¢ stamps • Base Step: We consider three base cases: 8¢, 9¢, and 10¢ It is true for 8¢, since 3+5 = 8 It is true for 9¢, since 3+3+3 = 9 It is true for 10¢, since 5+5 = 10 • (Strong) Induc+ve Hypothesis: Suppose there is some k such that claim is true for all numbers m, where 8 ≤ m ≤ k • Induc+ve Step: As 8 ≤ k
2 ≤ k, make postage for (k
2)¢ and add a 3¢ stamp. This gives answer for (k+1)¢. • Conclusion: Any amount of postage above 8¢ can be made with some combina+on of 3¢ and 5¢ stamps 31 Merge Sort: Correctness Idea /** * Sorts the Comparable array x between lo Property P(n): * (inclusive) and oi (length n aecursively and For any array h f exclusive), rs input, * in O(n log n() sorts the contents in place. mergeSort) +me */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do // at least 2 elements // split and recursively sort int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } /** * Merge 2 subarrays of x, using y as temp */ Any array of private void merge(T x, int lo, int mid, int hi, length 0 o r 1 T y) { int i = lo; // subarray pointers int j = mid; int k = lo; // des+na+on pointer Merge Sort: Correctness Idea /** * Sorts the Comparable array x between lo Property P(n): * (inclusive) and oi (length n aecursively and For any array h f exclusive), rs input, * in O(n log n() sorts the contents in place. mergeSort) +me */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do while (i < mid && j < hi) { Base Step: y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : Array of 1 or 0 elements i s x[i++]; already sorted } // at least 2 elements // split and recursively sort int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } // one of the subarrays is empty // copy remaining elements from the other System.arraycopy(x, i, y, k, mid
i); System.arraycopy(x, j, y, k, hi
j); // now copy everything back to original array System.arraycopy(y, lo, x, lo, hi
lo); } 33 Induc+on and Recursion /** * Merge 2 subarrays of x, using y as temp */ Any array of private void merge(T x, int lo, int mid, int hi, length k +1 T y) { int i = lo; // subarray pointers int j = mid; int k = lo; // des+na+on pointer while (i < mid && j < hi) { y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : x[i++]; } Strong Induc+ve Hypothesis: /of length m ≤ k, For any array / one of the subarrays is empty // copy remaining elements from the other mergeSort() system.arraycopy(x, i, y, k, mid
i); orts the array S
System.arraycopy(x, j, y, k, hi
j); // Induc+ve Step: now copy everything back to original array System.arraycopy(y, lo, x, lo, hi
lo); Show that merge() takes two } sorted halves and produces a single sorted array of length k+1 34 Summary of Today • Induc+on is a technique to prove statements Recursion: Recursion is a strategy to construct algorithms Useful for program correctness and complexity • A strategy used to solve a problem or compute result • Result is algorithm/program Provides the “Induc+ve” Strategy 32 Guides the Algorithm Development Induc5on: • Prove statements about the natural numbers • Prove proper+es of recursive algorithms • Result is a proof/argument 35 • But all induc+on requires a recursive strategy Hard part is ﬁnding the strategy Auerwards, induc+on is ouen straighzorward Diﬀerent varia+ons of induc+on exist to tailor to your recursive strategy 36 6 4/19/11 To Think About for Next Time /** * Sorts the Comparable array x between lo * (inclusive) and hi (exclusive), recursively and * in O(n log n) +me */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do // at least 2 elements // split and recursively sort lo+1; int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } /** * Merge 2 subarrays of x, using y as temp */ private void merge(T x, int lo, int mid, int hi, T y) { int i = lo; // subarray pointers int j = mid; int k = lo; // des+na+on pointer while (i < mid && j < hi) { y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : x[i++]; } What does this d o // one of the subarrays is empty to complexity? // copy remaining elements from the other System.arraycopy(x, i, y, k, mid
i); System.arraycopy(x, j, y, k, hi
j); // now copy everything back to original array System.arraycopy(y, lo, x, lo, hi
lo); } 37 7 ...
View
Full
Document
 '08
 STAFF
 Recursion, Sort

Click to edit the document details