22-Induction - 4/19/11 Goals for Today •  Be able...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 4/19/11 Goals for Today •  Be able to state the principle of induc+on Induc+on   Iden+fy its rela+onship to recursion   State how it is different from recursion •  Be able to understand induc+ve proofs   Iden+fy the reason why induc+on is necessary   Follow most important steps of the proof Lecture 22 Spring 2011 •  Be able to construct simple induc+ve proofs   More of this to come next lecture, discussion 1 Overview 2 How do we know this is true? Merge Sort /** /** * Merge 2 subarrays of x, using y as temp * Sorts the Comparable array x between lo */ * (inclusive) and hi (exclusive), recursively and private void merge(T x, int lo, int mid, int hi, * in O(n log n) +me Or that this is true? T y) { */ int i = lo; // subarray pointers int j = mid; private void mergeSort(T x, int lo, int hi, T y) { int k = lo; // des+na+on pointer // base case •  Recursion   A programming/algorithm strategy   Solves a problem by reducing it to simpler or smaller instance(s) of the same problem if (hi <= lo + 1) return; // nothing to do •  Induc5on   A mathema5cal proof technique   Proves statements about natural numbers 0,1,2,...   (or more generally, induc+vely defined objects) •  Closely related, but different // at least 2 elements // split and recursively sort int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } while (i < mid && j < hi) { y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : x[i++]; } // one of the subarrays is empty // copy remaining elements from the other System.arraycopy(x, i, y, k, mid  ­ i); System.arraycopy(x, j, y, k, hi  ­ j); // now copy everything back to original array System.arraycopy(y, lo, x, lo, hi  ­ lo); } 3 Is this s+ll true? Merge Sort /** * Sorts the Comparable array x between lo * (inclusive) and hi (exclusive), recursively and * in O(n log n) +me How about this? */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do while (i < mid && j < hi) { y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : x[i++]; // at least 2 elements } // split and recursively sort Merge Sort /** * Sorts the Comparable array x between lo * (inclusive) and hi (exclusive), recursively and * in O(n log n) +me */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do // at least 2 elements // split and recursively sort int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } // one of the subarrays is empty // copy remaining elements from the other System.arraycopy(x, i, y, k, mid  ­ i); System.arraycopy(x, j, y, k, hi  ­ j); // now copy everything back to original array System.arraycopy(y, lo, x, lo, hi  ­ lo); } 5 Recursion: •  The strategy you used to perform the sor+ng •  Result is algorithm/program Guides the Process /** * Merge 2 subarrays of x, using y as temp */ private void merge(T x, int lo, int mid, int hi, T y) { int i = lo; // subarray pointers int j = mid; int k = lo; // des+na+on pointer lo+1; int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } 4 Induc5on: •  How you show that the the program actually sorts •  Also, how you show it has O(n log n) performance •  Result is a proof/argument 6 1 4/19/11 Simpler Example: Sum of Integers What are We Proving? Examples •  Our claim must be a property of the natural numbers •  We can describe a func+on in different ways •  S(n) = “the sum of the integers from 0 to n” S(0) = 0, …, S(3) = 0+1+2+3 = 6, … •  P(n): The number n is even   is a statement with variable n   Write as P(n)   allows (numeric) values to be subs+tuted for n •  Itera5ve Defini5on S(n) = 0+1+ ... + n = Σn i i=0 •  P(n): MergeSort sorts any given array •  P(n): On any given array of length n, MergeSort finishes in less than c (n log n) steps •  For each number n, P(n) is either true or false •  Are S(n) and SC(n) the same func+on? •  P(n): S(n) = SC(n) •  P(n): MergeSort sorts any given array of length n P(0), P(1), P(2), … •  Closed form characteriza5on SC(n) = n(n+1)/2 •  P(n): Number n is even or odd 7 8 Are These Func+ons the Same? •  Are the same if same inputs give same outputs •  Property P(n): S(n) = SC(n) •  Let’s formulate S(n) in yet another way: S(n) = 0 + 1 + 2 + … + n ­1 + n this is S(n ­1) •  Test some values and see if work   S(0) = 0,   S(1) = 0+1 = 1,   S(2) = 0+1+2 = 3,   S(3) = 0+1+2+3 = 6, Recursive Defini+on •  This gives us a recursive defini+on: SC(0) = 0(1/2) = 0 ✓ SC(1) = 1(2/2) = 1 ✓ SC(2) = 2(3/2) = 3 ✓ SC(3) = 3(4/2) = 6 ✓   SR(0) = 0   SR (n) = SR (n ­1) + n, n > 0 Base Case Recursive Case •  Example:   SR (4) = SR (3) + 4 = SR (2) + 3 + 4 = SR (1) + 2+ 3 + 4= SR (0) + 1 + 2 + 3 + 4 = 0 + 1 + 2 + 3 + 4 •  This approach will never be complete, as there are infinitely many n to check 9 An Intermediate Problem 10 Induc+on over Natural Numbers Goal: Prove property P(n) holds for n ≥ 0 •  Are these func+ons the same?   Recursive defini5on: •  SR(0) = 0 •  SR (n) = SR (n ­1) + n, n > 0   Closed form characteriza5on: •  SC(n) = n(n+1)/2 1.  Base Step:   Show that P(0) is true Induc+ve Hypothesis 2.  Induc+ve Step:   Assume P(k) true for an unspecified integer k   Use assump+on to show that P(k+1) is true Conclusion: Because we could have picked any k, we conclude P(n) holds for all integers n ≥ 0 •  Property P(n): SR(n) = SC(n) 11 12 2 4/19/11 Dominoes 0 1 2 3 A Beser Argument •  Argument: 4   (Base Step) Domino 0 falls because we push it over   (Induc+ve Hypothesis) Assume domino k falls over   (Induc+ve Step) Because domino k’s length is larger than the spacing, it will knock over domino k+1   (Conclusion) Because we could have picked any domino to be the kth one, the dominoes will fall over •  Assume equally spaced dominos, where spacing between dominos is less than domino length. •  Want to argue that all dominoes fall: Domino 0 falls because we push it over Domino 0 hits domino 1, therefore domino 1 falls Domino 1 hits domino 2, therefore domino 2 falls Domino 2 hits domino 3, therefore domino 3 falls ...         Repe++ve argument. Requires one sentence per domino. •  This is an induc+ve argument   Much more compact than example from last slide   Works for an arbitrary number of dominoes! •  What is a beser way to make this argument? 13 14 SR(n) = SC(n) for all n? Proof (by Induc+on) •  Recall: P(0 ) P(1) P(k) P(2) •  SR(0) = 0, SR (n) = SR (n ­1) + n, n > 0 •  SC(n) = n(n+1)/2 P(k+1) •  •  •  •  •  Property P(n): SR(n) = SC(n) •  Base Step:   Prove P(0) using the defini+on •  Induc+ve Hypothesis (IH): Property P(n): SR(n) = SC(n) Base Step: SR(0) = 0 and SC(0) = 0, both by defini+on Induc+ve Hypothesis: Assume SR(k) = SC(k) Induc+ve Step: SR(k+1) = SR(k) + (k+1) Defini+on of SR(k+1)   Assume that P(k) holds for unspecified k •  Induc+ve Step:   Prove that P(k+1) is true using IH and the defini+on = SC(k) + (k+1) = k(k+1)/2 + (k+1) = [k(k+1)+2(k+1)]/2 = (k+1)(k+2)/2 = SC(k+1) •  Conclusion: SR(n) = SC(n) for all n ≥ 0 Induc+ve Hypothesis Defini+on of SC(k) Algebra Defini+on of SC(k+1) 15 Our Original Problem •  S(n) = “the sum of the integers from 0 to n” S(0) = 0, …, S(3) = 0+1+2+3 = 6, … Finishing the Proof •  Can just show that S(n) = SR(n)   For some, this is a convincing argument: S(n) = 0 + 1 + 2 + … + n ­1 + n •  Itera5ve Defini5on n S(n) = 0+1+ ... + n = Σi=0 i this is S(n ­1)   Can also do another induc+ve proof •  Or could have worked it into our original proof •  Closed form characteriza5on SC(n) = n(n+1)/2 •  Property P(n): S(n) = SC(n) 16   Old P(n): S(n) = SC(n)   New P(n): S(n) = SR(n) = SC(n) Did we show this? 17 Implies “Recursive Go ­Between” 18 3 4/19/11 A Complete Argument •  Recall: A Complete Argument •  Recall: •  S(n) = 0 + 1 + … + n •  SR(0) = 0, SR (n) = SR (n ­1) + n, n > 0 •  SC(0) = n(n+1)/2 •  •  •  •  •  S(n) = 0 + 1 + … + n •  SR(0) = 0, SR (n) = SR (n ­1) + n, n > 0 •  SC(0) = n(n+1)/2 Property P(n): S(n) = SR(n) = SC(n) Base Step: S(0) = 0 and SR(0) = 0 and SR(0) = 0, all by defini+on Induc+ve Hypothesis: Assume S(k) = SR(k) = SC(k) Induc+ve Step: First prove S(k+1) = SR(k+1) S(k+1) = 0 + 1 + … + k + (k+1) Defini+on of S(k+1) = S(k) + (k+1) = SR(k) + (k+1) = SR(k+1) Defini+on of S(k) Induc+ve Hypothesis Defini+on of SR(k+1) •  Property P(n): S(n) = SR(n) = SC(n) •  Induc+ve Step (Con+nued): Now prove SR(k+1) = SC(k+1) SR(k+1) = SR(k) + (k+1) Defini+on of SR(k+1) = SC(k) + (k+1) = k(k+1)/2 + (k+1) = [k(k+1)+2(k+1)]/2 = (k+1)(k+2)/2 = SC(k+1) Induc+ve Hypothesis Defini+on of SC(k) Algebra Defini+on of SC(k+1) •  Conclusion: S(n) = SR(n) = SC(n) for all n ≥ 0 19 Induc+on Requires Recursion •  Either a recursive algorithm is provided 20 Example With No (Ini+al) Recursion •  Claim: Can make any amount of postage above 8¢ with some combina+on of 3¢ and 5¢ stamps   Induc+on used to prove property of algorithm   Example: Correctness of MergeSort 5¢ 3¢ •  Or you must construct a recursive algorithm   May not be an actual program; could be a recursive func+on, or abstract process   Example: Our “recursive go ­between” for S(n), SC(n)   Ouen call this the “induc+ve” strategy •  Remember   Algorithm or strategy: recursion   Proof argument: induc+on •  Property P(n): You can make n¢ of postage from some combina+on of 3¢ and 5¢ stamps •  Induc+on: Prove that it can be done •  Recursion: A strategy that computes the number of 3¢, 5¢ stamps needed Recursion to be used in a proof only 21 22 e “Induc+ve” Strategy Recursiv •  Given: n¢ of postage •  Returns: amount of 3¢ and amount of 5¢ stamps A Note on Base Step P(0) if (n == 8) { return one 3¢, one 5¢ } else { Compute answer for (n ­1)¢ Result is p 3¢ stamps, q 5¢ stamps if (q > 0) { // If there is a 5¢ stamp, replace with two 3¢ ones return p+2 3¢ stamps, q ­1 5¢ stamps } else { // If no 5¢ stamp, must be at least three 3¢ ones return p ­3 3¢ stamps, q+2 5¢ stamps } } P(1 ) P(2) P(3) P(4) P(k) P(k+1) •  Some+mes want to show a property is true for integers ≥ b •  Intui5on:   Knock over domino b, and dominoes in front get knocked over   Not interested in 0, 1, ..., (b ­1) •  In general, the base step in induc+on does not have to be 0 •  If base step is some integer b   Induc+on proves the proposi+on for n = b, b+1, b+2, ...   Does not say anything about n = 0, 1, ..., b ­1 23 24 4 4/19/11 Induc+on: Base Step •  Given: n¢ of postage n 8 •  = Returns: amount of 3¢ and amount of 5¢ stamps Induc+on: Induc+ve Step •  Given: n¢ of postage n •  (k+1) = Returns: amount of 3¢ and amount of 5¢ stamps Base Step: 3¢+5¢ = 8¢ if (n == 8) { return one 3¢, one 5¢ } else { Compute answer for (n ­1)¢ Result is p 3¢ stamps, q 5¢ stamps if (q > 0) { // If there is a 5¢ stamp, replace with two 3¢ ones return p+2 3¢ stamps, q ­1 5¢ stamps } else { // If no 5¢ stamp, must be at least three 3¢ ones return p ­3 3¢ stamps, q+2 5¢ stamps } } if (n == 8) { return one 3¢, one 5¢ IH: (3p)¢+(5q)¢ = k¢ } else { Compute answer for (n ­1)¢ Induc+ve Step: Result is p 3¢ stamps, q 5¢ stamps (3p+6)¢+(5q ­5)¢ = (3p)¢+(5q)¢+1¢ = (k+1)¢ if (q > 0) { // If there is a 5¢ stamp, replace w ith t wo 3¢ ones return p+2 3¢ stamps, q ­1 5¢ stamps } else { // If no 5¢ stamp, must be at least three 3¢ ones return p ­3 3¢ stamps, q+2 5¢ stamps Induc+ve Step: } (3p ­9)¢+(5q+10)¢ = (3p)¢+(5q)¢+1¢ } = (k+1)¢ 25 Cleaning it Up: Induc+ve Proof 26 e Alternate “Induc+ve” Strategy Recursiv •  Given: n¢ of postage •  Returns: amount of 3¢ and amount of 5¢ stamps •  Claim: You can make any amount of postage above 8¢ with some combina+on of 3¢ and 5¢ stamps •  Base Step: It is true for 8¢, because 8 = 3 + 5 •  Induc+ve Hypothesis: Suppose true for some k ≥ 8 •  Induc+ve Step:   If we used a 5¢ stamp to make k, we replace it by two 3¢ stamps. This gives k+1   If did not use a 5¢ stamp to make k, we must have used at least three 3¢ stamps. Replace three 3¢ stamps by two 5¢ stamps. This gives k+1. •  Conclusion: Any amount of postage above 8¢ can be made with some combina+on of 3¢ and 5¢ stamps if (n == 8) { return one 3¢, one 5¢ stamp } else if (n == 9) { return three 3¢ stamps } else if (n == 10) { return two 5¢ stamps } else { Compute answer for (n ­3)¢ Result is p 3¢ stamps, q 5¢ stamps return p+1 3¢ stamps, q 5¢ stamps } 27 Strong Induc+on 28 Strong Induc+on: Base Step •  Given: n¢ of postage •  Weak induc5on n = • , 9Returns: amount of 3¢ and amount of 5¢ stamps 8 , 10   P(0): Show that property P is true for 0   P(k) => P(k+1): Show that if property P is true for k, it is true for k+1   Conclude that P(n) holds for all n Base Step (part 1): 3¢+5¢ = 8¢ if (n == 8) { return one 3¢, one 5¢ stamp } else if (n == 9) { Base Step (part 2): 3¢+3¢+3¢ = 9¢ return three 3¢ stamps } else if (n == 10) { Base Step (part 3): 5¢+5¢ = 10¢ return two 5¢ stamps } else { Compute answer for (n ­3)¢ Result is p 3¢ stamps, q 5¢ stamps return p+1 3¢ stamps, q 5¢ stamps } •  Strong induc5on   P(0), …, P(m): Show property P is true for 0 to m   P(0) and P(1) and ... and P(k) => P(k+1): Show that if P is true for numbers less than or equal to k, then it is true for k+1   Conclude that P(n) holds for all n •  Both proof techniques are equally powerful 29 30 5 4/19/11 Strong Induc+on: Induc+ve Step •  Given: n¢ of postage k+1 •n  =Returns: amount of 3¢ and amount of 5¢ stamps if (n == 8) { Strong Induc+on Hypothesis: return one 3¢, one 5¢ stamp Strategy works for any amount of } else if (n == 9) { postage m, where 8 ≤ m ≤ k return three 3¢ stamps } else if (n == 10) { return two 5¢ stamps } else { SIH: (3p)¢+(5q)¢ = (k ­2)¢ Compute answer for (n ­3)¢ Result is p 3¢ stamps, q 5¢ stamps return p+1 3¢ stamps, q 5¢ stamps Induc+ve Step: } (3p+3)¢+(5q)¢ = (k+1)¢ Clean Up: Strong Induc+ve Proof •  Claim: You can make any amount of postage above 8¢ with some combina+on of 3¢ and 5¢ stamps •  Base Step: We consider three base cases: 8¢, 9¢, and 10¢   It is true for 8¢, since 3+5 = 8   It is true for 9¢, since 3+3+3 = 9   It is true for 10¢, since 5+5 = 10 •  (Strong) Induc+ve Hypothesis: Suppose there is some k such that claim is true for all numbers m, where 8 ≤ m ≤ k •  Induc+ve Step: As 8 ≤ k ­2 ≤ k, make postage for (k ­2)¢ and add a 3¢ stamp. This gives answer for (k+1)¢. •  Conclusion: Any amount of postage above 8¢ can be made with some combina+on of 3¢ and 5¢ stamps 31 Merge Sort: Correctness Idea /** * Sorts the Comparable array x between lo Property P(n): * (inclusive) and oi (length n aecursively and For any array h f exclusive), rs input, * in O(n log n() sorts the contents in place. mergeSort) +me */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do // at least 2 elements // split and recursively sort int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } /** * Merge 2 subarrays of x, using y as temp */ Any array of private void merge(T x, int lo, int mid, int hi, length 0 o r 1 T y) { int i = lo; // subarray pointers int j = mid; int k = lo; // des+na+on pointer Merge Sort: Correctness Idea /** * Sorts the Comparable array x between lo Property P(n): * (inclusive) and oi (length n aecursively and For any array h f exclusive), rs input, * in O(n log n() sorts the contents in place. mergeSort) +me */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do while (i < mid && j < hi) { Base Step: y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : Array of 1 or 0 elements i s x[i++]; already sorted } // at least 2 elements // split and recursively sort int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } // one of the subarrays is empty // copy remaining elements from the other System.arraycopy(x, i, y, k, mid  ­ i); System.arraycopy(x, j, y, k, hi  ­ j); // now copy everything back to original array System.arraycopy(y, lo, x, lo, hi  ­ lo); } 33 Induc+on and Recursion /** * Merge 2 subarrays of x, using y as temp */ Any array of private void merge(T x, int lo, int mid, int hi, length k +1 T y) { int i = lo; // subarray pointers int j = mid; int k = lo; // des+na+on pointer while (i < mid && j < hi) { y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : x[i++]; } Strong Induc+ve Hypothesis: /of length m ≤ k, For any array / one of the subarrays is empty // copy remaining elements from the other mergeSort() system.arraycopy(x, i, y, k, mid  ­ i); orts the array S System.arraycopy(x, j, y, k, hi  ­ j); // Induc+ve Step: now copy everything back to original array System.arraycopy(y, lo, x, lo, hi  ­ lo); Show that merge() takes two } sorted halves and produces a single sorted array of length k+1 34 Summary of Today •  Induc+on is a technique to prove statements Recursion:   Recursion is a strategy to construct algorithms   Useful for program correctness and complexity •  A strategy used to solve a problem or compute result •  Result is algorithm/program Provides the “Induc+ve” Strategy 32 Guides the Algorithm Development Induc5on: •  Prove statements about the natural numbers •  Prove proper+es of recursive algorithms •  Result is a proof/argument 35 •  But all induc+on requires a recursive strategy   Hard part is finding the strategy   Auerwards, induc+on is ouen straighzorward   Different varia+ons of induc+on exist to tailor to your recursive strategy 36 6 4/19/11 To Think About for Next Time /** * Sorts the Comparable array x between lo * (inclusive) and hi (exclusive), recursively and * in O(n log n) +me */ private void mergeSort(T x, int lo, int hi, T y) { // base case if (hi <= lo + 1) return; // nothing to do // at least 2 elements // split and recursively sort lo+1; int mid = (lo + hi)/2; mergeSort(x, lo, mid, y); mergeSort(x, mid, hi, y); // merge sorted sublists merge(x, lo, mid, hi, y); } /** * Merge 2 subarrays of x, using y as temp */ private void merge(T x, int lo, int mid, int hi, T y) { int i = lo; // subarray pointers int j = mid; int k = lo; // des+na+on pointer while (i < mid && j < hi) { y[k++] = (x[i].compareTo(x[j]) > 0)? x[j++] : x[i++]; } What does this d o // one of the subarrays is empty to complexity? // copy remaining elements from the other System.arraycopy(x, i, y, k, mid  ­ i); System.arraycopy(x, j, y, k, hi  ­ j); // now copy everything back to original array System.arraycopy(y, lo, x, lo, hi  ­ lo); } 37 7 ...
View Full Document

Ask a homework question - tutors are online