MIT16_410F10_lec04

MIT16_410F10_lec04 - 7/2/11 Soundness and Completeness of...

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 7/2/11 Soundness and Completeness of State Space Search Sertac Karaman 16.410‐13 Sept 20th, 2010 1 Assignments •  Remember: Problem Set #2: Uninformed search Out last Wednesday, Due this Wednesday, September 22nd •  Reading: –  Today: Proofs and InducNon: Lecture 2 and 3 Notes of 6.042. –  Wednesday: [AIMA] Ch. 6.1; 24.3‐5 Constraint SaNsfacNon. •  To learn more: Constraint Processing, by Rina Dechter –  Chapter 2: Constraint Networks –  Chapter 3: Consistency Enforcing and PropagaNon 2 1 9/20/10  Autonomous Systems:  •  Plan complex sequences of acVons  •  Schedule Vght resources  •  Monitor and diagnose behavior  •  Repair or reconﬁgure hardware.   formulate as state space search.  Brian Williams, Fall  10  3  Formalizing Graph Search  Input: A search problem SP =  <g, S, G> where  •  graph g = <V, E>,   •  start vertex S in V, and  •  goal vertex G in V.   Output: A simple path P = <S, v2, … G> in g from S to G.           (i.e., <vi,vi+1> ∈ E, and vi ≠ vj if i ≠ j ).  C G A D S B Brian Williams, Fall  10  4  2  9/20/10  Graph Search is a Kind   of State Space Search  A G S B <S>  Graph Search is a Kind   Of Tree Search  S  <A, S>  A  <B, S>  B  < A, B, S>  B  <G, B, S>  G  Brian Williams, Fall  10  5  SoluVon: Depth First Search (DFS)  S A D C B C D G G C G SoluVon: Breadth First Search (BFS)  S A D C B C G D C G G Brian Williams, Fall  10  6  3  9/20/10  SoluVon: Depth First Search (DFS)  S A D C C Depth-first: B D G Add path extensions to front of Q G C Pick first element of Q G SoluVon: Breadth First Search (BFS)  S A D C C Breadth-first: B D G C Add path extensions to back of Q G    Pick first element of Q G Brian Williams, Fall  10  7  The Worst of The Worst  Which is beler, depth‐ﬁrst or breadth‐ﬁrst?  S A D C C B C D G C G A S G G D B •  Assume d = m in the worst case, and call both m.  •  Best‐ﬁrst can’t expand to level m+1, just m.  Search Method Worst Time Worst Space Shortest Path? Guaranteed to find path? Depth-first ~bm b*m No Yes for finite graph Breadth-first ~bm bm Yes unit lngth Yes Worst case time is proportional to number of nodes visited Worst case space is proportional to maximal length of Q Brian Williams, Fall  10  8  4  9/20/10  Elements of Algorithm Design  DescripVon: (last Monday)  –  Problem statement.  –  Stylized pseudo code, suﬃcient to analyze and implement the  algorithm.  –  ImplementaVon (last Wednesday).  Analysis: (last Wednesday)  •  Performance:  –  Time complexity:   •  how long does it take to ﬁnd a soluVon?  –  Space complexity:   •  how much memory does it need to perform search?  •  Correctness: (today)  –  Soundness:   •  when a soluVon is returned, is it guaranteed to be correct?  –  Completeness:   •  is the algorithm guaranteed to ﬁnd a soluVon when there is one?  Brian Williams, Fall  10  9  Outline  •  Review  •  Proof techniques and the axiomaVc method  •  Proofs of soundness and completeness of  search algorithms  •  Limits of axiomaVc method  Brian Williams, Fall  10  10  5  9/20/10  Envelope game  Probabili(es do not work! •  I put an amount \$N and \$2N into two diﬀerent envelopes   (you do not know N).  •  I open one of them, it has \$X.   •  Would you pick the open one or the other?  •  Reasoning 1: (I pick one at random) –  seeing inside an envelope does not maler…   •  Reasoning 2: (I pick the second one) –  If I get this envelope, I get \$X.  –  If I get the other envelope, I get, on average:  (1/2) X/2 + (1/2) 2X = (5/4) X   Unexpected hanging paradox  Induc(on does not work! •  A judge tells a criminal that   “the criminal will be hanged on a weekday at noon next week, he will not know when he will be hanged, it will be a total surprise”. •  Criminal’s reasoning:  –  He can not be hanged on a Friday (by Thursday awernoon, he  will know – it won’t be a surprise).  –  Then, he can not be hanged on Thursday either.  –  Then, he can not be hanged at all… So he feels safe.  (He was hanged on Wed. at noon – it was a total surprise…)  •  What went wrong with criminal’s deducVon?  6  9/20/10  The axiomaVc method  •  Invented by Euclid around 300BC (in  Alexandria, Egypt).   •  5 axioms of geometry menVoned in his  work Elements.   •  StarVng from these axioms,   Euclid established many “proposi2ons”   by providing “proofs”. Oldest surviving copy  of the Elements  on a  papyrus found in  Oxyrhynchus, Egypt. Euclid of Alexandria  Euclid statute in Oxford  Images are in the public domain. The axiomaVc method  •  A deﬁniVon of a “proof”:  •  Any sequence of logical deduc(ons from  axioms and previously proven proposi(ons/ statements that concludes with the  proposiVon in quesVon.  Euclid of Alexandria  •  There are many types of “proposi2ons”:  •  Theorem: Important results, main results  •  Lemma: a preliminary proposiVon for proving  later results  •  Corollary: An easy  (but important) conclusion,  an awerthought of a theorem.  Euclid statute in Oxford  7  9/20/10  The axiomaVc method  •  Euclid’s axiom‐proof approach is now  fundamental to mathemaVcs!  •  Amazingly, essenVally all mathemaVcs can  be derived from just a handful of axioms…  Euclid of Alexandria  •  How to even start a proof?  •  There are many “templates”   (outlines, or techniques)  •  The details diﬀer…  Euclid statute in Oxford  Proving an implica2on •  Several mathemaVcal claims are of the form:  •  “If P, then Q”, or equivalently “P implies Q”.  •  QuadraJcs:   ax + bx + c = 0 x a=0 •  If                                    and            , then    = 2 •  InequaliJes:  −b ± √ b2 − 4ac 2a 0≤x≤2 •  If                      , then   −x3 + 4x + 1 > 0 •  Goldbach’s Conjecture:   •   If n > 2, then n is a sum of two primes.  8  9/20/10  Proving implicaVons:  Simplest proof technique  •  To prove “P implies Q”,  –  “Assume P” and show that Q logically follows.  •  Theorem:  0≤x≤2 − –  If                      , then     x3 + 4x + 1 > 0 •  Proof:  0≤x≤2 –  Assume                        (P)   x 2−x x+2 –  Then,    ,           , and              are all non‐negaVve  x(2 − x)(2 + x) –  Then,                                 is non‐negaVve  x(2 − x)(2 + x) + 1 –  Then,                                        is non‐negaVve  –  Then, mulVplying out the lew side gives  •                                                                           (Q)  −x3 + 4x + 1 > 0 Proof by ContradicVon  •  To prove that a statement P is True.   –  Assume that it is not.  –  Show that some absurd (clearly false) statement follows.  •  Formalized: In order to prove a statement P is True –  Assume that P is False,  –  Deduce a logical contradicVon (negaVon of a previous  statement or an axiom).  9  9/20/10  Proof by ContradicVon  √ •  Theorem:         is an irraVonal number.  2 •  Proof:   √ 2 •  Assume that        is not irraVonal.  √ √ •  Then,       is a raVonal number and can be wrilen as        = a/b  2 2 where a  and b  are integers and frac(on is in lowest terms •  Then, squaring both sides and rearranging gives 2 = a2/b2  •  Then, a  must be even  •  Then, a2 must be a mulVple of 4  •  Then, 2b2 must also be a mulVple of 4  •  Then, b is also even  •  Then, the fracVon is not in the lowest terms   (since a  and b are both even)   Proof by InducVon  Pick parameter N to deﬁne problem size.  •  Number of edges in the soluVon.  •  Number of nodes in graph.  Base Step: N = 0 (or small N)  •  Prove property for N = 0  InducJon Step:  •  Assume property holds for N  •  Prove property for N+1  •  Conclusion: property holds for all problem sizes N.  20  10  9/20/10  Proof by InducVon formalized •  Let P(i) be a statement with parameter i. •  Proof by inducVon states the following implicaVon:  •  “P(0) is True” (1)     and     “P(i) implies P(i+1)” (2)  •  (1) and (2) implies “P(i) is True for all i”.  •  InducVon is one of the core principles of mathemaVcs.  •  It is generally taken as an axiom, or the axioms are  designed so that inducVon principle can be proven.  21  An inducVon example  n(n + 1) 1 + 2 + ··· + n = •  Theorem:                                             for all n.   2 •  Proof:  –  Base case: P(0) is True.  •  Because, 0 = 0.  –  InducVon step: P(n) implies P(n+1)  •  Assume that the hypothesis holds for n. •  For n + 1:  n(n + 1) 1 + 2 + · · · + n + (n + 1) = +n+1 2 (n + 1)(n + 2) = 2 11  9/20/10  A faulty inducVon  •  Theorem[!]: All horses are the same color.  •  Proof[!]:  –  Base case: P(1) is True.   •  because, there is only one horse.  –  Induc(on step: P(i) implies P(i+1).  •  Assume that P(i) is True.  •  By the induc(on hypothesis ﬁrst i horses are the same color, and the  last i horses are also the same color.  h1 , h2 , . . . , hi ,hi+1 h1 , h2 , . . . , hi , hi+1 •  So all the i+1 horses must be the same color.   •  Hence, P(i+1) is also True.  •  What went wrong  here?  Proof by Invariance  A common technique in algorithm analysis •  Show that a certain property holds throughout   in an algorithm.  •  Assume that the property holds iniVally.  •  Show that in any step that the algorithm takes,  the property sVll holds.   •  Then, property holds forever.  •  It is a simple applicaVon of inducVon. Why?   12  9/20/10  Proving statements about algorithms  Handle with care! •  Correctness of simplest algorithms may be   very hard to prove…  •  Collatz conjecture:  •  Algorithm (Half Or Triple Plus One ‐ HOTPO):   •  Given an integer n. 1.  If n is even, then n = n/2  2.  If n is odd, then n = 3n + 1  3.  If n = 1, then terminate, else go to step 1.  •  Conjecture:  For any n, the algorithm always  terminates (with n = 1).  Proving statements about algorithms  Handle with care! Collatz conjecture:  •  First proposed in 1937.  •  It is not known whether the  conjecture is true or false.     Paul Erdős (1913‐1996) ‐ famous number theorist –  “Mathema(cs is not yet ready for such problems”, 1985. First 100   numbers  First 1000   numbers  Images are in the public domain. Images by Keenan Pepper and Jon McLoone . 13  9/20/10  Soundness and Completeness of  Search Algorithms  •  Today:   •  prove statements about the search algorithms we  have studied in the class.  •  study whether the algorithm returns a correct  soluVon.   •  study whether the algorithm returns a soluVon at  all when one exists.  Soundness and Completeness  Given a problem PR, an algorithm that aYempts to solve this  problem may have the following properJes:  Soundness:    •  The soluVon returned by the algorithm is correct.  Completeness:  •  The algorithm always returns a soluVon, if one exists.  •  If there is no soluVon, the algorithm reports failure.  Also, OpJmality:  •  The algorithm returns the opVmal soluVon, if it returns one.  28  14  9/20/10  Some Other NoVons of   Soundness and Completeness  ProbabilisJc Completeness:  •  The algorithm returns a soluVon, if one exists, with probability  approaching to one as the number of iteraVons increases.  •  If there is no soluVon, it may run for forever.  ProbabilisJc Soundness:  •  The probability that the “soluVon” reported solves the  problem approaches one, as the number of iteraVons  increases.  AsymptoJc OpJmality:  •  The algorithm does not necessarily return an op(mal soluVon,  but the cost of the soluVon reported approaches the opVmal  as the number of iteraVons increases.  29  Problem: State Space Search  Input: A search problem S =  <g, S, G> where  •  graph g = <V, E>,   •  start vertex S in V, and  •  goal vertex G in V.   Output: A simple path P = <S, v2, … G> in g from S to G.  C G A D S B Brian Williams, Fall  10  30  15  9/20/10  Pseudo Code For Simple Search  Let g be a Graph S be the start vertex of g G be the Goal vertex of g. Q be a list of simple partial paths in GR, 1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. 31  Soundness and Completeness  Theorems  We would like to prove the following two theorems:  Theorem 1 (Soundness):   Simple search algorithm is sound.  Theorem 2 (Completeness):   Simple search algorithm is complete.   We will use a blend of proof techniques for proving them.  32  16  9/20/10  Soundness and Completeness  Theorems  Theorem 1 (Soundness):   Simple search algorithm is sound.  Let us prove 3 lemmas before proving this theorem.  33  A lemma towards the proof  •  Lemma 1: If <v1, v2, …, vk> is a path in the  queue at any given Vme, then vk = S.  •  Proof: (by invariance)  –  Base case: IniVally, there is only <S> in the queue.  Hence, the invariant holds.  –  Induc(on step: Let’s check that the invariant conVnues to hold in every step of the algorithm.  17  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Jme, then vk = S.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. 35  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Jme, then vk = S.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  36  18  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Jme, then vk = S.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  37  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Jme, then vk = S.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue (a path is removed)  38  19  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Jme, then vk = S.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     Several paths added, each saJsfy the invariant since N saJsﬁes it.  39  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Jme, then vk = S.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  40  20  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Jme, then vk = S.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  41  Another lemma towards the proof  •  DeﬁniJon: A path <v0, v1, …, vk> is valid if   (vi−1 , vi ) ∈ E                                   for all    i ∈ {1, 2, . . . , k} •  Lemma 2: If <v1, v2, …, vk> is a path in the queue  at any given Vme, then it is valid.  •  Proof: (by invariance)  –  Base case: IniVally there is only one path <S>, which is  valid. Hence, the invariant holds.  –  Induc(on step: Let’s check that the invariant conVnues to hold in every step of the algorithm.  21  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is valid  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. 43  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is valid  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  44  22  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is valid  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  45  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is valid  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  46  23  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is valid  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  47  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is valid  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     Note that validity holds for all newly added path (from Line 4.b)  48  24  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is valid  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  49  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is valid  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  50  25  9/20/10  Yet another lemma towards the proof  •  Lemma 3: If <v1, v2, …, vk> is a path in the  queue at any given Vme, then it is a simple  path (contains no cycles).  •  Proof: (by invariance)  –  Base case: IniVally, there is only <S> in the queue.  Hence, the invariant holds.  –  Induc(on step: Let’s check that the invariant conVnues to hold in every step of the algorithm.  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is a simple path.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. 52  26  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is a simple path.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  53  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is a simple path.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  54  27  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is a simple path.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  55  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is a simple path.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  56  28  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is a simple path.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. We would like to show that each newly added path is simple assuming N is simple.  Proof: (by contradic2on) Assume one path is not simple. Then, a children of head(N) appears in N. But, this is contradicts Line 4.b   57  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is a simple path.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  58  29  9/20/10  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is a simple path.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. Before this line: assume that invariant holds.  AIer this line: show that invariant is sJll true.     In this case no new path is added to the queue.  59  Soundness and Completeness  Theorems  Theorem 1 (Soundness):   Simple search algorithm is sound.  Proof: by contradicVon…  60  30  9/20/10  Proof of Soundness  Assume that the search algorithm is not sound:  Let the returned path be   < v0 , v1 , . . . , vk > Then, one of the following must be True:  •  1. Returned path does not start with S:  vk = S •  2. Returned path does not contain G at head:  v0 = G •  3. Some transiVon in the returned path is not valid:  (vi−1 , vi ) ∈ E /                                     for some  i ∈ {1, 2, . . . , vk } •  4. Returned path is not simple:  vi = vj i, j ∈ {0, 1, . . . , k}                          for some                                       with  i = j 61  Proof of Soundness  •  1. Returned path does not start with S:  vk = S •  But, this contradicts Lemma 1!  •  Lemma 1: If <v1, v2, …, vk> is a path in the  queue at any given Vme, then vk = S.  62  31  9/20/10  Proof of Soundness  •  2. Returned path does not contain G at head:  v0 = G •  But clearly, the returned path has the property that   Head(N)= G   •  Recall the pseudo code:  63  Pseudo Code For Simple Search  Invariant: If <v1, v2, …, vk> is a path in the queue at any given Vme, then it is a simple path.  1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. 64  32  9/20/10  Proof of Soundness  •  3. Some transiVon in the returned path is not valid:  (vi−1 , vi ) ∈ E /                                    for some  i ∈ {1, 2, . . . , vk } •  Contradicts Lemma 2!  •  Lemma 2: If <v1, v2, …, vk> is a path in the  queue at any given Vme, then it is valid.  65  Proof of Soundness  •  4. Returned path is not simple:  vi = vj i, j ∈ {0, 1, . . . , k}                          for some                                  with  i = j •  Contradicts Lemma 3!  •  Lemma 3: If <v1, v2, …, vk> is a path in the  queue at any given Vme, then it is a simple  path (contains no cycles).  66  33  9/20/10  Proof of Soundness  Assume that the search algorithm is not sound:  Let the returned path be   < v0 , v1 , . . . , vk > Then, one of the following must be True:  •  1. Returned path does not start with S:  vk = S •  2. Returned path does not contain G at head:  v0 = G •  3. Some transiVon in the returned path is not valid:  (vi−1 , vi ) ∈ E /                                     for some  i ∈ {1, 2, . . . , vk } •  4. Returned path is not simple:  vi = vj i, j ∈ {0, 1, . . . , k}                          for some                                       with  i = j 67  Proof of Soundness  Assume that the search algorithm is not sound:  We reach a contradicVon in all cases.  Hence, the simple search algorithm is sound.  34  9/20/10  Proof of Completeness  Theorem 2 (Completeness):   Simple search algorithm is complete.   Need to prove:  •  If there is a path to reach from S to G, then the  algorithm returns one path that does so.  69  Pseudo Code For Simple Search  Let g be a Graph S be the start vertex of g G be the Goal vertex of g. Q be a list of simple partial paths in GR, 1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. 70  35  9/20/10  A common technique in   analysis of algorithms   •  Let’s slightly modify the algorithm  •  We will analyze the modiﬁed algorithm.  •  Then, “project” our results to the original  algorithm.  Pseudo Code For Simple Search  Let g be a Graph S be the start vertex of g G be the Goal vertex of g. Q be a list of simple partial paths in GR, 1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  // If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. 72  36  9/20/10  Proof of Completeness  •  The modiﬁed algorithm terminates when the  queue is empty.  •  Let us prove a few lemmas regarding the  behavior of the modiﬁed algorithm  Proof of Completeness    Lemma 1: A path that is taken out of the queue is not  placed into the queue again at a later step.  •  Proof: (using logical deduc(on) •  Another way to state this: “If p =<v0, v1, …, vk> is a path  that is taken out of the queue, then p = <v0, v1, …, vk> is  not placed in to the queue at a later step.”  •  Assume that p = <v0, v1, …, vk> is taken out of the  queue.  •  Then, p must be placed in to the queue at an earlier  step.  •  Then, v0 must be in the visited list at this step.  •  Then, p = <v0, v1, …, vk> can not placed in to the queue  at a later step, since v0 is in the visited list.  37  9/20/10  Proof of Completeness  •  DeﬁniJon: A vertex v is reachable from S, if there exists a  path <v0, v1, …, vk> that starts from S and ends at v, i.e., vk =  S and v0 = v.  •  Lemma 2: If a vertex v is reachable from S, then v is placed  in to the visited list awer a ﬁnite number of steps.  Proof of Completeness  •  •  Lemma 2: If a vertex v is reachable from S, then v is placed in to the visited list  awer a ﬁnite number of steps.  Proof: (by contradic(on) •  Assume v is reachable from S, but it is never placed on the visited list.  •  Since v is reachable from S, there exists a path that is of the form <v0, v1, …,  vk>, where v0 = v and vk = S.  •  Let vi be the ﬁrst node (starVng from vk) in the chain that is never added to  the visited list.  •  (1) Note that vi was not in the visited list before this step.  •  (2) Note also that (vi+1, vi) is in E.  •  Since vi+1 was in the visited list, the queue included a path <vi+1, …, vk> (not  necessarily the same as above), where vk = S.  •  This path must have been popped from the queue, since there are only  ﬁnitely many diﬀerent parVal paths and no path is added twice (by Lemma  1)and vi was not in the visited list (see statement 1 above).  •  Since it is popped from the queue, then <vi+1, vi, …, vk> must be placed in to  queue (see statement 2 above) and vi placed in to the visited list  •  Red statements contradict!  38  9/20/10  Proof of Completeness  •  Lemma 2: If a vertex v is reachable from S,  then v is placed in to the visited list awer a  ﬁnite number of steps.  •  Corollary: In the modiﬁed algorithm, G is  placed into the visited queue.  •  “Project” back to the original algorithm: •  This is exactly when the original algorithm  terminates  Proof of Completeness  Theorem 2 (Completeness):   Simple search algorithm is complete.   •  Proof: Follows from Lemma 2 evaluated in the  original algorithm.  78  39  9/20/10  Pseudo Code For Simple Search  Let g be a Graph S be the start vertex of g G be the Goal vertex of g. Q be a list of simple partial paths in GR, 1.  Initialize Q with partial path (S) as only entry; set Visited = ( ); 2.  If Q is empty, fail. Else, pick some partial path N from Q; 3.  If head(N) = G, return N; (goal reached!) 4.  Else a)  Remove N from Q; b)  Find all children of head(N) (its neighbors in g) not in Visited and create a one-step extension of N to each child; c)  Add to Q all the extended paths; d)  Add children of head(N) to Visited; e)  Go to step 2. 79  Summarize Completeness and  Soundness  •  Hence, we have proven two theorems:  Theorem 1 (Soundness):   Simple search algorithm is sound.  Theorem 2 (Completeness):   Simple search algorithm is complete.   •  Soundness and completeness is a requirement for most algorithms,   although we will their relaxaVons quite owen  40  9/20/10  Back to the AxiomaVc Method  Does it really work?   •  EssenVally all of what we know in mathemaVcs today can  be derived from a handful of axioms called the   Zarmelo‐Frankel set theory with the axiom of Choice (ZFC).  •  These axioms were made up by Zarmelo   (they did not exist a priori, unlike physical phenomena).  •  We do not know whether these axioms are logically consistent!  –  Sounds crazy! But, happened before…   Around 1900, B. Russell discovered that the axioms of that (me were logically inconsistent, i.e., one could prove a contradic(on. Back to the AxiomaVc Method  Does it really work?   •  ZFC axioms gives one what she/he wants:   –  Theorem: 5 + 5 = 10.  •  However, absurd statements can also be driven:  –  Theorem (Banach‐Tarski): A ball can be cut into a ﬁnite  number of pieces and then the pieces can be rearranged  to build two balls of the same size of the original.  Clearly, this contradicts our geometric intui(on! Image by Benjamin D. Esham, in the public domain. 41  9/20/10  Back to the AxiomaVc Method  Does it really work?   Images of Godel, Turing, and Einstein removed due to copyright restrictions.   Back to the AxiomaVc Method  Does it really work?   On the fundamental limits of mathema2cs •  Godel showed in 1930 that there are some proposiVons that  are true, but do not logically follow from the axioms.  •  The axioms are not enough!    •  But, Godel also showed that simply adding more axioms  does not eliminate this problem. Any set of axioms that is  not contradictory will have the same problem!  •  Godel’s results are directly related to computaVon. These  results were later used by Alan Turing in 1950s to invent a  revoluVonary idea: computer…     42  9/20/10  What you should know  •  The deﬁniVons of a proposiVon, proof,  theorem, lemma, and corollary.  •  Proof techniques such as proof by  contradicVon, inducVon, invariance proofs.  •  NoVons of soundness and completeness.  •  Proving soundness and completeness of search algorithms.  43  MIT OpenCourseWare http://ocw.mit.edu 16.410 / 16.413 Principles of Autonomy and Decision Making Fall 2010 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms. ...
View Full Document

This note was uploaded on 12/26/2011 for the course SCIENCE 16.410 taught by Professor Prof.brianwilliams during the Fall '10 term at MIT.

Ask a homework question - tutors are online