Coursehero >> California >> Berkeley >> STAT 206a

Course Hero has millions of student submitted documents similar to the one below including study guides, homework solutions, papers, exam answer keys and textbook solutions.

Convergence Complete of Message Passing Algorithms for some Satis ability Problems Uriel Feige1 , Elchanan Mossel2 and Dan Vilenchik3 1 Micorosoft Research and The Weizmann Institute. urifeige@microsoft.com 2 U.C. Berkeley mossel@stat.berkeley.edu 3 Tel Aviv University vilenchi@post.tau.ac.il. Abstract. Experimental results show that certain message passing algorithms, namely, survey propagation, are very e ective in nding satisfying assignments in random satis able 3CNF formulas. In this paper we make a modest step towards providing rigorous analysis that proves the e ectiveness of message passing algorithms for random 3SAT. We analyze the performance of Warning Propagation, a popular message passing algorithm that is simpler than survey propagation. We show that for 3CNF formulas generated under the planted assignment distribution, running warning propagation in the standard way works when the clause-to-variable ratio is a su ciently large constant. We are not aware of previous rigorous analysis of message passing algorithms for satis ability instances, though such analysis was performed for decoding of Low Density Parity Check (LDPC) Codes. We discuss some of the di erences between results for the LDPC setting and our results. 1 Introduction and Results The e ectiveness of some message passing algorithms, in particular Survey Propagation, was experimentally shown for hard formulas with clause-variable ratio below (yet rather close to) the conjectured satis ability threshold, 4.2 [3]. In this paper we analyze the performance of Warning Propagation (WP for brevity), a simple popular message passing algorithm, when applied to random satis able formulas generated under the planted distribution with a constant clausevariable ratio. We show that the standard way of running message passing algorithms run message passing until convergence, simplify the formula according to the resulting assignment, and satisfy the remaining subformula (if nonempty), if possible, using a simple o the shelf heuristic works for planted random satis able formulas with a su ciently large yet constant clause-variable ratio. We are not aware of previous rigorous analysis of message passing algorithms for non-trivial SAT distributions. 1.1 Di erent SAT Distributions A CNF formula over the variables x1 , x2 , ..., xn is a conjunction of clauses C1 , C2 , ..., Cm where each clause is a disjunction of one or more literals. Each literal is 2 Uriel Feige, Elchanan Mossel and Dan Vilenchik either a variable or its negation. A formula is said to be in k-CNF form if every clause contains exactly k literals. A CNF formula is satis able if there is a boolean assignment to the variables s.t. every clause contains at least one literal which evaluates to true. 3SAT is the language of all satis able 3CNF formulas. Although 2SAT is known to be in P, 3SAT is one of the most famous NP-complete problems. In [12] it is proved that it is NP-hard to approximate MAX-3SAT (the problem of nding an assignment that satis es as many clauses as possible) within a ratio better than 7/8. Given the di culty of designing algorithms that work well in the worst case, we consider the average case performance of algorithms. One possibility for rigorously modeling average-case instances is to use random models. Algorithmic theory of random structures has been the focus of extensive research in recent years (see [10] for a survey). As part of this trend, uniformly random 3CNFs (generated by selecting at random m = m(n) clauses over the variables {x1 , ..., xn }) have attracted much attention. Random 3SAT is known to have a sharp satis ability threshold in the clause-to-variable ratio [9]. Namely, a random 3CNF with clause-to-variable ratio below the threshold is satis able whp (with high probability, meaning with probability tending to 1 as n goes to in nity ) and one with ratio above the threshold is unsatis able whp. This threshold is not known exactly (and not even known to be independent of n). The threshold is known to be at least 3.52 [14] and at most 4.506 [5]. In this work we mainly consider formulas with large clause-variable ratio. At such ratios almost all 3CNF formulas are not satis able, therefore more re ned de nitions are due. We consider three distributions on 3SAT instances. The rst, analogous to the well known random graph model Gn,p , is the distribution in which every clause, out of 23 n possible clauses, is included with probability p = 3 p(n). We denote this distribution by Pn,p . The second distribution is obtained sat from Pn,p by conditioning on satis ability, namely Pn,p [F ] = Pn,p [F |S] where S is the event that F is satis able. Lastly, we consider the planted distribution, plant Pn,p , which is obtained form Pn,p by conditioning on satis ability by a speci c plant planted assignment . Equivalently, in Pn,p , rst an assignment to the variables is picked uniformly at random. Then, every clause satis ed by is included with probability p = p(n). Throughout, we use to denote the planted assignment when the relevant instance is clear from context. sat In the context of satis ability algorithms, Pn,p is arguably the most interestsat ing and natural distribution to study. However, as pointed out frequently, Pn,p seems hard to tackle rigorously and experimentally. The planted 3SAT distribusat tion is an intermediate step towards analyzing Pn,p , and is an interesting, quite natural distribution on its own right, the analog the of planted clique, planted bisection, planted coloring, and planted bipartite hypergraphs studied e.g. in [2, 13, 6] The planted 3SAT distribution is also discussed e.g. in [8, 7]. Our main result (Theorem 2) relates to the planted 3SAT model, but some of our other sat results (such as Proposition 1 and Corollary 1) are relevant to the Pn,p setting as well. Convergence of Message Passing Algorithms for Some SAT Problems 3 1.2 3SAT and Factor Graphs Let F be a 3CNF formula on n variables and m clauses. The factor graph (e.g. [16]) of F, denoted by F G(F), is the following graph representation of F. The factor graph is a bipartite multigraph, F G(F) = (V1 V2 , E) where V1 = {x1 , x2 , ..., xn } (the set of variables) and V2 = {C1 , C2 , ..., Cm } (the set of clauses). (xi , Cj ) E i xi appears in Cj . For a 3CNF F with m clauses it holds that #E = 3m (as every clause contains exactly 3 variables). To make presentation clearer, we denote by #A the size of a set A and by |a| the absolute value of a real number a. For simplicity we assume that every clause contains three distinct variables, therefore F G is a graph (no parallel edges). 1.3 Warning Propagation Warning Propagation (WP) is a simple iterative message passing algorithm, and serves as an excellent intuitive introduction to more involved message passing algorithms such as Belief Propagation [19] and Survey Propagation [3]. These algorithms are based on the cavity method in which the messages that a clause (or a variable) receives are meant to re ect a situation in which a cavity is formed, namely, the receiving clause (or variable) is no longer part of the formula. Messages in the WP algorithm can be interpreted as warnings , telling a clause the values that variables will have if the clause keeps quiet and does not announce its wishes, and telling a variable which clauses will not be satis ed if the variable does not commit to satisfying them. We now present the algorithm in a formal way. Let F be a CNF formula. For a variable x, let N + (x) be the set of clauses in F in which x appears positively (namely, as the literal x), and N (x) be the set of clauses in which x appears negatively. For a clause C, let N + (C) be the set of positively appearing variables and respectively N (C) the set of negatively appearing ones. There are two types of messages involved in the WP algorithm. Messages sent from a variable xi to a clause Cj in which it appears: xi Cj = Ck N + (xi ),k=j Ck xi Ck N (xi ),k=j Ck xi If xi appears only in Cj then we set the message to 0. The intuitive interpretation of this message should be xi signals Cj what is currently its favorable assignment by the other clauses it appears in (a positive message means TRUE, negative one means FALSE and a 0 message means undecided). The second type are messages sent from a clause Cj to a variable xi appearing in Cj : Cj xi = xk N + (Cj ),k=i I<0 (xk Cj ) xk N (Cj ),k=i I>0 (xk Cj ) where I<0 (b) equals 1 if b < 0 and 0 otherwise (and symmetrically I>0 (b) for the case b > 0). If Cj contains only xi (which cannot be the case in 3CNF formulas) then the message is set to 1. Cj xi = 1 can be intuitively interpreted as Cj 4 Uriel Feige, Elchanan Mossel and Dan Vilenchik sending a warning to xi asking it to satisfy Cj (as all other literals signaled Cj that currently they evaluate to FALSE). Lastly, we de ne the current assignment of a variable xi to be Bi = Cj N + (xi ) Cj xi Cj N (xi ) Cj xi If Bi > 0 then x is assigned TRUE, if Bi < 0 then xi is assigned FALSE, otherwise xi is UNASSIGNED. Assume some order on the clause-variable messages (e.g. the lexicographical order on pairs of the form (j, i) representing the message Cj xi ). Given a vector {0, 1}3m in which every entry is the value of the corresponding Cj xi message, a partial assignment {T RU E, F ALSE, U N ASSIGN ED}n can be generated according to the corresponding Bi values (as previously explained). It would be convenient to think of the messages in terms of the corresponding factor graph. Every undirected edge (xi , Cj ) of the factor graph is replaced with two anti-parallel directed edges, (xi Cj ) associated with the message xi Cj and respectively the edge (Cj xi ). Warning Propagation(CNF formula F) : 1. construct the corresponding factor graph F G(F). 2. randomly initialize the clause-variable messages to 0 or 1. 3. repeat until no clause-variable message changed from the previous iteration: 3.a randomly order the edges of F G(F). 3.b update all clause-variable messages Cj xi according to the random edge order. 4. compute a partial assignment according to the Bi messages. 5. return . Note that in line 3.b. above when evaluating the clause-variable message along the edge C x, C = (x y z), the variable-clause messages concerning this calculation (z, y C) are evaluated on-the- y using the last updated values Ci y, Cj z (allowing feedback from the same iteration). We allow the algorithm not to terminate (the clause-variable messages may keep changing every iteration). If the algorithm does return an assignment then we say that it converged. In practice it is common to limit in advance the number of iterations, and if the algorithm didn t converge by then, return a failure. 1.4 Related Work and Techniques The Survey Propagation algorithm [3] experimentally outperforms all known algorithms in nding satisfying assignments to Pn,p formulas with clause-variable ratio close to the satis ability threshold (4 4.25). However, theoretical understanding of Survey Propagation and other message passing algorithm for random SAT problems is still lacking. This should be compared with the success of message passing algorithms for decoding low-density-parity-check (LDPC) Convergence of Message Passing Algorithms for Some SAT Problems 5 codes [11]. Here, the experimental success of message passing algorithms [11] was recently complemented rigourously by a large body of theoretical work, see e.g. [17, 20, 18]. Some important insights emerge from this theoretical work. In particular, it is shown that the quality of decoding improves exponentially with the number of iterations thus all but a small constant fraction of the received codeword can be decoded correctly using a constant number of iterations. Our plant analysis of WP shows that much of the coding picture is valid also for Pn,p thus providing important insights as to the success of message passing algorithms for random satis ability problems. The planted 3SAT model is similar to LDPC in many ways. Both constructions are based on random factor graphs. In codes, the received corrupted codeword provides noisy information on a single bit or on the plant parity of a small number of bits of the original codeword. In Pn,p , being the planted assignment, the clauses containing a variable xi contain noisy information on the polarity of (xi ) in the following sense each clause contains xi in a polarity coinciding with (xi ) with probability 4/7. Our results are similar in avor to the coding results. However, the combinatorial analysis provided here allows to recover an assignment satisfying all clauses, whereas in the random LDPC codes setting, message passing allows to recover only 1 o(1) fraction of the codeword correctly. In [18] it is shown that for the erasure channel, all bits may be recovered correctly using a message passing algorithm, however in this case the LDPC code is designed so that message passing works for it. We on the other hand take a well known SAT distribution and analyze the performance of a message passing algorithm on it. Moreover, the SAT setting is more involved, as there are many assignments satisfying the formula, while for the erasure channel there is a unique codeword satisfying the combinatorial constraints given by the message. As for relevant results in random graph theory, the seminal work of [2] paved the road towards dealing with large-constant-degree planted distributions. [2] present an algorithm that whp k-colors planted k-colorable graphs with a su ciently large constant expected degree. Building upon the techniques introduced by [2], [13] present an algorithm that 2-colors sparse planted 3-uniform bipartite hypergraphs and [8], solving an open question posed in [15], presents an algorithm for satisfying large constant degree planted 3SAT instances. Though in our analysis we use similar techniques to the aforementioned works, our result is conceptually di erent in the following sense. In [2, 13, 8] the starting point is the planted distribution, and then one designs an algorithm that works well under this distribution. The algorithm may be designed in such a way that makes its analysis easier. In contrast, our starting point is a given message passing algorithm (WP), and then we ask for which input distributions it works well. We cannot change the algorithm in ways that would simplify the analysis. Another di erence between our work and that of [2, 13, 8] is that unlike the algorithms analyzed in those other papers, WP is a randomized algorithm which makes its analysis more di cult. We could have simpli ed our analysis had we changed WP to be deterministic (for example, by initializing all clause-variable messages to 1 in step 2 of the algorithm), but there are good reasons why WP 6 Uriel Feige, Elchanan Mossel and Dan Vilenchik is randomized. For example, it can be shown that (the randomized version) WP converges with probability 1 on 2CNF formulas that form one cycle of implications, but might not converge if step 4 does not introduce fresh randomness in every iteration of the algorithm (details omitted). 1.5 Notation Given a 3CNF F, simplify F according to , when is a partial assignment, means: in every clause substitute every assigned variable with the value given to it by . Then remove all clauses containing literals which evaluate to true. In all remaining clauses, remove all literals which evaluate to false (the resulting instance is not necessarily in 3CNF form). Denote by F| the 3CNF F simpli ed according to . For a set of variables A V , denote by F[A] the set of clauses in which all variables belong to A. Given a 3CNF formula F, we say that a variable x is pure in F if it appears only in one polarity (namely, always appears as the literal x or always as the literal x). Let P0 be the set of pure variables in F, and C0 be the set of clauses containing a pure variable. Let L0 = F, and L1 = L0 \ C0 . Let P1 be the pure variables in L1 , namely the variables that become pure after setting the pure variables in a satisfying manner and simplifying F. Similarly, de ne C1 to be the set of clauses in L1 containing a variable from P1 . Generally, de ne Li = Li 1 \ Ci 1 , Pi to be the pure variables in Li , and Ci to be the clauses in Li containing a variable from Pi . We say that a 3CNF F is r-pure if Lr = . The following theorem is implicitly proved in [4]. Theorem 1. Let F be randomly sampled according to Pn,p , p = d/n2 , d < 1.225, then whp F is O(n)-pure. Note that if there exists an r s.t. F is r-pure then in particular F is satis able. To better understand our results it would be convenient to have the somewhat informal notion of a simple formula in mind. We call a CNF formula simple, if it can be satis ed using simple well-known heuristics (examples formulas include whose factor graph is tree-like and r-pure formulas both solvable using the pure-literal heuristic [4], formulas with small weight terminators to use the terminology of [1] e ciently solvable whp using a RWalkSat, etc). 1.6 Our Results plant Theorem 2. Let F be a 3CNF formula randomly sampled according to Pn,p , 2 where p d/n , d a su ciently large constant, and let be its planted assignment. Then the following holds with probability 1 e (d) (the probability taken over the choice of F, the random choices in line 2 of the WP algorithm, and the random order in the rst time line 4 executes): 1. WP(F) converges after at most O(log n) iterations. Convergence of Message Passing Algorithms for Some SAT Problems 7 2. Let be the partial assignment returned by WP(F), let VA denote the variables assigned to either TRUE or FALSE in , and VU the variables left UNASSIGNED. Then for every variable x VA , (x) = (x). Moreover, #VA (1 e (d) )n. 3. F| [VU ] is a simple formula which can be satis ed in time O(n). Remark 1. We also have a proof of Theorem 2 with 1 o(1) instead of 1 e (d) . This however involves a somewhat more complicated analysis exceeding the scope of this abstract. For the full details the reader is referred to the journal version. plant Proposition 1. Let F be a 3CNF formula randomly sampled according to Pn,p , 2 where p c log n/n , with c a su ciently large constant, and let be its planted assignment. Then whp after at most 2 iterations WP(F) converges, and the sat returned equals . (This result can be extended to Pn,p , see below.) Proposition 2. Let F be an r-pure CNF formula. Then after at most O(r) iterations of WP(F), regardless of the initial messages and the order of execution, the following holds: 1. WP(F) converges. 2. Let be the assignment returned by WP(F). If (x) = U N ASSIGN ED, then in every satisfying assignment x is assigned according to (x). 3. If F contains no unit clauses then is the all-UNASSIGNED vector. Corollary 1. In the setting of Theorem 1, whp WP(F) converges after at most O(n) iterations and the returned is the all-UNASSIGNED vector. The corollary follows immediately from Theorem 1 and Proposition 2. Proposition 3. Let F be a satis able CNF formula whose corresponding factor graph contains no cycles. Then F is O(n)-pure. The main idea behind the proof of Theorem 2 is to show that the formula is dense enough so that whp there exists a large subformula forcing WP to point in the correct direction. The rest of the formula induces a factor graph containing plant only trees, which are also easy for WP. We note that formulas in Pn,p , 2 with n p some large constant, are not known to be simple (in the sense that we de ned above). On the contrary, hardness evidence can be found in works such as [1], showing that RWalkSat is very unlikely to hit a satisfying assignment plant in polynomial time when running on a random Pn,p instance in the setting of Theorem 2. In the setting of Proposition 1, the formula is already dense enough so that whp it forces entirely WP to point to the planted assignment. Proposition 2 combined with Proposition 3 provide a proof to the convergence of WP on trees. Our proof of this known result gives an explicit characterization of the xed point to which WP converges (which is implicit for trees in [3]). The remainder of the paper is structured as follows. In Section 2 we discuss plant some properties that a typical instance in Pn,p possesses, and outline the proof of Theorem 2 and Proposition 1. In Section 3 we summarize our results and discuss potentially interesting lines for further research. Most details of the proofs are omitted and can be found in the journal version. 8 Uriel Feige, Elchanan Mossel and Dan Vilenchik plant Properties of a Random Pn,p Instance 2 plant In this section we discuss relevant properties of a random Pn,p instance. To simplify presentation, we assume w.l.o.g. (due to symmetry) that the planted assignment is the all-one vector. 2.1 Stable Variables De nition 1. A variable x supports a clause C with respect to a partial assignment , if it is the only variable to satisfy C under , and the other two variables are assigned by . Proposition 4. Let F be as in the setting of Theorem 2 and let FSU P P be a random variable counting the number of variables in F whose support w.r.t. is less than d/3. Then, E[FSU P P ] e (d) n. This follows from concentration arguments as every variable is expected to sup1 d port n2 n = d + O( n ) clauses. 2 2 Following the de nitions in Section 1.3, given a CNF F and a variable x, we let N ++ (x) be the set of clauses in F in which x appears positively but doesn t support w.r.t. . Let N s (x) be the set of clause in F which x supports w.r.t. . Let = (F) be some ordering of the clause-variable message edges in the factor graph of F. For an index i and a literal x (by x we denote a literal over the variable x) let i ( x ) be the set of clause-variable edges (C x) that appear before index i in the order and in which x appears in C as x . For a set of clause-variable edges E and a set of clauses C we denote by E C the subset of edges containing a clause from C as one endpoint. De nition 2. A variable x is stable in F w.r.t. an edge order if the following holds for every clause-variable edge C x (w.l.o.g. assume C = ( x y z ), C x is the i th message in ): 1. |# i (y) N ++ (y) # i ( ) N (y)| d/30. y 2. |#N ++ (y) #N (y)| d/30. 3. #N s (y) d/3 and the same holds for z. Proposition 5. Let F be as in the setting of Theorem 2, and let be a random ordering of the clause-variable messages. Let FU N ST AB be a random variable counting the number of variables in F which are not stable. Then, E[FU N ST AB ] e (d) n. This follows from concentration arguments since E[# i (y) N ++ (y) # i ( ) y N (y)] = 0, E[#N ++ (y) #N (y)] = 0, and since every variable is expected to appear in at most O(d) clauses. Let {0, 1}3#F be a clause-variable message vector. For a set of clausevariable message edges E let 1 (E) be the set of edges along which the value is Convergence of Message Passing Algorithms for Some SAT Problems 9 1 according to . For a set of clauses C, 1 (C) denotes the set of clause-variable message edges in the factor graph of F containing a clause from C as one endpoint and along which the value is 1 in . De nition 3. A variable x is violated by in if there exists a message C x, C = ( x y z ), in place i in s.t. one of the following holds: 1. |#1 ( i (y) N ++ (y)) #1 ( i ( ) N (y))| > d/30 y 2. |#1 (N ++ (y)) #1 (N (y))| > d/30 3. #1 (N s (y)) < d/7. Or one of the above holds for z. Proposition 6. Let F be as in the setting of Theorem 2, and let X be a set of stable variables w.r.t. an arbitrary ordering . Let be a random clause-variable message vector. Let FV IO be a random variable counting the number of violated variables in X. Then, E[FV IO ] e (d) #X. The proof again uses concentration arguments. 2.2 Dense Subformulas The next property we discuss is analogous to a property proved in [2] for random graphs. Loosely speaking, [2] prove that whp a random graph doesn t contain a small induced subgraph with a large average degree. Using rst moment calculations we show: Proposition 7. Let c > 1 be an arbitrary constant. Let p d/n2 , where d is plant a large constant. Then whp over F Pn,p as n there exists no subset (d) of variables U , s.t. #U e n and there are at least c#U clauses in F containing two variables from U . 2.3 The Core Variables We describe a subset of the variables, denoted throughout by H and referred to as the core variables, which plays a crucial role in the analysis. Loosely speaking, a variable is considered safe if it is stable w.r.t. the initial random order , and it is not violated by the initial clause-variable message assignments . If in addition, a safe variable xi supports many clauses w.r.t. (whose corresponding message is 1 in ), then its corresponding Bi value will agree with (xi ) after the rst iteration. This invariant needs to be preserved however in later iterations. The set H captures the notion of such variables with a self-preserving quality. There are several ways to obtain these desired properties. Formally, H = H(F, , , ) is constructed using the following iterative procedure: Let A1 be the set of variables whose support w.r.t. is at most d/3. Let A2 be the set of non-stable variables w.r.t. . 10 Uriel Feige, Elchanan Mossel and Dan Vilenchik Let A3 be the set of stable variables w.r.t. violated by . 1. Set H0 = V \ (A1 A2 A3 ). 2. While ai Hi supporting less than d/4 clauses in F[Hi ] OR appearing in more than d/30 clauses not in F[Hi ] : let Hi+1 = Hi \ {ai }. 3. Define H = Hm+1 where am := last variable removed in step 2. Proposition 8. If both and are chosen uniformly at random then whp #H (1 e (d) )n. The main idea of the proof is to observe that to begin with we eliminate very few variables (using the discussion in Section 2.1 to bound #A1 A2 A3 ). If too many variables were removed in the iterative step then a small but dense subformula exists. Proposition 7 bounds the probability of the latter occurring. 2.4 The Factor Graph of the Non-Core Variables Proposition 8 implies that for p = c log n/n2 , c a su ciently large constant, whp H contains already all variables. The following analysis is needed for the setting of Theorem 2. The non-core factor graph is the factor graph of the formula F simpli ed according to the partial assignment that assigns all core variables to their value in the plant. Proposition 9. Whp every connected component in the non-core factor graph contains O(log n) variables. Proposition 9 will not su ce to prove Theorem 2, and we need a further characterization of the non-core factor graph. Proposition 10. With probability 1 e (d) , there exists no cycle in the noncore factor graph. 2.5 Outline of Proof of Theorem 2 and Proposition 1 We start with Theorem 2 and derive Proposition 1 as an easy corollary of the analysis. The outline of the proof is as follows. We assume that the formula F and the run of WP are typical in the sense that Propositions 8, 9 and 10 hold. First we prove that after one iteration WP sets the core variables H correctly (Bi agrees with in sign) and this assignment does not change in later iterations. Therefore from iteration 2 and onwards WP is basically running on F in which variables belonging to H are substituted with their planted assignment. This subformula is satis able and its factor graph is a forest (namely, composed of disjoint trees). Therefore, convergence is guaranteed. The set VA of Theorem 2 is composed of all variables from H and those variables from the forest that get assigned. The set VU is composed of the UNASSIGNED variables from the forest. We say that a message C x, C = ( x y z ), is correct if its value is the same as it is when y C and z C agree in sign with their planted assignment (in other words, C x is 1 i x supports C w.r.t. ). Convergence of Message Passing Algorithms for Some SAT Problems 11 Proposition 11. If xi H and all messages C xi , C F[H] are correct at the beginning of an iteration (line 3 in the WP algorithm), then this invariant is kept by the end of that iteration. Proposition 12. If xi H and all messages C xi , C F[H] are correct by the end of a WP iteration, then Bi agrees in sign with (xi ) by the end of that iteration. Proposition 12 follows immediately from the de nition of H and the message Bi . It remains to show then that after the rst iteration all messages C xi , C F[H] are correct. Proposition 13. If F is a typical instance in the setting of Theorem 2, then after one iteration of WP(F), for every variable xi H, every message C xi , C F[H] is correct. Proposition 14. Let F be a typical instance in the setting of Theorem 2, then for every variable xj V \ H, after O(log n) iterations either Bj = 0 or Bj agrees in sign with (xj ). As for satisfying the set of unassigned variables in time O(n), Propositions 3 and 10 imply that the pure-literal procedure [4] solves the subformula induced by the unassigned variables in linear time. Theorem 2 then follows. To prove Proposition 1, observe that when p = c log n/n2 , with c a su ciently large constant, Proposition 8 implies H = V . Combing this with Proposition 13, Proposition 1 readily follows. 3 Discussion We conclude with an open problem. Can our analysis be extended to show that plant Belief Propagation (BP) nds a satisfying assignment to Pn,p in the setting of Theorem 2? Experimental results predict the answer to be positive. However, our analysis of WP does not extend as is to BP. In WP, all warnings received by a variable (or by a clause) have equal weight, but in BP this need not be the case (there is a probability level associated with each warning). In particular, this may lead to the case that messages received from non-core portions of the formula can e ect the core, a possibility that our analysis managed to exclude for the WP algorithm. Acknowledgements: We thank Eran Ofek for many useful discussions. This work was done while the authors were visiting Microsoft Research, Redmond, Washington. E.M is supported by a Sloan fellowship in Mathematics, by NSF Career award DMS0548249 and NSF grants DMS-0528488 and DMS-0504245. 12 Uriel Feige, Elchanan Mossel and Dan Vilenchik References 1. M. Alekhnovich and E. Ben-Sasson. Linear upper bounds for random walk on small density random 3-cnf. In Proc. 44th IEEE Symp. on Found. of Comp. Science, page 352, 2003. 2. N. Alon and N. Kahale. A spectral technique for coloring random 3-colorable graphs. SIAM J. on Comput., 26(6):1733 1748, 1997. 3. A. Braunstein, M. Mezard, and R. Zecchina. Survey propagation: an algorithm for satis ability. Random Structures and Algorithms, 27:201 226, 2005. 4. A. Z. Broder, A. M. Frieze, and E. Upfal. On the satis ability and maximum satis ability of random 3-cnf formulas. In Proc. 4th ACM-SIAM Symp. on Discrete Algorithms, pages 322 330, 1993. 5. O. Dubois, Y. Boufkhad, and J. Mandler. Typical random 3-sat formulae and the satis ability threshold. In Proc. 11th ACM-SIAM Symp. on Discrete Algorithms, pages 126 127, 2000. 6. U. Feige and R. Krauthgamer. Finding and certifying a large hidden clique in a semirandom graph. Random Structures and Algorithms, 16(2):195 208, 2000. 7. U. Feige and D. Vilenchik. A local search algorithm for 3SAT. Technical report, The Weizmann Institute of Science, 2004. 8. A. Flaxman. A spectral technique for random satis able 3CNF formulas. In Proc. 14th ACM-SIAM Symp. on Discrete Algorithms, pages 357 363, 2003. 9. E. Friedgut. Sharp thresholds of graph properties, and the k-sat problem. J. Amer. Math. Soc., 12(4):1017 1054, 1999. 10. A. M. Frieze and C. McDiarmid. Algorithmic theory of random graphs. Random Structures and Algorithms, 10(1-2):5 42, 1997. 11. T. G. Gallager. Low-density parity-check codes. IRE. Trans. Info. Theory, IT8:21 28, January 1962. 12. J. H astad. Some optimal inapproximability results. J. ACM, 48(4):798 859, 2001. 13. C. Hui and A. M. Frieze. Coloring bipartite hypergraphs. In Proceedings of the 5th International IPCO Conference on Integer Programming and Combinatorial Optimization, pages 345 358, London, UK, 1996. Springer-Verlag. 14. A. C. Kaporis, L. M. Kirousis, and E. G. Lalas. The probabilistic analysis of a greedy satis ability algorithm. In Proc. 10th Annual European Symposium on Algorithms, volume 2461 of Lecture Notes in Comput. Sci., pages 574 585. Springer, Berlin, 2002. 15. E. Koutsoupias and C. H. Papadimitriou. On the greedy algorithm for satis ability. Info. Process. Letters, 43(1):53 55, 1992. 16. F. R. Kschischang, B. J. Frey, and H. A. Loeliger. Factor graphs and the sumproduct algorithm. IEEE Transactions on Information Theory, 47(2):498 519, 2001. 17. M. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. Spielman. Analysis of low density parity check codes and improved designs using irregular graphs. In Proceedings of the 30th ACM Symposium on Theory of Computing, pages 249 258, 1998. 18. M. Luby, M. Mitzenmacher, M. A. Shokrollahi, and D. Spielman. E cient erasure correcting codes. IEEE Trans. Info. Theory, 47:569 584, February 2001. 19. J. Pearl. Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1988. 20. T. Richardson, A. Shokrollahi, and R. Urbanke. Design of capacity-approaching irregular low-density parity check codes. IEEE Trans. Info. Theory, 47:619 637, February 2001.

Find millions of documents here - Study Guides, Homework Solutions, Papers, Exam Answer Keys and more. Course Hero has millions of course related materials that will enable you to learn better, faster and get an A in all your courses.
Below is a small sample set of documents: