book - i Computational Complexity A Modern Approach Draft...

This preview shows page 1 out of 489 pages.

Unformatted text preview: i Computational Complexity: A Modern Approach Draft of a book: Dated January 2007 Comments welcome! Sanjeev Arora and Boaz Barak Princeton University [email protected] Not to be reproduced or distributed without the authors’ permission This is an Internet draft. Some chapters are more finished than others. References and attributions are very preliminary and we apologize in advance for any omissions (but hope you will nevertheless point them out to us). Please send us bugs, typos, missing references or general comments to [email protected] — Thank You!! DRAFT ii DRAFT About this book Computational complexity theory has developed rapidly in the past three decades. The list of surprising and fundamental results proved since 1990 alone could fill a book: these include new probabilistic definitions of classical complexity classes (IP = PSPACE and the PCP Theorems) and their implications for the field of approximation algorithms; Shor’s algorithm to factor integers using a quantum computer; an understanding of why current approaches to the famous P versus NP will not be successful; a theory of derandomization and pseudorandomness based upon computational hardness; and beautiful constructions of pseudorandom objects such as extractors and expanders. This book aims to describe such recent achievements of complexity theory in the context of the classical results. It is intended to both serve as a textbook as a reference for self-study. This means it must simultaneously cater to many audiences, and it is carefully designed with that goal. Throughout the book we explain the context in which a certain notion is useful, and why things are defined in a certain way. Examples and solved exercises accompany key definitions. We assume essentially no computational background and very minimal mathematical background, which we review in Appendix A. We have also provided a web site for this book at with related auxiliary material. This includes web chapters on automata and computability theory, detailed teaching plans for courses based on this book, a draft of all the book’s chapters, and links to other online resources covering related topics. The book is divided into three parts: Part I: Basic complexity classes. This volume provides a broad introduction to the field. Starting from the definition of Turing machines and the basic notions of computability theory, this volumes covers the basic time and space complexity classes, and also includes a few more modern topics such probabilistic algorithms, interactive proofs and cryptography. Part II: Lower bounds on concrete computational models. This part describes lower bounds on resources required to solve algorithmic tasks on concrete models such as circuits, decision trees, etc. Such models may seem at first sight very different from Turing machines, but looking deeper one finds interesting interconnections. Part III: Advanced topics. This part is largely devoted to developments since the late 1980s. It includes average case complexity, derandomization and pseudorandomness, the PCP theorem and hardness of approximation, proof complexity and quantum computing. Almost every chapter in the book can be read in isolation (though we recommend reading Chapters 1, 2 and 7 before reading later chapters). This is important because the book is aimed DRAFT Web draft 2007-01-08 21:59 iii Complexity Theory: A Modern Approach. © 2006 Sanjeev Arora and Boaz Barak. References and attributions are still incomplete. iv at many classes of readers: • Physicists, mathematicians, and other scientists. This group has become increasingly interested in computational complexity theory, especially because of high-profile results such as Shor’s algorithm and the recent deterministic test for primality. This intellectually sophisticated group will be able to quickly read through Part I. Progressing on to Parts II and III they can read individual chapters and find almost everything they need to understand current research. • Computer scientists (e.g., algorithms designers) who do not work in complexity theory per se. They may use the book for self-study or even to teach a graduate course or seminar. • All those —professors or students— who do research in complexity theory or plan to do so. They may already know Part I and use the book for Parts II and III, possibly in a seminar or reading course. The coverage of advanced topics there is detailed enough to allow this. This book can be used as a textbook for several types of courses. We will provide several teaching plans and material for such courses on the book’s web site. • Undergraduate Theory of Computation Course. Part I may be suitable for an undergraduate course that is an alternative to the more traditional Theory of Computation course currently taught in most computer science departments (and exemplified by Sipser’s excellent book with the same name [SIP96]). Such a course would have a greater emphasis on modern topics such as probabilistic algorithms and cryptography. We note that in contrast to Sipser’s book, the current book has a quite minimal coverage of computability and no coverage of automata theory, but we provide web-only chapters with more coverage of these topics on the book’s web site. The prerequisite mathematical background would be some comfort with mathematical proofs and elementary probability on finite sample spaces, topics that are covered in typical “discrete math”/“math for CS” courses currently offered in most CS departments. • Advanced undergraduate/beginning graduate introduction to complexity course. The book can be used as a text for an introductory complexity course aimed at undergraduate or non-theory graduate students (replacing Papadimitriou’s 1994 book [Pap94] that does not contain many recent results). Such a course would probably include many topics from Part I and then a sprinkling from Parts II and III, and assume some background in algorithms and/or the theory of computation. • Graduate Complexity course. The book can serve as a text for a graduate complexity course that prepares graduate students interested in theory to do research in complexity and related areas. Such a course can use parts of Part I to review basic material, and then move on to the advanced topics of Parts II and III. The book contains far more material than can be taught in one term, and we provide on our website several alternative outlines for such a course. We hope that this book conveys our excitement about this new field and the insights it provides in a host of older disciplines. DRAFT Web draft 2007-01-08 21:59 Contents About this book iii Introduction p0.1 (1) I p0.9 (9) Basic Complexity Classes 1 The computational model —and why it doesn’t matter 1.1 Encodings and Languages: Some conventions . . . . . . . . . . . 1.1.1 Representing objects as strings . . . . . . . . . . . . . . . 1.1.2 Decision problems / languages . . . . . . . . . . . . . . . 1.1.3 Big-Oh notation . . . . . . . . . . . . . . . . . . . . . . . 1.2 Modeling computation and efficiency . . . . . . . . . . . . . . . . 1.2.1 The Turing Machine . . . . . . . . . . . . . . . . . . . . . 1.2.2 Robustness of our definition. . . . . . . . . . . . . . . . . 1.2.3 The expressive power of Turing machines. . . . . . . . . . 1.3 Machines as strings and the universal Turing machines. . . . . . 1.3.1 The Universal Turing Machine . . . . . . . . . . . . . . . 1.4 Uncomputable functions. . . . . . . . . . . . . . . . . . . . . . . . 1.4.1 The Halting Problem . . . . . . . . . . . . . . . . . . . . . 1.5 Deterministic time and the class P. . . . . . . . . . . . . . . . . . 1.5.1 On the philosophical importance of P . . . . . . . . . . . 1.5.2 Criticisms of P and some efforts to address them . . . . . 1.5.3 Edmonds’ quote . . . . . . . . . . . . . . . . . . . . . . . Chapter notes and history . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.A Proof of Theorem 1.13: Universal Simulation in O(T log T )-time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 NP and NP completeness 2.1 The class NP . . . . . . . . . . . . . . . . . 2.1.1 Relation between NP and P . . . . 2.1.2 Non-deterministic Turing machines. 2.2 Reducibility and NP-completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DRAFT Web draft 2007-01-08 21:59 p1.1 (11) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p1.2 (12) p1.2 (12) p1.3 (13) p1.3 (13) p1.4 (14) p1.5 (15) p1.9 (19) p1.12 (22) p1.12 (22) p1.13 (23) p1.15 (25) p1.15 (25) p1.17 (27) p1.17 (27) p1.18 (28) p1.19 (29) p1.20 (30) p1.21 (31) p1.25 (35) p2.1 (39) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p2.2 (40) p2.3 (41) p2.4 (42) p2.5 (43) v Complexity Theory: A Modern Approach. © 2006 Sanjeev Arora and Boaz Barak. References and attributions are still incomplete. vi CONTENTS 2.3 The Cook-Levin Theorem: Computation is Local . . 2.3.1 Boolean formulae and the CNF form. . . . . 2.3.2 The Cook-Levin Theorem . . . . . . . . . . . 2.3.3 Warmup: Expressiveness of boolean formulae 2.3.4 Proof of Lemma 2.12 . . . . . . . . . . . . . . 2.3.5 Reducing SAT to 3SAT. . . . . . . . . . . . . 2.3.6 More thoughts on the Cook-Levin theorem . 2.4 The web of reductions . . . . . . . . . . . . . . . . . In praise of reductions . . . . . . . . . . . . . Coping with NP hardness. . . . . . . . . . . 2.5 Decision versus search . . . . . . . . . . . . . . . . . 2.6 coNP, EXP and NEXP . . . . . . . . . . . . . . . 2.6.1 coNP . . . . . . . . . . . . . . . . . . . . . . 2.6.2 EXP and NEXP . . . . . . . . . . . . . . . 2.7 More thoughts about P, NP, and all that . . . . . . 2.7.1 The philosophical importance of NP . . . . . 2.7.2 NP and mathematical proofs . . . . . . . . . 2.7.3 What if P = NP? . . . . . . . . . . . . . . . 2.7.4 What if NP = coNP? . . . . . . . . . . . . . Chapter notes and history . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Diagonalization 3.1 Time Hierarchy Theorem . . . . . . . . . . . . . . . . . . . 3.2 Space Hierarchy Theorem . . . . . . . . . . . . . . . . . . . 3.3 Nondeterministic Time Hierarchy Theorem . . . . . . . . . 3.4 Ladner’s Theorem: Existence of NP-intermediate problems. 3.5 Oracle machines and the limits of diagonalization? . . . . . Chapter notes and history . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p2.6 (44) p2.7 (45) p2.7 (45) p2.8 (46) p2.9 (47) p2.11 (49) p2.11 (49) p2.12 (50) p2.16 (54) p2.16 (54) p2.17 (55) p2.18 (56) p2.18 (56) p2.19 (57) p2.20 (58) p2.20 (58) p2.20 (58) p2.21 (59) p2.21 (59) p2.22 (60) p2.23 (61) p3.1 (65) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Space complexity 4.1 Configuration graphs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Some space complexity classes. . . . . . . . . . . . . . . . . . . . . . . . 4.3 PSPACE completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Savitch’s theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 The essence of PSPACE: optimum strategies for game-playing. 4.4 NL completeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Certificate definition of NL: read-once certificates . . . . . . . . 4.4.2 NL = coNL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter notes and history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DRAFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p3.2 (66) p3.2 (66) p3.3 (67) p3.4 (68) p3.6 (70) p3.8 (72) p3.9 (73) p4.1 (75) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p4.2 (76) p4.4 (78) p4.5 (79) p4.8 (82) p4.8 (82) p4.10 (84) p4.12 (86) p4.13 (87) p4.14 (88) p4.14 (88) Web draft 2007-01-08 21:59 CONTENTS vii 5 The Polynomial Hierarchy and Alternations 5.1 The classes Σp2 and Πp2 . . . . . . . . . . . . . . . 5.2 The polynomial hierarchy. . . . . . . . . . . . . . 5.2.1 Properties of the polynomial hierarchy. . . 5.2.2 Complete problems for levels of PH . . . 5.3 Alternating Turing machines . . . . . . . . . . . 5.3.1 Unlimited number of alternations? . . . . 5.4 Time versus alternations: time-space tradeoffs for 5.5 Defining the hierarchy via oracle machines. . . . Chapter notes and history . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . 6 Circuits 6.1 Boolean circuits . . . . . . . . . . . . . . . . . . 6.1.1 Turing machines that take advice . . . . 6.2 Karp-Lipton Theorem . . . . . . . . . . . . . . 6.3 Circuit lowerbounds . . . . . . . . . . . . . . . 6.4 Non-uniform hierarchy theorem . . . . . . . . . 6.5 Finer gradations among circuit classes . . . . . 6.5.1 Parallel computation and NC . . . . . . 6.5.2 P-completeness . . . . . . . . . . . . . . 6.6 Circuits of exponential size . . . . . . . . . . . 6.7 Circuit Satisfiability and an alternative proof of Chapter notes and history . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . p5.1 (91) . . . . . . . . . . . . . . . . . . . . . . . . SAT. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . DRAFT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p5.1 (91) p5.3 (93) p5.3 (93) p5.4 (94) p5.5 (95) p5.6 (96) p5.6 (96) p5.8 (98) p5.9 (99) p5.10 (100) p6.1 (101) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . the Cook-Levin . . . . . . . . . . . . . . . . . . 7 Randomized Computation 7.1 Probabilistic Turing machines . . . . . . . . . . . . . . . . . . . . 7.2 Some examples of PTMs . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Probabilistic Primality Testing . . . . . . . . . . . . . . . 7.2.2 Polynomial identity testing . . . . . . . . . . . . . . . . . 7.2.3 Testing for perfect matching in a bipartite graph. . . . . . 7.3 One-sided and zero-sided error: RP, coRP, ZPP . . . . . . . . 7.4 The robustness of our definitions . . . . . . . . . . . . . . . . . . 7.4.1 Role of precise constants, error reduction. . . . . . . . . . 7.4.2 Expected running time versus worst-case running time. . 7.4.3 Allowing more general random choices than a fair random 7.5 Randomness efficient error reduction. . . . . . . . . . . . . . . . . 7.6 BPP ⊆ P/poly . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7 BPP is in PH . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.8 State of our knowledge about BPP . . . . . . . . . . . . . . . . . Complete problems for BPP? . . . . . . . . . . . . . . . . Does BPTIME have a hierarchy theorem? . . . . . . . . Web draft 2007-01-08 21:59 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p6.1 (101) p6.5 (105) p6.6 (106) p6.7 (107) p6.8 (108) p6.8 (108) p6.9 (109) p6.10 (110) p6.11 (111) p6.12 (112) p6.13 (113) p6.13 (113) p7.1 (115) . . . . . . . . . . . . . . . . . . . . . . . . . . . coin. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p7.2 (116) p7.3 (117) p7.3 (117) p7.4 (118) p7.5 (119) p7.6 (120) p7.7 (121) p7.7 (121) p7.10 (124) p7.10 (124) p7.11 (125) p7.12 (126) p7.13 (127) p7.14 (128) p7.14 (128) p7.15 (129) viii CONTENTS 7.9 Randomized reductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.10 Randomized space-bounded computation . . . . . . . . . . . . . . . . . . Chapter notes and history . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.A Random walks and eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . 7.A.1 Distributions as vectors and the parameter λ(G). . . . . . . . . . . 7.A.2 Analysis of the randomized algorithm for undirected connectivity. 7.B Expander graphs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.B.1 The Algebraic Definition . . . . . . . . . . . . . . . . . . . . . . . . 7.B.2 Combinatorial expansion and existence of expanders. . . . . . . . . 7.B.3 Error reduction using expanders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Interactive proofs 8.1 Warmup: Interactive proofs with a deterministic verifier 8.2 The class IP . . . . . . . . . . . . . . . . . . . . . . . . 8.3 Proving that graphs are not isomorphic. . . . . . . . . . 8.4 Public coins and AM . . . . . . . . . . . . . . . . . . . 8.4.1 Set Lower Bound Protocol. . . . . . . . . . . . . Tool: Pairwise independent hash functions. . . . The lower-bound protocol. . . . . . . . . . . . . . 8.4.2 Some properties of IP and AM . . . . . . . . . . 8.4.3 Can GI be NP-complete? . . . . . . . . . . . . . 8.5 IP = PSPACE . . . . . . . . . . . . . . . . . . . . . . 8.5.1 Arithmetization . . . . . . . . . . . . . . . . . . . 8.5.2 Interactive protocol for #SATD . . . . . . . . . . Sumcheck protocol. . . . . . . . . . . . . . . . . . 8.5.3 Protocol for TQBF: proof of Theorem 8.17 . . . 8.6 The power of the prover . . . . . . . . . . . . . . . . . . 8.7 Program Checking . . . . . . . . . . . . . . . . . . . . . 8.7.1 Languages that have checkers . . . . . . . . . . . 8.8 Multiprover interactive proofs (MIP) . . . . . . . . . . Chapter notes and history . . . . . . . . . . . . . . . . . . . . Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.A Interactive proof for the Permanent . . . . . . . . . . . . 8.A.1 The protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...
View Full Document

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture