This preview shows page 1. Sign up to view the full content.
Unformatted text preview: UMass Lowell Computer Science 91.503 Analysis of Algorithms
Prof. Karen Daniels
Spring, 2011 i Lecture 6
Tuesday, 3/22/11 y, NPNPCompleteness Overview Graham Videotape Review: Reduction to Establish Lower Bounds Basics Practicing Some Reductions g How to Treat NP Hard or Complete Problems NPSimpsons Video p Where the Really Hard Problems Are "Order Parameter" Critical Value Graph Coloring Demo
source: clip art from web Resources Recommended: Computers & Intractability by Garey & Johnson W.H.Freeman 1990 ISBN 0716710455. 0716710455 For further study... Computational Complexity: A Conceptual Perspective by Oded Goldreich 2008 ISBN 9780521884730 978 52188473 Motivation Why do we care?? When faced with a problem in p p practice, need to , know whether or not one can expect produce a "fast" algorithm (i.e. O(nk)) to solve it O(n "Intractable" "I t t bl " problems occur frequently in bl f tl i practice Surface differences between tractable and intractable problems are often hard to see e.g. MST solvable in polynomial time but g p y Dominating Set intractable Dominating Set source: Garey & Johnson Industrial Motivation
Supporting Apparel Manufacturing
Maximum a u Rectangle Geometric Restriction DistanceDistanceBased Subdivision Limited Gaps Containment NPNPcomplete Maximal Cover Minimal Enclosure Ordered Containment TwoTwoPhase Layout Lattice Packing ColumnColumnBased Layout Covering Research Example: Translational 2D Covering
[CCCG'01, CCCG'03 conference papers] Input: STATIC Covering polygons Q = {Q1, Q2 , ... , Qm} {Q Target polygons (or pointsets) P = {P1, P2 , ... , Pn} g p yg ( point ) p {P Translations = { 1, 2 , ... , m} such that
P2 P1 Translational 2D Polygon Covering
Q3 Q1 Q2 Q3 Q1 Sample P and Q Translated Q Covers P Output: P
P2 P1
Q2
1 j m U j ( Qj ) NPNPCOMPLETE Covering Problem Taxonomy
covering nongeometric covering geometric covering 2D translational covering P: finite point sets Q: identical Q: convex
BOXCOVER VERTEXCOVER, SETCOVER, EDGECOVER, VLSI logic minimization, facility location P: shapes covering
P 1 j m U j (Q j) Q: nonconvex 1D interval covered by annuli decomposition:
P =
1 j m . . . . . . . . . . j (Q j ) U j (Q j) . . . i (Q i ) j (Q j ) Thin coverings of the plane with congruent convex shapes Translational covering of a convex set by a sequence of convex shapes Translational covering of nonconvex set with nonidentical covering shapes partition cannot : overlap cover
may overlap i (Q i ) Polynomialtime algorithms for triangulation and some tilings NPhard/complete polygon problems polynomialtime results for restricted orthogonal polygon covering and th l l i d horizontally convex polygons approximation algorithms for boundary, corner covers of orthogonal polygons Reduction to Establish d i bli h Lower Bound
Tightening a Lower Bound for MaximumMaximumArea AxisParallel AxisRectangle Problem from (n) to (nlgn) A Common (sub) Problem
Find a Good Approximation Outer Inner What's the Problem? Given a 2D polygon that: does not intersect itself may have holes has n vertices Find the LargestArea AxisParallel Rectangle argestAxisHow "hard" is the problem? hard How "fast" can we find an answer? Approach Go G around l d lower b bound brick wall by: d b i k ll b examining strong lower bounds for some similar problems bl transforming a similar problem to the rectangle problem
[this process is similar to what we do when we prove problems NPNPworstcomplete] worstcase bounds on problem 1 n n log2 n n2 n5 2n Lower Bound of (n log n) by Transforming a (seemingly unrelated) Problem
MAXMAXGAP instance: given n real numbers { x1, x2, ... xn } find the instance: maximum difference between 2 consecutive numbers in the sorted list. O(n) time transformation Rectangle specialized area is a polygon solution to the MAXMAXGAP x2 x4 instance x3 x1 Rectangle algorithm must take at least as much time as MAXGAP MAXGAP. MAXMAXGAP is known to be in (n log n). Rectangle algorithm must t k ( l n) ti R t l l ith t take (n log ) time for specialized polygons.
[Transforming yet another different problem yields bound for unspecialized polygons.] NPCompleteness C l Chapter 34 Basic Concepts
Polynomial Time & Problem Formalization l i l i bl li i PolynomialPolynomialTime Verification NPNPCompleteness & Reducibility NPNPCompleteness Proofs Expanding List of Hard Problems via Reduction Polynomial Time & Problem Formalization Formalize notions of abstract and concrete F li i f b d decision problems Encoding a problem Define class of polynomialtime solvable p y polynomialdecision problems Define complexity class Express relationship between decision problems and algorithms that solve them using framework of formal language theory Polynomial Time & Problem Formalization ( ti d) (continued) Solvable in Polynomial Time y i.e. (nk) for some positive constant k Tractable: k is typically small (e.g. 1, 2, 3) (e g 1 2 Typically holds across sequential computation models Closure properties (chaining/composition) Binary relation on set of problem instances & solutions Bi l i f bl i l i impose bound on optimization problem to recast e.g. Dominating Set Abstract Problem: Decision Problem: yes/no answer Problem: Dominating Set
parameter K introduced to recast optimization problem as a decision problem source: Garey & Johnson Polynomial Time & Problem Formalization ( ti d) (continued) Encoding of set S of abstract objects is mapping e from S to set of binary strings Concrete Problem: instance set is set of binary strings Problem: Concrete Problem is PolynomialTime Solvable if there exists an Polynomialalgorithm to solve it in time (nk) for some positive constant k Complexity Class P = set of polynomialtime solvable polynomialconcrete decision problems Given abstract decision problem Q that maps instance set I to {0,1} e : I {0,1} can induce a concrete decision problem e(Q) If solution to abstractproblem instance i I is Q(i) {0,1} abstract then solution to concreteproblem instance e : I {0,1} concrete is also Q(i) Q(i Polynomial Time & Problem Formalization ( ti d) (continued)
Goal: Goal: Extend definition of polynomialtime solvability from polynomialconcrete to abstract using encodings as bridge polynomialpolynomialtime algorithm A that, given any input x {0,1}, produces f(x) as output. For set I of problem instances encodings are polynomially related if there instances, exist 2 polynomialtime computable functions f12, f21 such that polynomial polynomialf : {0,1} {0,1} is polynomialtime computable if there exists a i I , f12 (e1 (i )) = e2 (i ) and f 21 (e2 (i )) = e1 (i ) If 2 encodings e1, e2 of abstract problem are polynomially related, then di f bt t bl l i ll l t d th polynomialpolynomialtime solvability of problem is encodingindependent: encoding Lemma 34.1: Let Q be an abstract decision problem on an instance set I 34.1:
and let e1, e2 be polynomially related encodings on I. Then, e1 (Q ) P e2 (Q ) P Polynomial Time & Problem Formalization ( ti d) (continued) Formal Language Framework Alphabet is finite set of symbols: = {0,1} * for = {0,1} Language L over is any set of strings consisting of symbols from Set of instances for decision problem Q is for = {0,1} View Q as language L over = {0,1} L = {x : Q( x) = 1} Express relation between decision problems and algorithms Algorithm Al i h A accepts string x {0,1} if, given input x, output A(x) i 1 i if i i A( ) is Language accepted by A is set of strings A accepts L = {x {0,1} : A( x) = 1} Polynomial Time & Problem Formalization ( ti d) (continued) Formal Language Framework (continued) Language L is decided by algorithm A if every binary string in L i accepted by A and every binary string not in L i rejected is db d bi i i is j d by A Language L is accepted in polynomial time by algorithm A if it is accepted by A and if exists constant k such that, for any lengthlengthn string x L, A accepts x in time O(nk) O(n Language L i d id d i polynomial time by algorithm A if L is decided in l i l i b l ih exists constant k such that, for any lengthn string x {0,1} * , lengthA correctly decides in time O(nk) if x L y O(n ( Polynomial Time & Problem Formalization ( ti d) (continued) Formal Language Framework (continued) Complexity Class is a set of languages whose membership is determined by a complexity measure (i.e. running time) of d i db l i (i i i ) f algorithm that determines whether a given string belongs to a language. g g
We already defined: Complexity Class P = set of polynomialtime polynomialsolvable concrete decision problems Language Framework gives other definitions: g g g P = { L {0,1} * : there exists an algorithm A that decides L in polynomial time} = { L : L is accepted by a polynomialtime algorithm} polynomial PolynomialTime Verification Examine algorithms that "verify" E i l ih h " if " membership in languages Define class of decision problems whose solutions can be verified in polynomial time PolynomialTime Verification
(continued) Verification Algorithms algorithm A verifies language L if for any string x L certificate y that A can use to prove that xL also, for any string x L certificate y that A can use / to prove that x L language verified by verification algorithm A is L = {x {0,1}* : y {0,1} * such that A( x, y ) = 1}
e.g. Dominating Set
certificate PolynomialTime Verification
(continued) The Complexity Class NP languages that can be verified by a polynomialtime polynomialalgorithm A L = {x {0,1}* : certificate y with  y = O( x c ) ifi ih such that A( x, y ) = 1} A verifies l ifi language L i polynomial time in l i l ti e.g. L P L NP why? The Complexity Class coNP co set of languages L such that Recall that L NP L = * L PolynomialTime Verification
(continued) P
= NP coNP NP = NP coNP NP coNP NP P NP coNP P
34.3 NP coNP source: 91.503 textbook Cormen et al. NPCompleteness & Reducibility PolynomialPolynomialTime Reducibility Define complexity class of NPcomplete NPproblems Relationships among complexity classes comple it Circuit satisfiability as an NPcomplete NPproblem bl NPCompleteness & Reducibility
(continued) Q can be reduced to Q' if any instance of Q can be "easily Q easily rephrased" as an instance of Q', the solution to which provides a Q', solution to instance of Q. L1 is polynomialtime reducible to L2 : L1 P L2 polynomialif exists polynomialtime computable function polynomial{0,1} * f : {0,1}* {0,1} * such that x {0,1} * }
{0,1} * x L1 f ( x) L2
34.4 source: 91.503 textbook Cormen et al. NPCompleteness & Reducibility
(continued)
If L1 , L2 {0,1} * are languages such that L1 P L2 Then L2 P L1 P
YES , f ( x) L2
f (x)
NO, f ( x) L2 /
34.3 YES , x L1 NO, x L1 / 34.5 source: 91.503 textbook Cormen et al. NPCompleteness & Reducibility
(continued)
Language L {0,1} * is NPcomplete if g g NPp
without this condition, language is only NPhard NPpopular theoretical computer science view L NP and L' P L, L' NP
set of all NPNPcomplete languages NP Theorem 34.4: 34.4:
 If any NPcomplete problem is polynomialtime solvable, then P=NP. NPpolynomial Equivalently, if any problem in NP is not polynomialtime solvable, then no polynomialNPNPcomplete problem is polynomialtime solvable. (contrapositive) polynomialcontrapositive)
source: 91.503 textbook Cormen et al. Expanding List of Hard Problems via Reduction i R d ti
Relationships among some NPcomplete problems NPCircuitCircuitSAT
First NPcomplete problem: NPBoolean Formula Satisfiability (Cook '71) source: 91 503 textbook Cormen et al. 91.503 al source: Garey & Johnson Need to show that starting problem is in NP & is at least as hard as any problem in NP. NPCompleteness & Reducibility
(continued): ( ti d) Circuit Satisfiability
circuit inputs single circuit output Satisfying Assignment: Assignment: truth assignment inducing output = 1 Language Def: CIRCUITSAT = Def: CIRCUIT{<C>:C is a satisfiable boolean combinational circuit.} circuit } Boolean Combinational Gate Types: AND, NOT, OR GOAL: GOAL: Show Circuit Satisfiability is NPcomplete NP 34.8 source: 91.503 textbook Cormen et al. NPCompleteness & Reducibility
(continued) ( ti d) Circuit Satisfiability CIRCUITCIRCUITSAT is in NP: Construct 2input polynomialtime algorithm A that verifies 2polynomialCIRCUITCIRCUITSAT Input 1: encoding of circuit C Input 2: certificate = assignment of boolean values to all wires of C For each logic gate A check if certificate value for gate s output wire gate's correctly computes gate's function based on input wire values If output of entire circuit is 1, algorithm outputs 1 Otherwise, algorithm outputs 0 Algorithm A : For satisfiable circuit, there is a (polynomiallength) (polynomialcertificate that causes A to output 1 For unsatisfiable circuit, no certificate can cause A to output 1 , p Algorithm A runs in polynomial time
source: 91.503 textbook Cormen et al. NPCompleteness & Reducibility
(continued) ( ti d) Circuit Satisfiability
certificate CIRCUITCIRCUITSAT is NPhard: NPL represents any language in NP Create polynomialtime algorithm polynomialF computing reduction function f that maps every binary string x to a circuit C=f(x) such that x L C CIRCUIT  SAT
Must i t l ith M t exist algorithm A verifying L if i in polynomial time. (A from NP) (A F uses A to compute f. Represent computation of A as sequence of configurations F constructs single circuit that p g computes all configurations produced by initial configuration F runs in time polynomial in n=x x T(n) copies Circuit is tailormade for instance string x. tailor Certificate y's length is in O(nk) source: 91.503 textbook Cormen et al. NPCompleteness Proofs (continued) Proving a Language NPComplete NPHW: Why not just pick any language in NP? Proving a Language NPHard NP all steps except (1)
source: 91.503 textbook Cormen et al. NPCompleteness Proofs (continued) Reducing Boolean Circuit Satisfiability to Boolean Formula Satisfiability Boolean Formula Satisfiability: Instance of language SAT Satisfiability: is a boolean formula consisting of: n boolean variables: x1, x2, ... , xn m boolean connectives: boolean function with 1 or 2 inputs and 1 output p e.g. AND, OR, NOT, implication, iff (recall logical equivalent of implication) parentheses truth, ti f i t th satisfying assignments notions apply i t ti l SAT = {< >: is a satisfiable boolean formula}
Example:
Is there a satisfying assignment for this? source: 91.503 textbook Cormen et al. NPCompleteness Proofs (continued) Show: Show: SAT is NPComplete NP SAT is in NP via argument similar to circuit case Reduce Boolean Circuit S ti fi bilit t Boolean R d B l Ci it Satisfiability to B l Formula Satisfiability CIRCUIT  SAT P SAT Careful! Naive approach might have shared subformulas and cause formula size to grow exponentially! source: 91.503 textbook Cormen et al. NPCompleteness Proofs (continued) Show: Show: SAT is NPComplete (continued) NP C Reduce Boolean Circuit Satisfiability to Boolean Formula Satisfiability (continued) CIRCUIT  SAT P SAT Approach: Approach: For each wire xi in circuit C, formula has variable xi. Express gate operation. = x10 ( x4 x3 ) ( x5 ( x1 x2 )) ( x6 x4 ) ( x7 ( x1 x2 x4 )) ( x8 ( x5 x6 )) ( x9 ( x6 x7 )) ( x10 ( x7 x8 x9 ))
source: 91.503 textbook Cormen et al. 34.10 NPCompleteness Proofs (continued) Reducing Formula Satisfiability to 3CNFg y 3CNFSatisfiability Boolean Formula Satisfiability: Instance of language Satisfiability: SAT is a boolean formula CNF = conjunctive normal form conjunction: conjunction: AND of clauses clause: clause: OR of literal(s) 3CNF: each clause has exactly 3 distinct literals Show: 3Show: 3CNF is NPComplete NP SAT P 3CNF
source: 91.503 textbook Cormen et al. NPCompleteness Proofs (continued) Show: 3Show: 3CNF is NPComplete (continued) SAT NP P 3CNF 3CNF is in NP via argument similar to circuit case Reduce Formula Satisfiability to 3CNFSatisfiability 3CNF 3 steps progressively transforming formula closer to 3CNF 1) Similar to CIRCUIT  SAT reduction P SAT Binary "ParseTree" y 34.11 source: 91.503 textbook Cormen et al. NPCompleteness Proofs (continued) Show: 3Show: 3CNF is NPComplete (continued)) SAT NPp ( P 3CNF 3CNF is in NP via argument similar to circuit case Reduce Formula Satisfiability to 3CNFSatisfiability 3CNF 3 steps progressively transforming formula closer to 3CNF 2) Convert each clause i' into conjunctive normal form construct truth table for i' using entries for 0, build DNF formula for i ' i i f 0 b ild f l f convert DNF into CNF formula i" by applying DeMorgan's laws each clause has at most 3 literals = (( x1 x 2 ) (( x1 x 3 ) x 4 )) x 2
truth table for 1 * * * *
* use to build DNF source: 91.503 textbook Cormen et al. NPCompleteness Proofs (continued) Show: 3Show: 3CNF is NPComplete (continued)) SAT NPp ( P 3CNF 3CNF is in NP via argument similar to circuit case Reduce Formula Satisfiability to 3CNFSatisfiability 3CNF 3 steps progressively transforming formula closer to 3CNF 3) Convert each clause i" so it has exactly 3 distinct literals add "padding variables" create i"' Size of resulting formula is polynomial in length of original formula and reduction takes only polynomial time and 3CNF 3formula o ua ' ' ' satisfiable satisfiable source: 91.503 textbook Cormen et al. Expanding List of Hard Problems via Reduction i R d ti
Relationships among some NPcomplete problems NPCircuitCircuitSAT
First NPcomplete problem: NPBoolean Formula Satisfiability (Cook '71) 71) source: 91.503 textbook Cormen et al. source: Garey & Johnson Clique
Clique of an undirected graph Cli f di t d h G=(V,E) is a complete subgraph of G.
clique of size 3 A C D E B F source: Garey & Johnson source: Cormen et al. CLIQUE= {<G,k>: graph G contains a clique of size k} {<G Reducing 3CNFSAT to Clique
source: 91.503 textbook Cormen et al. Construct graph G such that is satisfiable iff G has clique of size k x1
x
1 x 3 x1 x2 x2 34.14 For each clause C r = ( l1 l 2 l 3 ) place triple of vertices v1r , v2r , v3r into V.
r r r Put edge for vir , vjs if corresponding literals consistent and r not= s (different triples). Vertex Cover
Note difference between VERTEXCOVER and DOMINATINGSET VERTEXDOMINATING A B C D E F Vertex Cover of an undirected graph G=(V,E) is a subset vertex cover of size 2 V ' V such that if (u , v) E , then u V ' or v V ' or both source: Garey & Johnson source: Cormen et al. VERTEXVERTEXCOVER= {<G,k>: graph G has a vertex cover of size k} {<G Reducing Clique to VertexCover
Construct complement G =(V, E ) of graph G such that G has clique of size k iff graph G has a vertex cover of size Vk Vk. G: z w y
34.15 G: u v x G has clique V ' V with  V ' = k V  V ' is vertex cover in G G has vertex coverV ' ' V with  V ' ' = V  k V  V ' ' is clique in G
source: 91.503 textbook Cormen et al. Dominating Set Reduction from Vertex Cover: Board Work source: Garey & Johnson Hamiltonian Cycle 34.2 Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle that contains each vertex in V.
NPNPComplete: see textbook's reduction from VertexCover
source: 91.503 textbook Cormen et al. How to Treat NP Hard NPHard or Complete Problems Some Techniques for Treating NPComplete NP Complete Problems Approximation Algorithms (Chapter 35) Heuristic Upper or Lower Bounds Greedy, Greedy, Simulated Annealing, Genetic "Alg", AI "Alg", Linear Programming for part of problem Integer Programming Quadratic Programming... Gradient Descent, Local Search, Pruning, Subdivision Mathematical Programming M h i lP i Search Space Exploration: Randomization, Randomization, Derandomization Leverage/Impose Problem Structure Leverage Similar Problems "Where the Hard Problems Are" Graph coloring reduction heuristics handout Graph coloring demo ...
View
Full
Document
This note was uploaded on 02/13/2012 for the course CS 91.503 taught by Professor Staff during the Spring '11 term at UMass Lowell.
 Spring '11
 Staff
 Algorithms

Click to edit the document details