503_lecture7_S11

503_lecture7_S11 - UMass Lowell Computer Science 91.503...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Spring, 2011 i Lecture 7 Tuesday, 4/5/11 y, Approximation Algorithms 1 Approximation Algorithms A i i Al i h Chapter 35 (except Section 35.4) 2 Motivation: Some Techniques for Treating NP-Complete Problems NP Complete Approximation Algorithms Heuristic Upper or Lower Bounds Greedy, Simulated Annealing, Genetic "Alg", AI "Alg", Linear Programming for part of problem Integer Programming Quadratic, Convex Programming... Gradient Descent, Local Search, Pruning, Subdivision Mathematical Programming M h i lP i Search Space Exploration: Randomization, Derandomization Leverage/Impose Problem Structure Leverage Similar Problems 3 Basic Concepts Definitions fi i i 4 Definitions Approximation Algorithm Algorithm has approximation ratio (n) if: C C * max , ( n) C* C min max produces "near-optimal" solution "near- Approximation Scheme C = cost of algorithm's solution f l ih ' l i C* = cost of optimal solution n = number of inputs = size of instance (1+ (1+)-approximation algorithm for fixed > 0 is an input time is polynomial in n time is also polynomial in (1/) (1/ constantconstant-factor decrease in causes only constant-factor running-time constantrunning5 increase source: 91.503 textbook Cormen et al. PolynomialPolynomial-Time Approximation Scheme l l h Fully Polynomial-Time Approximation Scheme y Polynomialy pp Resources beyond textbook... "Approximation Algorithms for NP-Hard Problems", editor NPProblems", Dorit Hochbaum, PWS Publishing, Co., 1997. Hochbaum, "Approximation Algorithms" by Vazirani, 2001. Algorithms" Vazirani, "Computers and Intractability" by Garey & Johnson, 1979. Intractability" UMass Lowell course taught by Prof. Jie Wang. 6 Overview V VERTEX-COV VERTEX-COVER Polynomial-time 2-approximation algorithm Polynomial2TSP TSP with triangle inequality Polynomial-time 2-approximation algorithm Polynomial2 TSP without triangle inequality Negative result on polynomial-time (n)-approximation algorithm polynomial(n)MAX-3-C MAX- CNF S i fi bili A Satisfiability Randomized (n)-approximation algorithm (n)SETSET-COVER polynomial-time (n)-approximation algorithm polynomial(n) (n) is a logarithmic function of set size SUBSETSUBSET-SUM Fully polynomial-time approximation scheme polynomial7 Vertex Cover Polynomial- i 2Polynomial-Time 2-Approximation Algorithm l i l i i l ih 8 Vertex Cover A B C D E F vertex cover of size 2 Vertex Cover of an undirected graph G=(V,E) is a subset V ' V such that if (u , v) E , then u V ' or v V ' or both NPNP-Complete (via reduction from CLIQUE in Ch. 34 of Cormen et al.) al.) 9 source: Garey & Johnson Vertex Cover edge (b,c) chosen in line 4 b c d d a e f g a e f g d a g a c (vertex cover) a a f g 10 source: 91.503 textbook Cormen et al. Vertex Cover Theorem: APPROX-VERTEXTheorem: APPROX-VERTEX-COVER is a polynomialpolynomial-time 2-approximation algorithm. 2algorithm. Proof: Let Proof: L t C * b an optimal cover be ti l C be cover from APPROX - VERTEX - COVER A edges chosen by APPROX - VERTEX - COVER g y Observe: Observe: no 2 edges of A share any vertices in C due to removal of incident edges Any vertex cover must include >= 1 endpoint of each edge in A > APPROX-VERTEXAPPROX-VERTEX-COVER adds both endpoints of each edge of A C* A C = 2| A| 11 C = 2 A 2C* transitivity C 2C* C /C* 2 Algorithm runs in time polynomial in n. Traveling Salesman TSP with triangle inequality ih i l i li PolynomialPolynomial-time 2-approximation algorithm 2- TSP without triangle inequality Negative result on polynomial-time (n)-approximation polynomialalgorithm l ith 12 Hamiltonian Cycle Hamiltonian Schlegl diagram of dodecahedron: 12 sides (one of 5 3D Platonic solids) lid ) Not Hamiltonian 34.2 Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle th t contains each vertex i V i l l that t i h t in V. NPNP-Complete (via reduction from 13 VERTEXVERTEX-COVER in Ch. 34 of Cormen et al.) al.) source: 91.503 textbook Cormen et al. Traveling Salesman Problem (TSP) TSP Tour of a complete, complete, undirected, weighted graph G=(V,E) is a Hamiltonian Cycle with a designated starting/ending vertex. {< G, c, k >: u v x w TSP Decision Problem: Cost function provides edge weights. Note these are integers. Why? c : V V Z k Z G has TSP - tour of cost k} NP-Complete (via reduction from NP-C l t HAMHAM-CYCLE in Ch. 34 of Cormen et al.) al.) 14 source: 91.503 textbook Cormen et al. Minimum Spanning Tree: Greedy Algorithms Time: Time: O(|E|lg|E|) given fast FINDFIND-SET, UNION Invariant: Minimum weight spanning forest Produces minimum weight tree of edges that includes every vertex. A Becomes single tree at end 3 1 2 4 G 5 E 6 8 2 7 F 4 6 B Time: Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue 1 Invariant: Minimum weight tree Spans all vertices at end D C for Undirected, Connected, Weighted Graph G=(V,E) G (V E) 15 source: 91.503 textbook Cormen et al. TSP with Triangle Inequality Why Prim? u 5 3 5 x 4 w 3 v Cost Function Satisfies "full walk" = abcbhbadefegeda Triangle Inequality u , v, w V c(u , w) c(u , v) + c(v, w) final approximate tour source: 91.503 textbook Cormen et al. (removes vertex duplication) optimal tour (not necessarily found by 16 approximation algorithm) TSP with Triangle Inequality Theorem: APPROX-TSPTheorem: APPROX-TSP-TOUR is a polynomialpolynomial-time 2-approximation algorithm 2for TSP with triangle inequality. inequality. Proof: Proof: Algorithm runs in time polynomial in n = |V |. |V Let H * be an optimal tour and T be a MST c (T ) c( H *) ) c(W ) = 2c(T ) ( (since deleting 1 edge from a tour creates a spanning tree) g g p g ) (lists vertices when they are first visited and when returned to after visiting subtree) Let W be a full walk of T (since full walk traverses each edge of T twice) c(W ) 2c( H *) c( H ) c(W ) Now make W into a tour H using triangle inequality. (New inequality holds since H is formed by deleting duplicate vertices from W.) c( H ) 2c( H *) 17 TSP without Triangle Inequality Theorem: Theorem: If P NP , then for any constant 1 polynomialpolynomial-time approximation algorithm with ratio for TSP without triangle inequality. inequality. Proof: (b contradiction) Suppose there is one --- call it A Proof: (by P f di i ) Showing how to use A to solve NP-complete Hamiltonian Cycle problem NPcontradiction! Convert instance G of Hamiltonian Cycle into instance of TSP (in polynomial time): E ' = {(u , v) : u , v V , u v} (u , v) E 1 c(u , v) = | V | +1 otherwise (G'=(V,E') is complete graph on V) (assign integer cost to each edge in E') Cost of that tour of G ( | V | +1) + (| V | -1) = | V | + | V |> | V | G' Can use A on (G',c)! A finds tour of cost at most (length of optimal tour of G') G has Hamiltonian Cycle A finds tour of cost at most |V| cost = |V| 18 G does not have Hamiltonian Cycle A finds tour of cost > |V| If P NP , solving NP-complete hamiltonian cycle in polynomial time is a contradiction! NP- For F TSP problem (G',c): bl (G' ) (G',c) contains tour of cost |V| G has Hamiltonian Cycle Tour of G' must use some edge not in E G does not have Hamiltonian Cycle MAX-3-CNF Satisfiability 3-CNF Satisfiability Background i fi bili k d Randomized Algorithms Randomized MAX-3-CNF SAT Approximation Algorithm MAX- 19 MAX-3-CNF Satisfiability Background on Boolean Formula Satisfiability Boolean Formula Satisfiability: Instance of language SAT is Satisfiability: a boolean formula consisting of: n boolean variables: x1, x2, ... , xn m boolean connectives: boolean function with 1 or 2 inputs and 1 output e.g. AND, OR, NOT, implication, iff parentheses truth, satisfying assignments notions apply SAT = {< >: is a satisfiable boolean formula} NPNP-Complete (via reduction from CIRCUITCIRCUIT-SAT in Ch. 34 of Cormen et al.) al.) 20 source: 91.503 textbook Cormen et al. MAX-3-CNF Satisfiability (continued) Background on 3-CNF-Satisfiability g 3-CNFy Instance of language SAT is a boolean formula consisting of: literal: variable or its negation l l i bl i i CNF = conjunctive normal form conjunction: conjunction: AND of clauses clause: clause: OR of literal(s) 3-CNF: each clause has exactly 3 distinct literals MAXMAX-3-CNF Satisfiability: optimization version of 3-CNF-SAT Satisfiability: 3-CNF- Maximization: satisfy as many clauses as possible - Input Restrictions: - exactly 3 lit l / l tl literals/clause - no clause contains both variable and its negation (this assumption can be removed) 21 NPNP-Complete (via reduction from SAT in Ch. 34 of Cormen et al.) al.) source: 91.503 textbook Cormen et al. Definition Randomized Algorithm has approximation ratio (n) if for expected cost C of solution if, produced by randomized algorithm: C C * max , ( n) C* C min max size of input = n 22 source: 91.503 textbook Cormen et al. Randomized Approximation Algorithm for MAX 3 CNF SAT MAX-3-CNF Theorem: Theorem: Given an instance of MAX-3-CNF satisfiability with n MAXvariables x1, x2,..., xn with m clauses, the randomized algorithm th t i bl ith l th d i d l ith that independently sets each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7-approximation algorithm. C * 8 8/7C 7 Proof: independen set each varia to 0 or 1 with = probability = 1/2 Proof: P f tly ble indicator random variable = Yi = I {event that clause i is satisfied} Pr{clause i is not satisfied} = (1/2)3 = 1 / 8 Pr{clause i is satisfied} = 1 - (1 / 8) = 7 / 8 Lemma : Given a sample space S and an event A in the sample space S , Let X A = I{A}. Then E[X A ] = Pr {A}. { } [ { } By Lemma : E[Yi ] = 7 / 8 LetY = Y1 + Y2 +L+ Ym m 7 7m m m E[Y ] = E Yi = E [Yi ] = = = expected cost C 8 8 i =1 i =1 i =1 number of satisfied clauses m C* m source: 91.503 textbook Cormen et al. C* m 8 = = ( n) 23 C 7m / 8 7 Set Cover Greedy Approximation Algorithm d i i l ih polynomialpolynomial-time (n)-approximation algorithm (n) is a logarithmic function of set size i l ith i f ti f t i 24 Set Cover Problem Instance ( X , F ) : finite set X (e.g. of points) family F of subsets of X X= U S SF Problem Find P bl : Fi d a minimum - sized subset C F i i i d b whose members cover all of X: X = U S SC NPNP-Complete (via reduction from VERTEXVERTEXCOVER as noted in Ch. 35 of Cormen et al.) al.) 25 source: 91.503 textbook Cormen et al. Greedy Set Covering Algorithm Greedy: Greedy: select set that covers the most uncovered elements 26 source: 91.503 textbook Cormen et al. Set Cover Theorem: GREEDY-SETTheorem: GREEDY-SET-COVER is a polynomialpolynomial-time (n)-approximation algorithm for ( n ) = H (max{| S |: S F }) d Proof: Proof: P f 1 d th harmonic number H d = i i =1 = H (d ) H(0)=0 Algorithm runs in time polynomial in n. S i = i th subset selected (notational caveat) by algorithm selecting S i costs 1 cx = cost of element x X cx = 1 paid only when x is covered for the first time assume x is covered for the first time by Si (spread cost evenly across all elements covered for first time by Si ) Number of elements covered for first time by Si 27 S i - ( S 1 S 2 L S i -1 ) elements already covered by first i-1 chosen sets i- Set Cover (proof continued) Theorem: GREEDY-SETTheorem: GREEDY-SET-COVER is a polynomialpolynomial-time (n)-approximation algorithm for ( n) = H (max{| S |: S F }) Proof: (continued) Proof: P f Let C * be an optimal cover C be cover from GREEDY - SET - COVER Cost assigned to optimal cover: C = x X c x 1 unit is charged at each stage of algorithm * S c x S C x Each x is in >= 1 S in C* SC* xS cx xcx X C cx SC* xS 28 Set Cover (proof continued) Theorem: GREEDY-SETTheorem: GREEDY-SET-COVER is a polynomialpolynomial-time (n)-approximation algorithm (n)for ( n) = H (max{| S |: S F }) Proof: (continued) Proof: P f How does this relate to harmonic numbers?? We'll show that: d th harmonic number H d = H(S ) d i =1 1 = H (d ) i c x S x for any set S F C And then using conclude that: which will finish the proof cx SC* xS C S C * H(S ) C * H (max{ S : S F }) 29 Set Cover (proof continued) Proof of: c x S x H(S ) for any set S F For some set S: Number of elements of S remaining uncovered after S1...Si selected u i =| S - ( S1 S 2 L S i ) | u 0 =| S | c x S x = k k i =1 1 ( u i -1 - u i ) S i - ( S 1 S 2 L S i -1 ) k = least index f for which uk=0. c x S x i =1 1 ( u i -1 - u i ) u i -1 Since, due to greedy | S i - ( S 1 S 2 L S i -1 ) | nature of algorithm: | S - ( S 1 S 2 L S i -1 ) |= u i -1 = = i =1 k k u i -1 1 j = u i + 1 u i -1 k i =1 1 +1 j j =ui u i -1 k i =1 (since j <= ui-1) ui u i -1 1 1 - j =1 j j =1 j = (H ( u i =1 i -1 ) - H (u i ) ) = H (u 0 ) - H (u k ) = H (u 0 ) - H ( 0 ) = H (u 0 ) = H ( S ) 30 telescoping sum Subset-Sum Exponential- i Exponential-Time Exact Algorithm i l l ih Fully Polynomial-Time Approximation Scheme Polynomial- 31 Subset-Sum Problem Instance ( S , t ) : Decision Problem: S = {x1,x2 ,...,xn } S ' S : s = t ? sS ' positive integers Optimization Problem seeks subset with largest sum <= t NPNP-Complete (via reduction from 3-CNF-SAT in Ch. 34 of Cormen et al.) CNFal.) 32 source: 91.503 textbook Cormen et al. Exponential-Time Exact Algorithm Pi = set of values obtainable by selecting subset of {x1,x2 ,...,xi } and summing elements. Identity: Pi = Pi -1 ( Pi -1 + xi ) Li is sorted list of every element of Pi <= t Li = list of sums of subsets of {x1,x2 ,...,xi } t MERGEMERGE-LISTS( L,L' ) returns sorted list = LL L' merge of sorted L, L' with duplicates removed. (applying identity) 33 source: 91.503 textbook Cormen et al. Fully Polynomial-Time Approximation Scheme If 2 values are close to each other, then we don't need to explicitly keep them both. both trimming parameter 0 < < 1 y zy 1+ z represents y Theorem: Theorem: Th APPROXAPPROXSUBSETSUBSET-SUM is a fully polynomialpolynomialtime approximation scheme for subset-sum. subset-sum. Proof: Proof: see textbook (differs from 2nd edition) chosen to obtain bound | Li | 3n ln t +2 34 source: 91.503 textbook Cormen et al. ...
View Full Document

Ask a homework question - tutors are online