This preview shows page 1. Sign up to view the full content.
Unformatted text preview: UMass Lowell Computer Science 91.503 Analysis of Algorithms
Prof. Karen Daniels
Spring, 2011 i Lecture 7
Tuesday, 4/5/11 y, Approximation Algorithms
1 Approximation Algorithms A i i Al i h Chapter 35
(except Section 35.4) 2 Motivation: Some Techniques for Treating NPComplete Problems NP Complete Approximation Algorithms Heuristic Upper or Lower Bounds Greedy, Simulated Annealing, Genetic "Alg", AI "Alg", Linear Programming for part of problem Integer Programming Quadratic, Convex Programming... Gradient Descent, Local Search, Pruning, Subdivision Mathematical Programming M h i lP i Search Space Exploration: Randomization, Derandomization Leverage/Impose Problem Structure Leverage Similar Problems 3 Basic Concepts
Definitions fi i i 4 Definitions Approximation Algorithm Algorithm has approximation ratio (n) if: C C * max , ( n) C* C min max produces "nearoptimal" solution "near Approximation Scheme C = cost of algorithm's solution f l ih ' l i C* = cost of optimal solution n = number of inputs = size of instance (1+ (1+)approximation algorithm for fixed > 0 is an input time is polynomial in n time is also polynomial in (1/) (1/ constantconstantfactor decrease in causes only constantfactor runningtime constantrunning5 increase source: 91.503 textbook Cormen et al. PolynomialPolynomialTime Approximation Scheme l l h Fully PolynomialTime Approximation Scheme y Polynomialy pp Resources beyond textbook... "Approximation Algorithms for NPHard Problems", editor NPProblems", Dorit Hochbaum, PWS Publishing, Co., 1997. Hochbaum, "Approximation Algorithms" by Vazirani, 2001. Algorithms" Vazirani, "Computers and Intractability" by Garey & Johnson, 1979. Intractability" UMass Lowell course taught by Prof. Jie Wang. 6 Overview V VERTEXCOV VERTEXCOVER Polynomialtime 2approximation algorithm Polynomial2TSP TSP with triangle inequality Polynomialtime 2approximation algorithm Polynomial2 TSP without triangle inequality Negative result on polynomialtime (n)approximation algorithm polynomial(n)MAX3C MAX CNF S i fi bili A Satisfiability Randomized (n)approximation algorithm (n)SETSETCOVER polynomialtime (n)approximation algorithm polynomial(n) (n) is a logarithmic function of set size SUBSETSUBSETSUM Fully polynomialtime approximation scheme polynomial7 Vertex Cover
Polynomial i 2PolynomialTime 2Approximation Algorithm l i l i i l ih 8 Vertex Cover
A B C D E F
vertex cover of size 2 Vertex Cover of an undirected graph G=(V,E) is a subset V ' V such that if (u , v) E , then u V ' or v V ' or both NPNPComplete
(via reduction from CLIQUE in Ch. 34 of Cormen et al.) al.)
9 source: Garey & Johnson Vertex Cover
edge (b,c) chosen in line 4
b c d d a e f g a e f g d a g a c (vertex cover)
a a f g 10 source: 91.503 textbook Cormen et al. Vertex Cover
Theorem: APPROXVERTEXTheorem: APPROXVERTEXCOVER is a polynomialpolynomialtime 2approximation algorithm. 2algorithm. Proof: Let Proof: L t C * b an optimal cover be ti l
C be cover from APPROX  VERTEX  COVER A edges chosen by APPROX  VERTEX  COVER g y
Observe: Observe: no 2 edges of A share any vertices in C due to removal of incident edges
Any vertex cover must include >= 1 endpoint of each edge in A > APPROXVERTEXAPPROXVERTEXCOVER adds both endpoints of each edge of A C* A
C = 2 A
11 C = 2 A 2C* transitivity C 2C* C /C* 2 Algorithm runs in time polynomial in n. Traveling Salesman
TSP with triangle inequality ih i l i li
PolynomialPolynomialtime 2approximation algorithm 2 TSP without triangle inequality
Negative result on polynomialtime (n)approximation polynomialalgorithm l ith
12 Hamiltonian Cycle
Hamiltonian Schlegl diagram of dodecahedron: 12 sides (one of 5 3D Platonic solids) lid ) Not Hamiltonian 34.2 Hamiltonian Cycle of an undirected graph G=(V,E) is a simple cycle th t contains each vertex i V i l l that t i h t in V. NPNPComplete (via reduction from 13
VERTEXVERTEXCOVER in Ch. 34 of Cormen et al.) al.)
source: 91.503 textbook Cormen et al. Traveling Salesman Problem (TSP)
TSP Tour of a complete, complete, undirected, weighted graph G=(V,E) is a Hamiltonian Cycle with a designated starting/ending vertex.
{< G, c, k >:
u v x w TSP Decision Problem:
Cost function provides edge weights. Note these are integers. Why? c : V V Z k Z G has TSP  tour of cost k} NPComplete (via reduction from NPC l t
HAMHAMCYCLE in Ch. 34 of Cormen et al.) al.)
14 source: 91.503 textbook Cormen et al. Minimum Spanning Tree: Greedy Algorithms
Time: Time: O(ElgE) given fast FINDFINDSET, UNION Invariant: Minimum weight spanning forest Produces minimum weight tree of edges that includes every vertex. A
Becomes single tree at end 3 1 2 4 G 5 E 6 8 2 7 F 4 6 B Time: Time: O(ElgV) = O(ElgE) slightly faster with fast priority queue 1 Invariant: Minimum weight tree Spans all vertices at end D C for Undirected, Connected, Weighted Graph G=(V,E) G (V E)
15 source: 91.503 textbook Cormen et al. TSP with Triangle Inequality
Why Prim? u 5 3 5 x 4 w 3 v Cost Function Satisfies "full walk" = abcbhbadefegeda Triangle Inequality u , v, w V c(u , w) c(u , v) + c(v, w)
final approximate tour source: 91.503 textbook Cormen et al.
(removes vertex duplication) optimal tour (not necessarily found by 16
approximation algorithm) TSP with Triangle Inequality
Theorem: APPROXTSPTheorem: APPROXTSPTOUR is a polynomialpolynomialtime 2approximation algorithm 2for TSP with triangle inequality. inequality. Proof: Proof: Algorithm runs in time polynomial in n = V . V Let H * be an optimal tour and T be a MST c (T ) c( H *) )
c(W ) = 2c(T ) ( (since deleting 1 edge from a tour creates a spanning tree) g g p g )
(lists vertices when they are first visited and when returned to after visiting subtree) Let W be a full walk of T (since full walk traverses each edge of T twice) c(W ) 2c( H *) c( H ) c(W ) Now make W into a tour H using triangle inequality. (New inequality holds since H is formed by deleting duplicate vertices from W.) c( H ) 2c( H *) 17 TSP without Triangle Inequality
Theorem: Theorem: If P NP , then for any constant 1 polynomialpolynomialtime approximation algorithm with ratio for TSP without triangle inequality. inequality. Proof: (b contradiction) Suppose there is one  call it A Proof: (by P f di i )
Showing how to use A to solve NPcomplete Hamiltonian Cycle problem NPcontradiction! Convert instance G of Hamiltonian Cycle into instance of TSP (in polynomial time): E ' = {(u , v) : u , v V , u v}
(u , v) E 1 c(u , v) =  V  +1 otherwise (G'=(V,E') is complete graph on V) (assign integer cost to each edge in E') Cost of that tour of G (  V  +1) + ( V  1) =  V  +  V >  V  G' Can use A on (G',c)! A finds tour of cost at most (length of optimal tour of G') G has Hamiltonian Cycle A finds tour of cost at most V cost = V 18 G does not have Hamiltonian Cycle A finds tour of cost > V If P NP , solving NPcomplete hamiltonian cycle in polynomial time is a contradiction! NP For F TSP problem (G',c): bl (G' ) (G',c) contains tour of cost V G has Hamiltonian Cycle Tour of G' must use some edge not in E G does not have Hamiltonian Cycle MAX3CNF Satisfiability 3CNF Satisfiability Background i fi bili k d Randomized Algorithms
Randomized MAX3CNF SAT Approximation Algorithm MAX 19 MAX3CNF Satisfiability Background on Boolean Formula Satisfiability Boolean Formula Satisfiability: Instance of language SAT is Satisfiability: a boolean formula consisting of: n boolean variables: x1, x2, ... , xn m boolean connectives: boolean function with 1 or 2 inputs and 1 output e.g. AND, OR, NOT, implication, iff parentheses truth, satisfying assignments notions apply SAT = {< >: is a satisfiable boolean formula}
NPNPComplete (via reduction from
CIRCUITCIRCUITSAT in Ch. 34 of Cormen et al.) al.)
20 source: 91.503 textbook Cormen et al. MAX3CNF Satisfiability
(continued) Background on 3CNFSatisfiability g 3CNFy Instance of language SAT is a boolean formula consisting of: literal: variable or its negation l l i bl i i CNF = conjunctive normal form conjunction: conjunction: AND of clauses clause: clause: OR of literal(s) 3CNF: each clause has exactly 3 distinct literals MAXMAX3CNF Satisfiability: optimization version of 3CNFSAT Satisfiability: 3CNF Maximization: satisfy as many clauses as possible  Input Restrictions:  exactly 3 lit l / l tl literals/clause  no clause contains both variable and its negation (this assumption can be removed)
21 NPNPComplete (via reduction from SAT in Ch. 34 of Cormen et al.) al.) source: 91.503 textbook Cormen et al. Definition Randomized Algorithm has approximation ratio (n) if for expected cost C of solution if, produced by randomized algorithm: C C * max , ( n) C* C min max size of input = n 22 source: 91.503 textbook Cormen et al. Randomized Approximation Algorithm for MAX 3 CNF SAT MAX3CNF
Theorem: Theorem: Given an instance of MAX3CNF satisfiability with n MAXvariables x1, x2,..., xn with m clauses, the randomized algorithm th t i bl ith l th d i d l ith that independently sets each variable to 1 with probability 1/2 and to 0 with probability 1/2 is a randomized 8/7approximation algorithm. C * 8 8/7C 7 Proof: independen set each varia to 0 or 1 with = probability = 1/2 Proof: P f tly ble
indicator random variable = Yi = I {event that clause i is satisfied}
Pr{clause i is not satisfied} = (1/2)3 = 1 / 8 Pr{clause i is satisfied} = 1  (1 / 8) = 7 / 8
Lemma : Given a sample space S and an event A in the sample space S , Let X A = I{A}. Then E[X A ] = Pr {A}. { } [ { } By Lemma : E[Yi ] = 7 / 8 LetY = Y1 + Y2 +L+ Ym
m 7 7m m m E[Y ] = E Yi = E [Yi ] = = = expected cost C 8 8 i =1 i =1 i =1 number of satisfied clauses m C* m source: 91.503 textbook Cormen et al. C* m 8 = = ( n) 23 C 7m / 8 7 Set Cover
Greedy Approximation Algorithm d i i l ih
polynomialpolynomialtime (n)approximation algorithm
(n) is a logarithmic function of set size i l ith i f ti f t i 24 Set Cover Problem
Instance ( X , F ) :
finite set X (e.g. of points) family F of subsets of X X= U S
SF Problem Find P bl : Fi d a minimum  sized subset C F i i i d b whose members cover all of X: X = U S
SC NPNPComplete (via reduction from VERTEXVERTEXCOVER as noted in Ch. 35 of Cormen et al.) al.) 25 source: 91.503 textbook Cormen et al. Greedy Set Covering Algorithm Greedy: Greedy: select set that covers the most uncovered elements
26 source: 91.503 textbook Cormen et al. Set Cover
Theorem: GREEDYSETTheorem: GREEDYSETCOVER is a polynomialpolynomialtime (n)approximation algorithm for ( n ) = H (max{ S : S F }) d Proof: Proof: P f 1
d th harmonic number H d = i
i =1 = H (d ) H(0)=0 Algorithm runs in time polynomial in n. S i = i th subset selected
(notational caveat) by algorithm selecting S i costs 1 cx = cost of element x X
cx = 1 paid only when x is covered for the first time assume x is covered for the first time by Si
(spread cost evenly across all elements covered for first time by Si ) Number of elements covered for first time by Si 27 S i  ( S 1 S 2 L S i 1 )
elements already covered by first i1 chosen sets i Set Cover (proof continued)
Theorem: GREEDYSETTheorem: GREEDYSETCOVER is a polynomialpolynomialtime (n)approximation algorithm for ( n) = H (max{ S : S F }) Proof: (continued) Proof: P f
Let C * be an optimal cover C be cover from GREEDY  SET  COVER
Cost assigned to optimal cover: C = x X c x 1 unit is charged at each stage of algorithm * S c x S C x Each x is in >= 1 S in C* SC* xS cx xcx X C cx SC* xS 28 Set Cover (proof continued)
Theorem: GREEDYSETTheorem: GREEDYSETCOVER is a polynomialpolynomialtime (n)approximation algorithm (n)for ( n) = H (max{ S : S F }) Proof: (continued) Proof: P f
How does this relate to harmonic numbers?? We'll show that: d th harmonic number H d = H(S ) d i =1 1 = H (d ) i c
x S x for any set S F C
And then using conclude that:
which will finish the proof cx SC* xS C S C * H(S ) C * H (max{ S : S F })
29 Set Cover (proof continued)
Proof of: c
x S x H(S ) for any set S F
For some set S: Number of elements of S remaining uncovered after S1...Si selected u i = S  ( S1 S 2 L S i )  u 0 = S  c
x S x = k k i =1 1 ( u i 1  u i ) S i  ( S 1 S 2 L S i 1 ) k = least index f for which uk=0. c
x S x i =1 1 ( u i 1  u i ) u i 1 Since, due to greedy  S i  ( S 1 S 2 L S i 1 )  nature of algorithm:  S  ( S 1 S 2 L S i 1 ) = u i 1 = = i =1
k k u i 1 1 j = u i + 1 u i 1 k i =1 1 +1 j j =ui u i 1 k i =1 (since j <= ui1) ui u i 1 1 1  j =1 j j =1 j = (H ( u
i =1 i 1 )  H (u i ) ) = H (u 0 )  H (u k ) = H (u 0 )  H ( 0 ) = H (u 0 ) = H ( S )
30 telescoping sum SubsetSum
Exponential i ExponentialTime Exact Algorithm i l l ih Fully PolynomialTime Approximation Scheme Polynomial 31 SubsetSum Problem Instance ( S , t ) :
Decision Problem: S = {x1,x2 ,...,xn }
S ' S : s = t ?
sS ' positive integers Optimization Problem seeks subset with largest sum <= t NPNPComplete (via reduction from
3CNFSAT in Ch. 34 of Cormen et al.) CNFal.) 32 source: 91.503 textbook Cormen et al. ExponentialTime Exact Algorithm
Pi = set of values obtainable by selecting subset of {x1,x2 ,...,xi } and summing elements.
Identity: Pi = Pi 1 ( Pi 1 + xi ) Li is sorted list of every element of Pi <= t Li = list of sums of subsets of {x1,x2 ,...,xi } t
MERGEMERGELISTS( L,L' ) returns sorted list = LL L' merge of sorted L, L' with duplicates removed. (applying identity) 33 source: 91.503 textbook Cormen et al. Fully PolynomialTime Approximation Scheme
If 2 values are close to each other, then we don't need to explicitly keep them both. both trimming parameter 0 < < 1 y zy 1+ z represents y Theorem: Theorem: Th APPROXAPPROXSUBSETSUBSETSUM is a fully polynomialpolynomialtime approximation scheme for subsetsum. subsetsum. Proof: Proof:
see textbook (differs from 2nd edition) chosen to obtain bound  Li  3n ln t +2
34 source: 91.503 textbook Cormen et al. ...
View Full
Document
 Spring '11
 Staff
 Algorithms

Click to edit the document details