598CSC: CS Approximation Algorithms Instructor: Chandra Chekuri
Lecture date: January 23, 2009 Scribe: Sungjin Im
In the previous lecture, we had a quick overview of several basic aspects of approximation algorithms. We also addressed approximation (both oine and online) algorithms for the Steiner Tree Problem. In this lecture, we explore another important problem the Traveling Salesperson Problem (TSP).
1
1.1
The Traveling Salesperson Problem (TSP)
TSP in Undirected Graphs
In the Traveling Salesperson Problem, we are given an undirected graph G = (V, E) and cost c(e) > 0 for each edge e E. Our goal is to nd a Hamiltonian cycle with minimum cost. A cycle is said to be Hamiltonian if it visits every vertex in V exactly once. TSP is known to be NP-complete, and so we cannot expect to exactly solve TSP in polynomial time. What is worse, there is no good approximation algorithm for TSP unless P = N P . This is because if one can give a good approximation solution to TSP in polynomial time, then we can exactly solve the NP-Complete Hamiltonian cycle problem (HAM) in polynomial time, which is impossible unless P = N P . Recall that HAM is the decision problem of deciding whether the given graph G = (V, E) has a Hamiltonian cycle or not. Theorem 1 ([2]) The Traveling Salesperson Problem cannot be approximated within any factor, unless P = N P . Proof: For the sake of contradiction, suppose we have an approximation algorithm A on TSP with an approximation ratio C. We show a contradiction by showing that using A, we can exactly solve HAM in polynomial time. Let G = (V, E) be the given instance of HAM. We create a new graph H = (V, E ) with cost c(e) for each e E such that c(e) = 1 if e E, otherwise c(e) = B, where B = nC + 1 and n = |V |. We observe that if G has a Hamiltonian cycle, OPT = n, otherwise OPT n1+B = n(C +1). (Here, OPT denotes the cost of an optimal TSP solution in H.) Note that there is a gap between when G has a Hamiltonian cycle and when it does not. Thus, if A has an approximation ratio of C, we can tell whether G has a Hamiltonian cycle or not: Simply run A on the graph H; if G has a Hamiltonian cycle, A returns a TSP tour in H of cost at most COPT = Cn. Otherwise, H has no TSP tour of cost less than n(C + 1), and so A must return a tour of at least this cost. 2 Since we cannot solve the general TSP problem, we consider TSP with relaxed conditions. There are two possible ways: allowing visiting each vertex many times or requiring a metric condition on cost on edges. Interestingly, both problems are equivalent; we leave this to readers as an exercise. We consider the second one which we call Metric-TSP. In Metric-TSP, the instance is a complete graph G = (V, E) with cost c(e) on e E, where c satises the triangle inequality, i.e. c(uw) c(uv) + c(vw) for any u, v, w V . We rst consider a natural greedy approach, the Nearest Neighbor Heuristic (NNH).
Nearest Neighbor Heuristic(G(V, E), c : E R+ ): Start at an arbitrary vertex s, While (there are unvisited vertices) From the current vertex u, go to the nearest unvisited vertex v. Return to s. Problems to Work On: 1. Prove that NNH is an O(log n)-approximation algorithm. (Hint: Think back to the proof of the 2H|S| -approximation for the Greedy Steiner Tree Algorithm.) 2. NNH is not an O(1)-approximation algorithm; can you nd an example to show this? There are O(1)-approximation algorithms for TSP; we now consider an MST-based algorithm. (See Fig. 1.) TSP-MST(G(V, E), c : E R+ ): Compute an MST T of G. Obtain an Eulerian graph H = 2T by doubling edges of T An Eulerian tour of 2T gives a tour in G. Obtain a Hamiltonian cycle by shortcutting the tour.
Figure 1: MST Based Heuristic Recall that graph G has an Eulerian tour if an only if every vertex v of G has even degrees. Thus we can always nd an Eulerian graph H by doubling edges of T . This simple algorithm gives a 2-approximation. Theorem 2 MST heuristic(TSP-MST) is a 2-approximation algorithm. Proof: We have c(T ) = eE(T ) c(e) OPT, since we can get a tree by removing an edge from the optimal Hamiltonian cycle. Thus c(H) = 2c(T ) 2OPT. Also shortcutting only decreases the cost. 2
We observe that the loss of a factor 2 in the approximation ratio is due to doubling edges; we did this in order to obtain an Eulerian tour. But any graph in which all vertices have even degree is Eulerian, so one can still get an Eulerian tour by adding edges only between odd degree vertices. Christodes Heuristic [1] exploits this and improves the approximation ratio from 2 to 1.5. See Fig. 2 for a snapshot.
Figure 2: Christodes Heuristic
Christofides Heuristic(G(V, E), c : E R+ ): Compute an MST T of G. Let S be the vertices of odd degree in T . (Note: |S| is even) Find a minimum cost matching M on S in G Add M to T to obtain an Eulerian graph H. Compute an Eulerian tour of H. Obtain a Hamilton cycle by shortcutting the tour. Theorem 3 Christodes Heuristic is a 1.5-approximation algorithm. Proof: The main part of the proof is to show that c(M ) .5OPT. Suppose that c(M ) .5OPT. Then, since the solution of Christodes Heuristic is obtained by shortcutting the Eulerian tour on H, its cost is no more than c(H) = c(T ) + c(M ) 1.5OPT. (Refer to the proof of Lemma 2 for the fact c(T ) OPT.) Therefore we focus on proving that c(M ) .5OPT. Let T our be a Hamiltonian tour which achieves OPT. We can get T ourS on S by shortcutting T our . Clearly, c(T ourS ) OPT. Let v1 , v2 , ...., v|S| , v|S|+1 = v1 be the order of vertices in |S| visited by T ourS . Recall that |S| is even. Let M1 = {v1 v2 , v3 v4 , ...v|S|1 v|S| } and M2 = {v2 v3 , v4 v5 , ...v|S| v1 }. Intuitively, we get M1 by taking every alternate edge from T ourS and M2 by taking the other edges. Note that both M1 and M2 are matchings, and c(M1 )+c(M2 ) = c(T ourS ) OPT. WLOG, suppose that c(M1 ) c(M2 ). Then we have c(M1 ) .5OPT. Also we know that c(M ) c(M1 ), since M is a min cost matching on S. Hence we have c(M ) c(M1 ) .5OPT, which completes the proof. 2
Figure 3: A directed graph and a valid Hamiltonian walk
Notes: 1. In practice, local search heuristics are widely used and they perform extremely well. A popular heuristic 2-Opt is to swap pairs from xy, zw to xz, yw or xw, yz, if it improves the tour. 2. There have been no improvements to Metric-TSP since Christodes heuristic was discovered in 1976. It remains a major open problem to improve the approximation ratio of 3 for 2 Metric-TSP; it is conjectured that the Held-Karp LP relaxation [3] gives a ratio of 4 . 3
1.2
TSP in Directed Graphs
In this subsection, we consider TSP in directed graphs. As in undirected TSP, we need to relax the problem conditions to get any positive result. Again, allowing each vertex to be visited multiple times is equivalent to imposing the triangle inequality c(uw) c(uv) + c(vw) for all u, v, w. We focus on the rst model, which we call asymmetric TSP (ATSP). We are given a directed graph G = (V, A) with cost c(a) > 0 for each arc a A and our goal is to nd a closed walk visiting all vertices. Note that we are allowed to visit each vertex multiple times, as we are looking for a walk, not a cycle. For an example of a valid Hamiltonian walk, see Fig. 3. It is important to observe that the MST-based heuristic does not work in this setting. This is because costs on edges are not symmetric, and thus doubling edges might blow up the cost by more than a factor of two. Hence, we need another approach. The Cycle Shrinking Algorithm repeatedly nds a min-cost cycle cover and shrinks cycles, combining the cycle covers found. Recall that a cycle cover is a collection of disjoint cycles covering all vertices. It is known that nding a minimum cost cycle cover can be done in polynomial time. The Cycle Shrinking Algorithm achieves log2 n approximation ratio. Cycle Shrinking Algorithm(G(V, A), c : A R+ ): Transform G s.t. G is complete and satises c(uv) + c(vw) c(uv) for u, v, w Find a minimum cost cycle cover Pick a proxy node for each cycle Recursively solve problem on proxies - extend using cycles The rst step which transforms G so that it satises c(uv) c(uw) + c(wv) for u, v, w can be easily done by setting c(uv) to be the cost of the shortest path for each pair u, v V . Note that
we are implicitly assuming that G is strongly connected. For a snapshot of the Cycle Shrinking Algorithm, see Fig. 4, where cycle covers C1 , C2 and C3 are added one by one.
Figure 4: A snapshot of Cycle Shrinking Algorithm. To the left, a cycle cover C1 . In the center, blue vertices indicate proxy nodes, and a cycle cover C2 is found on the proxy nodes. To the right, pink vertices are new proxy nodes, and a cycle cover C3 is found on the new proxy nodes. Theorem 4 The Cycle Shrinking Algorithm is a log2 n-approximation for ATSP. Let C = {C1 , C2 , ..., Ck } be the collection of cycle covers, where Ci is the ith cycle cover found by the algorithm. Let c(Ci ) be the cost of Ci . We prove Theorem 4 by showing that the cost of each cycle cover c(Ci ) OPT and the number cycle covers k log2 n in Lemma 5 Lemma and 6, respectively. Theorem 4 will directly follow by combining these two lemmas. Lemma 5 For each cycle cover Ci , c(Ci ) OPT. Proof: Let Vi be the proxy nodes at state i, i.e. Vi = V (Ci ). Let Gi be the induced subgraph of G on Vi , i.e. Gi = G[Vi ]. We observe that (1) OPT(Vi ) OPT(G), where OPT(Vi ) and OPT(G) denote the cost of the optimal Hamiltonian walk on Vi and G, respectively. This can be shown by shortcutting the optimal Hamiltonian tour of G so that it passes only Vi . Let Wi be the optimal Hamiltonian walk on Vi . We can show that Wi can be transformed into a cycle cover Ci only by shortcutting. It can be done by repeatedly shortcutting two remaining arcs uv and vw where v is visited more than one time, until Wi becomes a cycle cover. This keeps the invariant that all vertices are visited and each shortcutting decreases the cost of the tour. Thus we have (2) c(Ci ) OPT(Vi ). Finally, we know (3) c(Ci ) c(Ci ), since Ci is a minimum cost cycle cover on Vi . Combining (1), (2) and (3) completes the proof. 2 Lemma 6 There are at most log2 n cycle covers in the outcome of Cycle Shrinking Algorithm, i.e. k log2 n Proof: This can be shown by observing that at most half the vertices survive to the next stage, i.e. |Vi+1 | 1/2|Vi+1 |. 2 Notes: 1. The running time of the Cycle Shrinking Algorithm is O(n2.5 ); it is dominated by nding a min cost cycle cover in the rst stage. It is known that the running time can be reduced to O(n2 ) with a small loss in the approximation ratio. 2. The best approximation ratio so far known is 0.842 log2 n. 3. It has remained an open problem for more than 25 years whether there exists a constant factor approximation for ATSP.
2
2.1
Some Denitions
NP Optimization Problems
In this section, we cover some formal denitions related to approximation algorithms. We start from the denition of optimization problems. Let be an optimization problem. can be either a min or max type. Instances I of are a subset of . We denote the set of feasible solutions by S(I). We assume that for each I and solution S S(I) there is a real/rational number val(S, I). The goal is, given I, to nd OPT(I) = minSS(I) val(S, I) if is a minimization problem. Now let us formally dene NP optimization(NPO) which is the class of optimization problems corresponding to N P . Denition 7 is in NPO if given x , cwe an check if x is an instance of in poly(|x|) time. for each I, and S S(I), |S| is poly(|I|). there exists a poly-time decision procedure that for each I and x , decides if x S(I) val(I, S) is a poly-time computable function We observe that for an NPO problem , one can always get its decision version L(, B) = {I : OPT(I) B}. It is easy to see that L(, B) N P .
2.2
Relative Approximation
When is a minimization problem, recall that we say an approximation algorithm A is said to have approximation ratio i A is a polynomial time algorithm for all instance I of , A produces a feasible solution A(I) s.t. val(A(I), I) val (OPT(I), I). (Note that 1.) Approximation algorithms for maximization problems are dened similarly. An approximation algorithm A is said to have approximation ratio i A is a polynomial time algorithm for all instance I of , A produces a feasible solution A(I) s.t. val(A(I), I) val (OPT(I), I). (Note that 1.) For maximization problems, it is also common to see use 1/ (which must be 1) as approximation ratio.
2.3
Additive Approximation
Note that all the denitions above are about relative approximations; one could also dene additive approximations. A is said to be an -additive approximation algorithm, if for all I, val(A(I)) OPT(I)+. Most NPO problems, however, do not allow any additive approximation ratio because OPT(I) has a scaling property. To illustrate the scaling property, let us consider Metric-TSP. Given an instance I, let I denote the instance obtained by increasing all edge costs by a factor of . It is easy to observe that for each S S(I) = S(I ), val(S, I ) = val(S, I ) and OPT(I ) = OPT(I). Intuitively, scaling edge by a factor of scales the value by the same factor . Thus by choosing suciently large, we can essentially make the additive approximation(or error) negligible. Lemma 8 Metric-TSP does not allow any additive approximation algorithm unless P = N P . Proof: For simplicity, suppose every edge has integer cost. For the sake of contradiction, suppose there exists an additive approximation A for Metric-TSP. Given I, we run the algorithm on I and let S be the solution, where = 2. We claim that S is the optimal solution for I. We have val(S, I) = val(S, I )/ OPT(I )/ + / = OPT(I) + 1/2, as A is -additive approximation. Thus we conclude that OPT(I) = val(S, I), since OPT(I) val(S, I), and OPT(I), val(S, I) are integers. This is impossible unless P = N P . 2 Now let us consider two problems which allow additive approximations. In the Planar Graph Coloring, we are given a planar graph G = (V, E). We are asked to color all vertices of the given graph G such that for any vw E, v and w have dierent colors. The goal is to minimize the number of dierent colors. It is known that to decide if a planar graph admits 3-coloring is NPcomplete, while one can always color any planar graph G with using 4 colors. Further, one can eciently check whether a graph is 2-colorable (that is, if it is bipartite). Thus, the following algorithm is a 1-additive approximation for Planar Graph Coloring: If the graph is bipartite, color it with 2 colors; otherwise, color with 4 colors. As a second example, consider the Edge Coloring Problem, in which we are asked to color edges of a given graph G with the minimum number of dierent colors so that no two adjacent edges have dierent colors. By Vizings theorem [5], we know that one can color edges with either (G) or (G) + 1 dierent colors, where (G) is the maximum degree of G. Since (G) is a trivial lower bound on the minimum number, we can say that the Edge Coloring Problem allows a 1-additive approximation. Note that it is known to be NP-complete to decide whether the exact minimum number is (G) or (G) + 1.
2.4
Hardness of Approximation
Now we move to hardness of approximation. Denition 9 (Approximability Threshold) Given a minimization optimization problem , it is said that has an approximation threshold (), if for any > 0, allows () + approximation but does not allow () approximation. If () = 1, it implies that is solvable in polynomial time. Many NPO problems are known to have () > 1 assuming that P = N P . We can say that approximation algorithms try to decrease the upper bound on (), while hardness of approximation attempts to increase lower bounds on ().
To prove hardness results on NPO problems in terms of approximation, there are largely two approaches; a direct way by reduction from NP-complete problems and an indirect way via gap reductions. Here let us take a quick look at an example using a reduction from an NP-complete problem. In the (metric) k-center problem, we are given an undirected graph G = (V, E) and an integer k. We are asked to choose k centers from V . The goal is to minimize the maximum distance to a center, i.e. minSV,|S|=k maxvV distG (v, S), where distG (v, S) = minuS distG (u, v). The k-center problem has approximation threshold 2, since there are a few 2-approximation algorithms for k-center and there is no 2 approximation algorithm for any > 0 unless P = N P . We can prove the inapproximability using a reduction from the Dominating Set Problem. Given an undirected graph G = (V, E), S V is said to be a dominating set if any v V , either v S or v is adjacent to some u in S. The goal is to decide whether G has a dominating set in G of size k, i.e. |S| = k. This problem is known to be NP-complete. Theorem 10 ([4]) Unless P = N P , there is no 2 approximation for k-center for any > 0.
Proof: Let I be an instance of Dominating Set Problem. We create an instance I of k-center while keeping graph G and k the same. If I has a dominating set of size k then OPT(I ) = 1, since every vertex can be reachable from the Dominating Set by at most one hop. Otherwise, we claim that OPT(I ) 2. This is because if OPT(I ) < 2, then every vertex must be within distance 1, which implies the k-center that witnesses OPT(I ) is a dominating set of I. Therefore, the (2 ) approximation for k-center can be used to solve the Dominating Set Problem. This is impossible, unless P = N P . 2
References
[1] N. Christodes. Worst-case analysis of a new heuristic for the traveling salesman problem. Technical Report, Graduate School of Industrial Administration, CMU 1976. [2] S. Sahni and T.F. Gonzalez. P-Complete Approximation Problems. J. of the ACM 23: 555-565, 1976. [3] M. Held and R. M. Karp. The traveling salesman problem and minimum spanning trees. Operations Research 18: 11381162, 1970. [4] W. L. Hsu and G.L. Nemhauser. Easy and hard bottleneck location problems. Discrete Applied Mathematics. 1:209-216, 1979. [5] D. West. Introductin to Graph Theory. 2d ed. Prentice Hall. 277-278, 2001.