This preview shows page 1. Sign up to view the full content.
Unformatted text preview: 15.083J/6.859J Integer Optimization
Lecture 9: Solving Relaxations 1 Outline The key geometric result behind the ellipsoid method The ellipsoid method for the feasibility problem The ellipsoid method for optimization Problems with exponentially many constraints Slide 1 2 The Ellipsoid method is called an ellipsoid with center z 2 <n E = E (z D) = x 2 <n j (x ; z ) D 1(x ; z) 1
0 ; 2.1 The algorithm intuitively
is nonempty Problem: Decide whether a given polyhedron P = x 2 <n j Ax b Et + 1 Et . xt 1 A set E of vectors in <n of the form P xt+ 1 a'x >b . a ' x > a ' xt D is an n n positive de nite symmetric matrix Slide 2 Slide 3 2.2 Key Theorem E = E (z D) be an ellipsoid in <n a nonzero n-vector. H = fx 2 <n j a x a z g
0 0 0 1 Da z = z + n + 1 pa Da 2 D 2 D = n2n; 1 D ; n + 1 Daa a : aD
0 0 The matrix D is symmetric and positive de nite and thus E = E (z ellipsoid
0 0 ; D) is an E \H E Vol(E ) < e 0 1=(2(n+1)) Vol(E ) 2.3 Illustration
x2 E' E x1 2 Key property: We can nd a new ellipsoid Et+1 that covers the halfellipsoid and whose volume is only a fraction of the volume of the previous ellipsoid Et Slide 4 Slide 5 2.4 Assumptions 2.5 Input-Output
A matrix A and a vector b that de ne the polyhedron P = fx 2 <n j aix bi i = 1 : : : mg A number v, such that either P is empty or Vol(P ) > v A ball E0 = E (x0 r2I ) with volume at most V , such that P E0 Output: A feasible point x 2 P if P is nonempty, or a statement that P is empty
0 � Input: 2.6 The algorithm
1. (Initialization) Let t = 2(n + 1) log(V =v) E0 = E (x0 r2 I ) D0 = r2 I t = 0. 2. (Main iteration) If t = t stop P is empty. If xt 2 P stop P is nonempty. If xt 2 P nd a violated constraint, that is, nd an i such that ai xt < bi . = Let Ht = fx 2 <n j ai x ai xt g. Find an ellipsoid Et+1 containing Et \ Ht : Et+1 = E (xt+1 Dt+1 ) with 1 xt+1 = xt + n + 1 pDt ai ai Dt ai 2 2 i Dt+1 = n2n; 1 D t ; n + 1 Dat aDatiaDt : i i
0 0 0 0 0 0 t := t + 1. 3 A polyhedron P is full-dimensional if it has positive volume The polyhedron P is bounded: there exists a ball E0 = E (x0 r2I ), with volume V , that contains P Either P is empty, or P has positive volume, i.e., Vol(P ) > v for some v>0 E0, v V , are a priori known We can make our calculations in in nite precision square roots can be computed exactly in unit time Slide 6 Slide 7 Slide 8 and for which the prior information x0 , r, v, V is available. Then, the ellipsoid method decides correctly whether P is nonempty or not, i.e., if xt� 1 2 P , then = P is empty
; Theorem: Let P be a bounded polyhedron that is either empty or full-dimensional 2.7 Correctness Slide 9 2.8 Proof ; � Suppose P Ek for some k < t . Since xk 2 P , there exists a violated = inequality: ai(k)x bi(k) be a violated inequality, i.e., ai(k)xk < bi(k), where xk is the center of the ellipsoid Ek For any x 2 P , we have
� 0 0 a ( ) x b ( ) > a ( )x
0 ik ik 0 ik k Hence, P Hk = x 2 <n j ai(k)x ai(k)xk Therefore, P Ek \ Hk By key geometric property, Ek \ Hk Ek+1 hence P
complete 0 0 Ek+1 and the induction is Vol(Et� ) < V e 2(n+1) log v =(2(n+1)) V e log v = v If the ellipsoid method has not terminated after t iterations, then Vol(P ) Vol(Et� ) v. This implies that P is empty
;d e ; Vol(Et+1 ) < e 1=(2(n+1)) Vol(Et ) Vol(Et� ) < e t� =(2(n+1)) Vol(E0 )
; ; V V 2.9 Binary Search 4 P = x2<jx 0 x 1 x 2 x 3 E0 = 0 5], centered at x0 = 2:5 If xt 2 P for t < t , then the algorithm correctly decides that P is nonempty Suppose x0 : : : xt� 1 2 P . We will show that P is empty. = We prove by induction on k that P Ek for k = 0 1 : : : t . Note that P E0 , by the assumptions of the algorithm, and this starts the induction.
� Slide 10 Slide 11 Slide 12 Slide 13 Let A be an m n integer matrix and let b a vector in <n. Let U be the largest absolute value of the entries in A and b. Every extreme point of the polyhedron P = fx 2 <n j Ax bg satis es
;(nU )n 2.10 Boundedness of P xj (nU )n j = 1 ::: n
Slide 15 All extreme points of P are contained in PB = x 2 P jxj j (nU )n j = 1 : : : n Since PB E 0 n(nU )2nI , we can start the ellipsoid method with E0 = ; E 0 n(nU )2nI ; V ol(E0) V = 2n(nU )n n = (2n)n (nU )n ; 2 Let P = fx 2 <n j Ax bg. We assume that A and b have integer entries, which are bounded in absolute value by U . Let ; = 2(n 1 1) (n + 1)U (n+1) : + Let P = x 2 <n j Ax b ; e where e = (1 1 : : : 1). (a) If P is empty, then P is empty. (b) If P is nonempty, then P is full-dimensional. Let P = x 2 <n j Ax b be a full-dimensional bounded polyhedron, where the entries of A and b are integer and have absolute value bounded by U . Then,
; 2.11 Full-dimensionality Vol(P ) > v = n n(nU )
; ;n 2 (n+1) 5 Since x0 2 P , the algorithm chooses the violated inequality x 2 and = constructs E1 that contains the interval E0 \ fx j x 2:5g = 0 2:5] The ellipsoid E1 is the interval 0 2:5] itself Its center x1 = 1:25 belongs to P This is binary search Slide 14 Slide 16 Slide 17 2.12 Complexity 3 The ellipsoid method for optimization
By strong duality, both problems have optimal solutions if and only if the following system of linear inequalities is feasible: b p = c x Ax b A p = c p 0: LO with integer data can be solved in polynomial time. min s:t: cx Ax b
0 max s:t: b A =c
0 0 0: 3.1 Sliding objective 0 6 More generally, every time a better feasible solution xt is found, we take P \ fx 2 <n j c x < c xt g as the new set of inequalities and reapply the ellipsoid method.
0 0 We rst run the ellipsoid method to nd a feasible solution x0 2 P = x 2 <n j Ax b . We apply the ellipsoid method to decide whether the set P \ x 2 <n j c x < c x0 is empty. If it is empty, then x0 is optimal. If it is nonempty, we nd a new solution x1 in P with objective function value strictly smaller than c x0.
0 0 0 0 0 and the numbers computed during the algorithm have polynomially bounded size The linear programming feasibility problem with integer data can be solved in polynomial time P = fx 2 <n j Ax bg, where A, b have integer entries with magnitude bounded by some U and has full rank. If P is bounded and either empty or full-dimensional, the ellipsoid method decides if P is empty in ; O n log(V =v) iterations v = n n (nU ) n2(n+1) V = (2n)n (nU )n2 ; Number of iterations O n4 log(nU ) If P is arbitrary, we rst form PB , then perturb PB to form PB and apply the ellipsoid method to PB ; Number of iterations is O n6 log(nU ) : It has been shown that only O(n3 log U ) binary digits of precision are needed, Slide 18 ; ; Slide 19 Slide 20 Slide 21 Slide 22 -c x .t+ 1 c ' x < c' x t + 1 .x t
P c ' x < c' x t 3.2 Performance in practice 4 Problems
jS j X
i ci xi ai xi for all subsets S of f1 : : : ng There are 2n constraints, but are described concisely in terms of the n scalar parameters a1 : : : an Question: Suppose we apply the ellipsoid algorithm. Is it polynomial? In what? Consider min c x s.t. x 2 P P belongs to a family of polyhedra of special structure A typical polyhedron is described by specifying the dimension n and an integer vector h of primary data, of dimension O(nk ), where k 1 is some constant. In example, h = (a1 : : : an) and k = 1 U0 be the largest entry of h
0 4.2 The input 7 Very slow convergence, close to the worst case Contrast with simplex method The ellipsoid method is a tool for classifying the complexity of linear programming problems Slide 23 Slide 24 Slide 25 Slide 26 log U C n` log` U0
Slide 27 Given a polyhedron P <n and a vector x 2 <n, the separation problem is to: Either decide that x 2 P , or Find a vector d such that d x < d y for all y 2 P What is the separation problem for
0 0 5 The separation problem X
i2S ai xi jS j for all subsets S of f1 : : : ng? 6.1 Theorem 6 Polynomial solvability
Slide 28 If we can solve the separation problem (for a family of polyhedra) in time polynomial in n and log U , then we can also solve linear optimization problems in time polynomial in n and log U . If log U C n` log` U0 , then it is also polynomial in log U0 Proof ? Converse is also true Separation and optimization are polynomially equivalent cexe eP E xe 1 e (S ) P xe = n ; 1 eE xe 2 f0 1g: How can you solve the LP relaxation?
2 2 6.2 MST 2 I ZM S T = min s:t: P 8S V S= V 6 8 Given n and h, P is described as Ax b A has an arbitrary number of rows U largest entry in A and b. We assume Slide 29 6.3 TSP
xe = 1 if edge e is included in the tour. 0 otherwise. min s:t:
2 Slide 30 ce xe xe 2 S E e (S ) P xe = 2 i 2 V e (i) xe 2 f0 1g How can you solve the LP relaxation?
e2E P 2 6.4 Probability Theory 6.4.1 Formulation
x(S ) = P
f ; \i2S Ai \ \i2S Ai = ; X S ji2S g x(S ) x(S ) pi pij i2N i j 2N i<j f S ji j 2S g S X x(S ) = 1 x(S ) 0 8 S: The previous LP is feasible if and only if there does not exist a vector (u that X X yij + ui + z 0 8S
i j 2S i<j i j 2N i<j i2N y z ) such pij yij + X i2S pi ui + z ;1 9 yij 0 ui 0 i j 2 N i < j: P(Ai ) pi i2N P(Ai \ Aj ) pij i j 2 N i < j: Given the numbers pi and pij , which are between 0 and 1, are these beliefs consistent? Events A1 A2 P (A1 ) = 0:5 P (A2 ) = 0:7, P (A1 \ A2 ) 0:1 Are these beliefs consistent? General problem: Given n events Ai i 2 N = f1 : : : ng, beliefs Slide 31 Slide 32 Slide 33 Slide 34 Separation problem: z + min f (S ) = S
� � � � � � � X
i j 2S i<j
� yij +
� � X
i2S ui � 0?
� � Example: y12 = ;2, y13 = ;4, y14 = ;4, y23 = ;4, y24 = ;1, y34 = ;7, u1 = 9, u2 = 6, u3 = 4, u4 = 2, and z = 2
� Slide 35 1,2
4 4 s 4 2,3 1 1,4 2,4 7 3,4 The minimum cut corresponds to S0 = f3 4g with value c(S0 ) = 21. f (S0 ) =
� X yij +
� � X ui = ;7 + 4 + 2 = ;1
� i j 2S0 i<j i2S0 f (S ) + z f (S0 ) + z = ;1 + 2 = 1 > 0 Given solution (y u z ) is feasible
� � � 7 Conclusions 10 Ellipsoid algorithm can characterize the complexity of solving LOPs with an exponential number of constraints For practical purposes use dual simplex Ellipsoid method is an important theoretical development, not a practical one 1 9 2 6 4 3 2 4 t Slide 36 8 S Slide 37 ...
View Full Document
- Spring '04