171A: Math Linear Programming
Instructor: Philip E. Gill
Winter Quarter 2011
Homework Assignment #6 Due Friday February 18, 2011 Starred exercises require the use of Matlab. Exercise 6.1. (a) Consider the matrix of active constraints -1 -8 4 -7 -3 -4 2 0 4 -2 . Aa = -6 0 -3 3 5 0 0 5 2 7 Use solve.m to construct a direction p along which the residual of the third constraint increases, but all the other residuals remain the same. If the rows of Aa are labeled 1 through 4, what are the indices of the active set after a positive step along this direction? >> format compact >> Aa = [ -1 -8 -4 2 -6 0 0 0 >> e3 = [ 0 0 >> rank(Aa) ans = 4 4 0 -3 5 1 -7 -3 4 -2 3 5 2 7 ]; 0 ]';
The equation Aa p = e3 is guaranteed to be compatible (do you recall why?) >> p=solve(Aa,e3) p = -0.06484375000000 -0.06328125000000 -0.09296875000000 0 0.06640625000000 >> Aa*p ans = 0.00000000000000 0.00000000000000 1.00000000000000 0 >> diary off
2
Mathematics 171A Based on the components of the vector Aa p, we deduce that constraints 1, 2 and 4 have constant residual along p, whereas constraint 3 has increasing residual along p. This means that taking a small positive step along p will move us off constraint 3 but leave us on constraints 1, 2 and 4. Therefore the active set changes from {1, 2, 3, 4} to {1, 2, 4}.
(b) Repeat part (a) for the vertex 0 9 0 Aa = -3 3 -8 -1 3 6 -5 -4 1 6 -4 5 1 6 -5 -4 -4 -5 3 2 . 5 -6 3 2 -3 0 7 -1 0 0 0 0
Comment on your results. Is the vertex degenerate or nondegenerate? >> Aa = [ 0 9 0 -3 3 -8 -1 >> rank(Aa) ans = 5 3 1 1 -4 5 -3 0 6 6 6 -5 -6 0 0 -5 -4 -5 3 3 7 0 -4 5 -4 2 2 -1 0 ];
The active constraint matrix has full column rank, but has more columns than rows. In this situation, the equations Aa p = e3 are unlikely to be compatible, which turns out to be the case. >> p=solve(A,e3) The equations Ax = b are incompatible. p = [] >> diary off Since the active constraint matrix has rank n, we are at a vertex. Since more than n constraints are active, the vertex is degenerate (see Page 59 of the Class Text). The incompatibility of the equations Aa p = e3 implies that we cannot move off constraint 3 while staying on the other constraints. Exercise 6.2. Let Aa be a nonzero m n matrix, and let CN denote the normal cone CN = {w : w = AT , a (a) Show that CN is a convex set. If w1 and w2 are any two vectors in CN , then w1 = AT 1 and w2 = AT 2 for nonnegative a a 1 and 2 . Given any [0, 1], it holds that w = (1 - )w1 + w2 = (1 - )AT 1 + AT 2 a a = AT (1 - )1 + 2 . a (6.1) Clearly (1 - )1 + 2 0 since 1 0 and 2 0. Hence w CN . As w1 and w2 are arbitrary and (6.1) holds for 0 1, we conclude that CN is a convex set. 0}.
Homework #6 (b) Is CN a subspace of Rn ? Explain why or why not.
3
The normal cone CN is a subspace if w1 + w2 CN for all w1 , w2 CN . If CN = Rn (see, e.g., Exercise 6.3 below), then it is trivially a subspace because Rn is a subspace of itself. However, if CN Rn , then given any y CN , the vector -y CN . Hence, CN is not a subspace in general. Exercise 6.3. Suppose that the objective vector of a linear program is c = (-1, -2)T , and that the matrix of active constraints at a feasible point x is 1 1 1 -2 Aa = 1 -3 . 1 2 -1 0 (a) Is x a vertex? If so, is it degenerate or nondegenerate? Explain your answer. The point x is a vertex since Aa has rank n, i.e., rank(Aa ) = 2. Since more than two constraints are active (Aa is 5 2), the vertex is degenerate. (b) Graph the normal cone. Use your graph to either verify that x is optimal or find a direction p such that cT p < 0 and Aa p 0.
a4 a1 a5
a2 a3
5
The normal cone is the whole plane R2 since every vector can be written as a nonnegative combination of the rows of Aa . This implies that every vector c satisfies c CN and there is no direction p such that cT p < 0 and Aa p 0. It follows that x is optimal. Exercise 6.4. Consider the linear program minimize subject to
x1 ,x2 ,x3
3x1 - x2 + 2x3 - 2x1 + 4x2 + 4x3 x1 + 4x2 + x3 - 2x1 + x2 + 2x3 2x1 - 2x2 - 3x2 + x3 x1 x2 x3 6 5 1 0 -2 0 0 0,
4
Mathematics 171A
and the point x = (1, 1, 1)T . Find the active set at x and determine if the point x is optimal. If x is not optimal, find a direction p such that cTp < 0 and Aa p 0. We use the m-file constraintValues.m to calculate the constraint values. >>type constraintValues >> function [r,feasible,infeasible]=constraintValues(x,A,b,tol) % [r,feasible,infeasible]=constraintValues(x,A,b,tol) % % Given a point x and a set of linear inequality constraints % Ax >=b, constraintValues computes the residual vector % r = Ax - b and the indices of the feasible and % infeasible constraints at x. % % On entry, x is the point at which feasibility is being tested. % A is the matrix for the inequalitues Ax >= b. % b is the right-hand side vector for Ax >= b. % On exit, r is the vector of constraint values. % feasible is the number of feasible constraints. % infeasible is the number of infeasible constraints. % This version dated 14-Jan-2007.
if nargin < 4 tol = 10*eps; % Allows for rounding error when computing r; end r = A*x - b; feasible = length( find( r >= -tol ) ); infeasible = length( find( r < -tol ) ); >> >> A = [ -2 4 4 1 4 1 -2 1 2 2 -2 0 0 -3 1 1 0 0 0 1 0 0 0 1 ]; >> b = [ 6 5 1 0 -2 0 0 0 ]'; >> c = [ 3 -1 2 ]'; >> xbar = [ 1 1 1 ]'; >> [r,feas,infeas] = constraintValues(xbar,A,b) r = 0 1 0 0 0
Homework #6
5
1 1 1 feas = 8 infeas = 0 >> active = [1 3 4 5]; Aa = -2 4 4 -2 1 2 2 -2 0 0 -3 1 >> rank(Aa) ans = 3
Aa = A(active,:)
Since Aa has full column rank but not full row rank, we conclude that x is a degenerate vertex. To check for optimality, we must find all the basic solutions of AT = c. There are a at most four (i.e., "4 choose 3") basic sets. >> basic = [ 1 2 3 ]; rank(Aa(basic,:)) ans = 3 >> lambda = solve(Aa(basic,:)',c) lambda = 0.7500 -0.5000 1.7500 This multiplier vector has a negative component, so we have to press on. >> basic = [ 1 2 4 ]; rank(Aa(basic,:)) ans = 3 >> lambda = solve(Aa(basic,:)',c) lambda = 1.7222 -3.2222 1.5556 >> basic = [ 1 3 4 ]; rank(Aa(basic,:)) ans = 3 >> lambda = solve(Aa(basic,:)',c) lambda = 0.5714 2.0714 -0.2857 So far, every basic solution has a negative component. One more basic solution to check. >> basic = [ 2 3 4 ]; rank(Aa(basic,:))
6
Mathematics 171A
ans = 3 >> lambda = solve(Aa(basic,:)',c) lambda = 1.6000 3.1000 -1.2000 Every basic solution has a negative component, so we can conclude that x is not optimal. To find a feasible descent direction, we use the solutions generated in the first part and attempt to move off constraints with a negative Lagrange multiplier. The multipliers will be recalculated as needed. >> basic = [1 2 3]; lambda = solve(Aa(basic,:)',c) lambda = 0.7500 -0.5000 1.7500 Since the second multiplier is negative, a move off of the second constraint in the active set (which is the third constraint of A) is a descent direction for the objective function. >> e2 = [ 0 1 0 ]'; p = -0.5000 -0.5000 0.2500 p = solve(Aa(basic,:),e2)
We must see if p is a feasible direction, i.e., we must check if Aa p 0. >> format long >> Aa*p ans = -0.00000000000000 1.00000000000000 0 1.75000000000000 Good enough for government work! This means that we have a feasible direction with respect to the matrix of active constraints. We know that p is a descent direction, but let's check it anyway. >> c'*p ans = -0.50000000000000 If p had not been a feasible direction, then it would have been necessary to continue calculating a p corresponding to each of the three remaining basic sets until a feasible descent direction is found. In each case, the p must be defined so that it moves off of the constraint with a negative multiplier. Can you explain why this strategy is guaranteed to work?
Homework #6
7
Exercise 6.5. Consider the linear program minimizexRn cTx subject to Ax b. Suppose that the objective vector is c = (0, 2, 2)T , and that at a vertex x, the active constraint matrix is given by 1 2 3 0 2 1 0 0 1 . Aa = 1 2 -1 0 -1 0 (a) Is x a degenerate or nondegenerate vertex? Explain your answer. The vertex is degenerate since more than n = 3 constraints are active at x. This tells us that the Lagrange multipliers associated with the vertex are not unique. 1 (b) Verify that = ( 4 , 1, 0, - 1 , 0)T is a vector of Lagrange multipliers at x. Is x optimal? 4 Explain your answer. The product AT is given by:
a
1 0 AT = 0 a 1 0
1 2 3 4 0 2 1 1 0 1 0 = 2 = c. 2 2 -1 - 1 4 -1 0 0
Hence is a Lagrange multiplier. As the fourth component of is negative, does not is not unique. A short survey imply optimality. But from part (a), we know that of the basic solutions of the equation AT = c reveals that = (0, 1, 1, 0, 0)T is a a Lagrange multiplier also. As 0, we conclude that x is optimal.