Unformatted text preview: Fall 2007 Problem Set #05 Answer key Second Graphical Problem Set ARE211 (1) Convexity and quasiconvexity: (a) Give a "diagrammatic proof" that if f is strictly convex then a local minimum of f is a strict global minimum. Make the argument that if f has a local minimum that isn't a strict global minimum, then f isn't strictly convex. Use the fact that f is strictly convex if the set above its graph is a "strictly convex" set.
f f f(x) f(y) Lower contour set of f Level set of f
x x' y Figure 1. Question 1a Ans: We'll first show that if f is strictly convex, then a local minimum of f is a strict local minimum. If not, then the graph of f must look like the left panel of Fig. 1, i.e., the set above the graph has a flat spot, so that it is not a strictly convex set. Now assume that f has a strict local minimum that isn't a strict global minimum, as in the right panel of Fig. 1. Well, (x, f (x)) and (y, f (y)) both live in the set (weakly) above the graph. For f to be concave, the entire linesegment joining (x, f (x)) and (y, f (y)) must also live in this set (in the interior of it, in fact). But since f attains a local minimum at x, it follows that for x sufficiently close to x, f (x ) > f (x), while since f (y) < f (x), the point vertically above x on the line segment joining (x, f (x)) and (y, f (y)) must be strictly below f (x). But this means that (x, ) does not belong to the set above the graph. (b) Now establish the same result for the case in which f is strictly quasiconvex. Make the argument that if f has a local minimum that isn't a strict global minimum, then f isn't strictly quasiconvex.
1 2 Ans: Once again, we'll first show that if f is strictly quasiconvex, then a local minimum of f is a strict local minimum. If not, then the level set of f corresponding to the locally minimized value of f must contain an interval (well, almost an interval, but the distinction is purely technical), as in the left panel of Fig. 1. But the level sets of a strictly quasiconvex function cannot be "thick", as in this case. Conclude that if f has a local minimum that is not a strict local minimum, then f is not strictly quasiconvex. Now assume, again, that f has a strict local minimum that isn't a strict global minimum, as in the right panel of Fig. 1. But in this case, the lowe contour set of f corresponding to the locally minimized value of f must include x plus an interval around y (as in the right hand panel), but cannot include points immediately to the right of x. Hence this lower contour set is not a convex set. Conclude that f cannot be quasiconvex, and hence cannot be strictly quasiconvex. (2) Consider the constrained optimization problem: minimize a continuously differentiable function f (x) s.t. x < 0. (a) In what way is this problem fundamentally different from the standard constrained optimization problem that we have been studying in lass? Ans: The fundamental difference is the "<" condition. In the p oblems we study in class, the feasible set is always closed. The other differences, i.e., "min," etc., can be rewritten to match our standard formulation. (b) Write down the necessary conditions for x to be a solution to this problem. Ans: Same as for an unconstrained max, i.e., f (x ) = 0. Since the feasible set has no boundary, we don't need to consider what happens on the boundary of the constraint. (c) Illustrate with a graphical example (i.e., construct an f ) with the property that at the unique solution to the problem, the constraint is satisfied with strict inequality and is linding but not binding (see lecture notes mathGraphical3 for a definition of linding). Explain carefully why the constraint is linding but not binding. The remaining parts of this question refer to the following modification of the original problem: minimize a continuously differentiable function f (x) s.t. x 0. (d) How does this change your answer to (b)? 3 (e) Show that you can now no longer construct an example comparable to the one you constructed in (c). Explain carefully. (f) Now answer the following slight variant of (c). Illustrate with a graphical example (i.e., construct an f ) with the property that at a solution to the problem, the constraint is satisfied with strict inequality but is nonetheless binding. (g) Write down the weakest condition on f you can find which guarantees that if x is a solution to this problem, and at x the constraint is satisfied with strict inequality, then the constraint cannot be binding. Ans: The condition is that f is strictly quasiconvex. Strict convexity guarantees the property also, but stronger than quasiconvexity. Regular quasiconvexity doesn't guarantee the property. (3) Consider the problem max f (x) s.t. x = 0, where x R2 . In this question you will be constructing functions that satisfy certain properties. In each case, choose an affine function. (Recall that a function f : Rn R is affine if f (x) = a + b x, for some a R and b Rn .) (a) This problem can be written as max f (x) s.t. gi (x) bi , for i = 1, ...4. Write down four affine gi 's and corresponding bi 's for this problem. Ans: g1 (x) = x1 ; g2 (x) = x1 ; g3 (x) = x2 ; g4 (x) = x2 . b = (0, 0, 0, 0). (b) Graph the constraint set as the intersection of four lower contour sets. Ans: See Fig. 2. 4 (d) Pick one of the f i 's you've constructed in 3c, indicate graphically what happens when (c) For each gi in your answer to 3a, construct a function f i such that at the solution to Ans: We'll take f 1 (x) = x1 . The graphs are in the four panels of Fig. 3. In each panel, the feasible set is indicated by a thick solid line, whil the gradient of f 1 is indicated by a thick dashed arrow, based at the solution (or possibly, one of the several solutions) to the problem. In the top left panel, we relax the constraint g1 . The feasible set expands from a point at zero to a line segment whose right boundary is zero. Conclude that g1 is binding, because relaxing the constraint results in an increase in the maximized value of f 1 . When g2 is relaxed, the constraint again expands from a point to a line segment, but the segment now lies to the right of zero. Since f 1 increases to the left, this doesn't help at all. When either g3 or g4 is relaxed, the resulting segment is parallel to the level set of f 1 through zero. Now there are multiple solutions but the original solution at zero remains a solution. Thus, for i = 2, 3, 4, relaxing gi doesn't increase the maximized value of f i . Conclude that gi is satisfied with equality but not binding. Ans: f 1 (x) = x1 ; f 2 (x) = x1 ; f 3 (x) = x2 ; f 4 (x) = x2 . to the feasible set and to the maximized value of f i when you do so. Explain to the you relax each of the constraints in turn. Your picture should indicate what happens the problem, constraint gi is binding, and the other three constraints are satisfied with {x : g2 (x) 0} equality but are not binding. reader how to conclude from the pictures that constraint gi is binding and the others Figure 2. Question 3 are not. {x : g1 (x) 0} {x : g3 (x) 0} {x : g4 (x) 0} 1 0 1 0 1 0 ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 1 0 1 0 1 0 ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 1 0 1 0 1 0 ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 1 0 1 0 1 0 ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 1 0 1 0 1 0 ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 1 0 1 0 1 0 ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 1 0 1 0 1 0 ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 1 0 1 0 1 0 ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 ) ( ) ( ) ( 1 0 ) ( 1 0 1 0 1 0 1 0 {x : g2 (x) 0} {x : g2 (x) 0} ) ( ) ( 1 0 ) ( 1 0 ) ( 1 0 {x : g3 (x) 0} 1 0 1 0 1 0 {x : g4 (x) 0} {x : g2 (x) 0} (e) For the f i you considered in 3d, compute the rate at which f i increases as the i'th (f) Now construct an f such that two of the four constraints are binding, but one of them constraint is relaxed (i.e., how much does f i go up when the i'th constraint is relaxed ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & " " " " " " " " " " " ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & # # # # # # # # # # # " " " " " " " " " " " ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & # # # # # # # # # # # " " " " " " " " " " " ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & # # # # # # # # # # # " " " " " " " " " " " # # # # # # # # # # # " " " " " " " " " " " # # # # # # " " " " " " " " " " " # # # # # # # # # # # {x : g1 (x) 0} {x : g2 (x) 0} {x : g3 (x) 0} # # # # # {x : g4 (x) 0} ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & ' & {x : g1 (x) 0} ! ! ! ! ! ! ! ! ! ! % $ ! % $ ! % $ ! % $ ! % $ ! % $ ! % $ ! % $ ! % $ % $ % $ % $ % $ % $ % $ % $ % $ % $ by one unit?). Ans: f (x) = x1 + x2 , where Ans: When g1 is relaxed by one unit, the constraint becomes g1 (x) 1, i.e., x1 1 or x1 1. The solution to the maximization problem moves from (0, 0) to (1, 0) and so the maximized value of the solution increases by 1. The rate at which f 1 increases as the constraint is relaxed is thus one. ! ! ! ! ! ! ! ! ! ! % $ ! % $ ! % $ ! % $ ! % $ ! % $ ! % $ ! % $ ! % $ % $ % $ % $ % $ % $ % $ % $ % $ % $ binding. is "much more binding" than the other. Indicate which of the two constraints are (i) For each of the two constraints, compute the rate at which f increases as the con is much more binding than the other. straint is relaxed, and using these rates, explain the sense in which one constraint Figure 3. Question 3d > 0 is small. Constraints g2 and g4 are binding {x : g1 (x) 0} {x : g1 (x) 0} {x : g4 (x) 0} {x : g4 (x) 0} {x : g3 (x) 0} {x : g3 (x) 0}
5 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! % $ ! % $ ! % $ ! % $ ! % $ $ ! % $ ! % $ ! % $ ! % $ ! % $ $ ! % $ ! % $ ! % $ ! % $ ! % $ $ % $ % $ % $ % $ % $ $ % $ % $ % $ % $ % $ $ % $ % $ % $ % $ % $ $ 6 Ans: When g2 is relaxed by one unit, the solution becomes (1, 0) and f increases by units. Thus the rate of increase is . When g4 is relaxed by one unit, the solution becomes (0, 1) and f increases by one unit. Thus the rate of increase is one. g4 is much more binding than g2 in the sense that as g4 is relaxed f increases at a faster rate. The owner of f would "pay more" to have g4 relaxed than she would pay to have g2 relaxed. (ii) Illustrate graphically, by relaxing each constraint and drawing exactly three level sets, that one constraint is much more binding than the other. Relaxing g4 Figure 4. Question 3(f)ii Relaxing g2 Ans: In Fig. 4, when g4 is relaxed, the level set through the solution shifts significantly; when g2 is relaxed by a comparable amount, the level set through the solution shifts barely at all. (4) The Mantra: Consider the problem: max f (x) s.t. x 0 and b1 x1 + b2 x2 , b2 R. In parts 4a and 4b, set b1 = 1. (a) Pick a value of b2 > 1 and then Ans: I've set b2 = 2. 2 i=1 xi 1, where f (x) = (i) Graph the problem, indicating the constraint set and the level set of f corresponding to the value of f at the solution. Draw the gradient of f and the gradients of the two constraints that are satisfied with equality at the solution. {x : g1 (x) 0} {x : g1 (x) 0} {x : g4 (x) 0} {x : g3 (x) {x : g2 (x) 0} 0} {x : g2 (x) 0} {x : g3 (x) 0} {x : g4 (x) 0} 7 x2 gradient of f gradient of 2 i=1 xi constraint cone generated by constraint gradients grad of x1 0 constraint 1 Figure 5. Question 4(a)i Ans: See Fig. 5 (ii) Are both parts of the mantra satisfied? Explain. Ans: Both the 2 xi constraint and the nonnegativity constraint on x1 are bindi=1 ing. (The former is obviously binding. To see that the latter is binding also, note that if it were relaxed, you could move to the higher, dashed, level set of f . The gradient of f lies in the positive cone generated by the gradients of the two binding constraints. It also lies in the nonnegative cone generated by these same constraints, both of which are satisfied with equality. (iii) What can you say about the relationship between the size of b2 and the location of the gradient of f relative to the edges of the cone generated by the two constraints that are satisfied with equality at the solution? Ans: If b2 1, the gradient of f is close to the right hand edge of the cone. As b2 increases, the gradient of f becomes steeper and steeper and the gradient vector into the middle of the cone. level set of f 1 x1 8 (iv) What is the connection between the relationship identified in 4(a)iii and the "relative bindingness" of the two constraints? Ans: The closer is b2 to one, the "less binding" is the nonnegativity constraint relative to the summation constraint. That is, a one unit relaxation in the nonnegativity constraint doesn't buy you much. As you increase b2 and thus swivel the gradient of f counterclockwise, the rate at which you gain utility by substituting x2 for x1 increases, so the nonnegativity constraint becomes more "costly" (b) Now set b2 = 1 (i) Consider the solution you identified in part 4a. Repeat parts 4(a)i and 4(a)ii for this case. level set of f x2 gradient of f gradient of 2 xi constraint i=1 cone generated by constraint gradients grad of x1 0 constraint 1 Ans: See Fig. 6. Observe that in this case the nonnegativity constraint is satisfied with equality but is not binding: if it is relaxed, additional solutions to the problem become available, but none of them increase the solution value of f . The gradient of f is a positive scalar multiple of the gradient on the 2 xi constraint, and hence i=1 belongs to the positive cone generated by this vector, i.e., the gradient vector of the 1
Figure 6. Question 4b x1 9 unique binding constraint. The gradient of f also belongs to the nonnegative cone generated by the gradients of the two constraints that are satisfied with equality. (ii) Identify a different solution to the problem for which a different set of two constraints are satisfied with equality. Verify graphically that the gradient of f belongs to the nonnegative cone generated by the two constraints that were satisfied with equality in subpart 4(b)i. Does this contradict the mantra? If not, why not? Ans: See Fig. 6 again. The new solution is at the bottom right corner of the triangle. Once again, the unique binding constraint is the constraint on 2 xi . i=1 Observe that since the gradient at this point is parallel to the gradient at our original solution, it belongs to the same cones as the original one did. (The "trick" here is that you can move gradient vectors around whereever you like; their "starting points" are immaterial; what matters is their lengths and directions.) Conclude that the gradient of f belongs to the nonnegative cone generated by the gradients of two constraints, one of which is satisfied with strict inequality. This does not violate the mantra however. The mantra merely specifies that the gradient of f must belong to two particular cones; it is silent about all the other cones that this gradient may belong to. (c) Now relax the condition that b1 = 1. (i) For what values of (b1 , b2 ) would the set of solutions to the problem differ, depending on whether the constraint were
2 i=1 xi 1, vs 2 i=1 xi = 1? Ans: The set of solutions will differ iff bi 0, i = 1, 2. If, to the contrary, bi > 0 for some i, then if x is feasible but not on the northeast frontier of the constraint, f can always be increased by increasing xi holding xj constant. Since under these conditions, there will never be a solution such that 2 xi < 1, the solution set i=1 is the same under the two constraints. On the other hand, if bi 0, i = 1, 2, then there will always be at least one solutionspecifically (0, 0)to the problem requiring that 2 xi 1 which will not be a solution to the problem requiring i=1 that 2 xi = 1. i=1 (ii) Pick some vector (b1 , b2 ) R2 such that the set of solutions to the problem does depend on whether the constraint is
2 i=1 xi 1, or 2 i=1 xi = 1, but in each case the problem has a unique solution. On the same graph, indicate the solution 10 to the problem for both of these cases. In each case, verify that the mantra is satisfied, and explain in terms of the mantra the difference between the two cases. x2 1 1 x1 gradient of f grad of  2 i=1 xi constraint Figure 7. Question 4(c)ii Ans: Set b1 = 1, b2 = 2. If the constraint is 2 xi = 1, then for these values i=1 of b, the constraint that  2 xi 1 is binding. Once the equality is replaced i=1 by an inequality, the feasible set increases and f attains a higher optimum at zero. ...
View
Full
Document
This note was uploaded on 08/01/2008 for the course ARE 211 taught by Professor Simon during the Fall '07 term at Berkeley.
 Fall '07
 Simon

Click to edit the document details