This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: EE364b Prof. S. Boyd EE364b Homework 2 1. Subgradient optimality conditions for nondifferentiable inequality constrained optimiza- tion. Consider the problem minimize f ( x ) subject to f i ( x ) , i = 1 , . . . , m, with variable x R n . We do not assume that f , . . . , f m are convex. Suppose that x and followsequal 0 satisfy primal feasibility, f i ( x ) , i = 1 , . . . , m, dual feasibility, f ( x ) + m summationdisplay i =1 i f i ( x ) , and the complementarity condition i f i ( x ) = 0 , i = 1 , . . . , m. Show that x is optimal, using only a simple argument, and definition of subgradient. Recall that we do not assume the functions f , . . . , f m are convex. Solution. Let g be defined by g ( x ) = f ( x ) + m i =1 i f i ( x ). Then, 0 g ( x ). By definition of subgradient, this means that for any y , g ( y ) g ( x ) + 0 T ( y x ) . Thus, for any y , f ( y ) f ( x ) m summationdisplay i =1 i ( f i ( y ) f i ( x )) . For each i , complementarity implies that either i = 0 or f i ( x ) = 0. Hence, for any feasible y (for which f i ( y ) 0), each i ( f i ( y ) f i ( x )) term is either zero or negative. Therefore, any feasible y also satisfies f ( y ) f ( x ), and x is optimal. 2. Optimality conditions and coordinate-wise descent for 1-regularized minimization. We consider the problem of minimizing ( x ) = f ( x ) + bardbl x bardbl 1 , where f : R n R is convex and differentiable, and 0. The number is the regularization parameter, and is used to control the trade-off between small f and small bardbl x bardbl 1 . When 1-regularization is used as a heuristic for finding a sparse x for which f ( x ) is small, controls (roughly) the trade-off between f ( x ) and the cardinality (number of nonzero elements) of...
View Full Document
- Fall '09