hw2sol - EE364b Prof. S. Boyd EE364b Homework 2 1....

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE364b Prof. S. Boyd EE364b Homework 2 1. Subgradient optimality conditions for nondifferentiable inequality constrained optimiza- tion. Consider the problem minimize f ( x ) subject to f i ( x ) , i = 1 , . . . , m, with variable x R n . We do not assume that f , . . . , f m are convex. Suppose that x and followsequal 0 satisfy primal feasibility, f i ( x ) , i = 1 , . . . , m, dual feasibility, f ( x ) + m summationdisplay i =1 i f i ( x ) , and the complementarity condition i f i ( x ) = 0 , i = 1 , . . . , m. Show that x is optimal, using only a simple argument, and definition of subgradient. Recall that we do not assume the functions f , . . . , f m are convex. Solution. Let g be defined by g ( x ) = f ( x ) + m i =1 i f i ( x ). Then, 0 g ( x ). By definition of subgradient, this means that for any y , g ( y ) g ( x ) + 0 T ( y x ) . Thus, for any y , f ( y ) f ( x ) m summationdisplay i =1 i ( f i ( y ) f i ( x )) . For each i , complementarity implies that either i = 0 or f i ( x ) = 0. Hence, for any feasible y (for which f i ( y ) 0), each i ( f i ( y ) f i ( x )) term is either zero or negative. Therefore, any feasible y also satisfies f ( y ) f ( x ), and x is optimal. 2. Optimality conditions and coordinate-wise descent for 1-regularized minimization. We consider the problem of minimizing ( x ) = f ( x ) + bardbl x bardbl 1 , where f : R n R is convex and differentiable, and 0. The number is the regularization parameter, and is used to control the trade-off between small f and small bardbl x bardbl 1 . When 1-regularization is used as a heuristic for finding a sparse x for which f ( x ) is small, controls (roughly) the trade-off between f ( x ) and the cardinality (number of nonzero elements) of...
View Full Document

Page1 / 5

hw2sol - EE364b Prof. S. Boyd EE364b Homework 2 1....

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online