This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: EE364b Prof. S. Boyd EE364b Homework 2 1. Subgradient optimality conditions for nondifferentiable inequality constrained optimiza- tion. Consider the problem minimize f ( x ) subject to f i ( x ) ≤ , i = 1 , . . . , m, with variable x ∈ R n . We do not assume that f , . . . , f m are convex. Suppose that ˜ x and ˜ λ followsequal 0 satisfy primal feasibility, f i (˜ x ) ≤ , i = 1 , . . . , m, dual feasibility, ∈ ∂f (˜ x ) + m summationdisplay i =1 ˜ λ i ∂f i (˜ x ) , and the complementarity condition ˜ λ i f i (˜ x ) = 0 , i = 1 , . . . , m. Show that ˜ x is optimal, using only a simple argument, and definition of subgradient. Recall that we do not assume the functions f , . . . , f m are convex. Solution. Let g be defined by g ( x ) = f ( x ) + ∑ m i =1 ˜ λ i f i ( x ). Then, 0 ∈ ∂g (˜ x ). By definition of subgradient, this means that for any y , g ( y ) ≥ g (˜ x ) + 0 T ( y − ˜ x ) . Thus, for any y , f ( y ) ≥ f (˜ x ) − m summationdisplay i =1 ˜ λ i ( f i ( y ) − f i (˜ x )) . For each i , complementarity implies that either λ i = 0 or f i (˜ x ) = 0. Hence, for any feasible y (for which f i ( y ) ≤ 0), each ˜ λ i ( f i ( y ) − f i (˜ x )) term is either zero or negative. Therefore, any feasible y also satisfies f ( y ) ≥ f (˜ x ), and ˜ x is optimal. 2. Optimality conditions and coordinate-wise descent for ℓ 1-regularized minimization. We consider the problem of minimizing φ ( x ) = f ( x ) + λ bardbl x bardbl 1 , where f : R n → R is convex and differentiable, and λ ≥ 0. The number λ is the regularization parameter, and is used to control the trade-off between small f and small bardbl x bardbl 1 . When ℓ 1-regularization is used as a heuristic for finding a sparse x for which f ( x ) is small, λ controls (roughly) the trade-off between f ( x ) and the cardinality (number of nonzero elements) of...
View Full Document
- Fall '09
- Derivative, Monotonic function, Convex function, CVX, coordinate-wise descent