hw2sol

# hw2sol - EE364b Prof S Boyd EE364b Homework 2 1 Subgradient...

This preview shows pages 1–2. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE364b Prof. S. Boyd EE364b Homework 2 1. Subgradient optimality conditions for nondifferentiable inequality constrained optimiza- tion. Consider the problem minimize f ( x ) subject to f i ( x ) ≤ , i = 1 , . . . , m, with variable x ∈ R n . We do not assume that f , . . . , f m are convex. Suppose that ˜ x and ˜ λ followsequal 0 satisfy primal feasibility, f i (˜ x ) ≤ , i = 1 , . . . , m, dual feasibility, ∈ ∂f (˜ x ) + m summationdisplay i =1 ˜ λ i ∂f i (˜ x ) , and the complementarity condition ˜ λ i f i (˜ x ) = 0 , i = 1 , . . . , m. Show that ˜ x is optimal, using only a simple argument, and definition of subgradient. Recall that we do not assume the functions f , . . . , f m are convex. Solution. Let g be defined by g ( x ) = f ( x ) + ∑ m i =1 ˜ λ i f i ( x ). Then, 0 ∈ ∂g (˜ x ). By definition of subgradient, this means that for any y , g ( y ) ≥ g (˜ x ) + 0 T ( y − ˜ x ) . Thus, for any y , f ( y ) ≥ f (˜ x ) − m summationdisplay i =1 ˜ λ i ( f i ( y ) − f i (˜ x )) . For each i , complementarity implies that either λ i = 0 or f i (˜ x ) = 0. Hence, for any feasible y (for which f i ( y ) ≤ 0), each ˜ λ i ( f i ( y ) − f i (˜ x )) term is either zero or negative. Therefore, any feasible y also satisfies f ( y ) ≥ f (˜ x ), and ˜ x is optimal. 2. Optimality conditions and coordinate-wise descent for ℓ 1-regularized minimization. We consider the problem of minimizing φ ( x ) = f ( x ) + λ bardbl x bardbl 1 , where f : R n → R is convex and differentiable, and λ ≥ 0. The number λ is the regularization parameter, and is used to control the trade-off between small f and small bardbl x bardbl 1 . When ℓ 1-regularization is used as a heuristic for finding a sparse x for which f ( x ) is small, λ controls (roughly) the trade-off between f ( x ) and the cardinality (number of nonzero elements) of...
View Full Document

{[ snackBarMessage ]}

### Page1 / 5

hw2sol - EE364b Prof S Boyd EE364b Homework 2 1 Subgradient...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online