This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: EECS 227A: Convex Optimization Fall 2009 Lecture 1 — December 1, 2009 Lecturer: Martin Wainwright Scribe: Martin Wainwright 1.1 Motivation Thus far, we have explored a variety of optimization methods (e.g., generalized descent methods, Newton’s method, projected gradient methods), all of which apply to differentiable functions. However, many functions that arise in practice may be non-differentiable at certain places. A common example is the absolute value function f ( x ) = | x | , or its multivariate extension, the ℓ 1-norm f ( x ) = bardbl x bardbl 1 = ∑ n i =1 | x i | . In this lecture, we discuss a generalization of the gradient for non-differentiable convex functions. 1.2 Subgradients and subdifferentials In order to motivate the definition that follows, let us recall the tangent approximation in- terpretation of the gradient for a convex function. If f : R n → R is convex and differentiable, then the tangent plane inequality guarantees that f ( y ) ≥ f ( x ) + (∇ f ( x ) , y − x ) for all y ∈ dom( f ). (1.1) Geometrically, this inequality means that the tangent plane with normal vector ∇ f ( x ) sup- ports the epigraph of f at x . With this intuition, we have the following: Definition: A vector v ∈ R n is a subgradient of a convex function at x if f ( y ) ≥ f ( x ) + ( v, y − x ) for all y ∈ dom( f ). The set of all subgradients at x is called the subdifferential , and is denoted by ∂f ( x ). Remarks: (a) It can be shown that for a convex function and an element x ∈ dom( f ), the subdiffer- ential ∂f ( x ) is always a non-empty set. (See Appendix B of Bertsekas  for details of this argument, which uses the separating hyperplane theorem that we have covered.) From the definition, we also see that ∂f ( x ) must be a convex set, since any convex combination of subgradients is also a subgradient....
View Full Document
This note was uploaded on 09/03/2011 for the course EE 227A taught by Professor Staff during the Spring '11 term at Berkeley.
- Spring '11