ee227a_subgradient_lecture - EECS 227A: Convex Optimization...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EECS 227A: Convex Optimization Fall 2009 Lecture 1 — December 1, 2009 Lecturer: Martin Wainwright Scribe: Martin Wainwright 1.1 Motivation Thus far, we have explored a variety of optimization methods (e.g., generalized descent methods, Newton’s method, projected gradient methods), all of which apply to differentiable functions. However, many functions that arise in practice may be non-differentiable at certain places. A common example is the absolute value function f ( x ) = | x | , or its multivariate extension, the ℓ 1-norm f ( x ) = bardbl x bardbl 1 = ∑ n i =1 | x i | . In this lecture, we discuss a generalization of the gradient for non-differentiable convex functions. 1.2 Subgradients and subdifferentials In order to motivate the definition that follows, let us recall the tangent approximation in- terpretation of the gradient for a convex function. If f : R n → R is convex and differentiable, then the tangent plane inequality guarantees that f ( y ) ≥ f ( x ) + (∇ f ( x ) , y − x ) for all y ∈ dom( f ). (1.1) Geometrically, this inequality means that the tangent plane with normal vector ∇ f ( x ) sup- ports the epigraph of f at x . With this intuition, we have the following: Definition: A vector v ∈ R n is a subgradient of a convex function at x if f ( y ) ≥ f ( x ) + ( v, y − x ) for all y ∈ dom( f ). The set of all subgradients at x is called the subdifferential , and is denoted by ∂f ( x ). Remarks: (a) It can be shown that for a convex function and an element x ∈ dom( f ), the subdiffer- ential ∂f ( x ) is always a non-empty set. (See Appendix B of Bertsekas [1] for details of this argument, which uses the separating hyperplane theorem that we have covered.) From the definition, we also see that ∂f ( x ) must be a convex set, since any convex combination of subgradients is also a subgradient....
View Full Document

This note was uploaded on 09/03/2011 for the course EE 227A taught by Professor Staff during the Spring '11 term at Berkeley.

Page1 / 5

ee227a_subgradient_lecture - EECS 227A: Convex Optimization...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online