This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EECS 227A: Convex Optimization Fall 2009 Lecture 1 — December 1, 2009 Lecturer: Martin Wainwright Scribe: Martin Wainwright 1.1 Motivation Thus far, we have explored a variety of optimization methods (e.g., generalized descent methods, Newton’s method, projected gradient methods), all of which apply to differentiable functions. However, many functions that arise in practice may be nondifferentiable at certain places. A common example is the absolute value function f ( x ) =  x  , or its multivariate extension, the ℓ 1norm f ( x ) = bardbl x bardbl 1 = ∑ n i =1  x i  . In this lecture, we discuss a generalization of the gradient for nondifferentiable convex functions. 1.2 Subgradients and subdifferentials In order to motivate the definition that follows, let us recall the tangent approximation in terpretation of the gradient for a convex function. If f : R n → R is convex and differentiable, then the tangent plane inequality guarantees that f ( y ) ≥ f ( x ) + (∇ f ( x ) , y − x ) for all y ∈ dom( f ). (1.1) Geometrically, this inequality means that the tangent plane with normal vector ∇ f ( x ) sup ports the epigraph of f at x . With this intuition, we have the following: Definition: A vector v ∈ R n is a subgradient of a convex function at x if f ( y ) ≥ f ( x ) + ( v, y − x ) for all y ∈ dom( f ). The set of all subgradients at x is called the subdifferential , and is denoted by ∂f ( x ). Remarks: (a) It can be shown that for a convex function and an element x ∈ dom( f ), the subdiffer ential ∂f ( x ) is always a nonempty set. (See Appendix B of Bertsekas [1] for details of this argument, which uses the separating hyperplane theorem that we have covered.) From the definition, we also see that ∂f ( x ) must be a convex set, since any convex combination of subgradients is also a subgradient....
View
Full
Document
This note was uploaded on 09/03/2011 for the course EE 227A taught by Professor Staff during the Spring '11 term at Berkeley.
 Spring '11
 Staff

Click to edit the document details