01-subgradients_notes

01-subgradients_notes - Subgradients S Boyd and L...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Subgradients S. Boyd and L. Vandenberghe Notes for EE364b, Stanford University, Winter 2006-07 April 13, 2008 1 Definition We say a vector g ∈ R n is a subgradient of f : R n → R at x ∈ dom f if for all z ∈ dom f , f ( z ) ≥ f ( x ) + g T ( z − x ) . (1) If f is convex and differentiable, then its gradient at x is a subgradient. But a subgradient can exist even when f is not differentiable at x , as illustrated in figure 1. The same example shows that there can be more than one subgradient of a function f at a point x . There are several ways to interpret a subgradient. A vector g is a subgradient of f at x if the affine function (of z ) f ( x ) + g T ( z − x ) is a global underestimator of f . Geometrically, g is a subgradient of f at x if ( g, − 1) supports epi f at ( x, f ( x )), as illustrated in figure 2. A function f is called subdifferentiable at x if there exists at least one subgradient at x . The set of subgradients of f at the point x is called the subdifferential of f at x , and is denoted ∂f ( x ). A function f is called subdifferentiable if it is subdifferentiable at all x ∈ dom f . Example. Absolute value. Consider f ( z ) = | z | . For x < 0 the subgradient is unique: ∂f ( x ) = {− 1 } . Similarly, for x > 0 we have ∂f ( x ) = { 1 } . At x = 0 the subdifferential is defined by the inequality | z | ≥ gz for all z , which is satisfied if and only if g ∈ [ − 1 , 1]. Therefore we have ∂f (0) = [ − 1 , 1]. This is illustrated in figure 3. 2 Basic properties The subdifferential ∂f ( x ) is always a closed convex set, even if f is not convex. This follows from the fact that it is the intersection of an infinite set of halfspaces: ∂f ( x ) = intersectiondisplay z ∈ dom f { g | f ( z ) ≥ f ( x ) + g T ( z − x ) } . 1 x 1 x 2 f ( x 1 ) + g T 1 ( z − x 1 ) f ( x 2 ) + g T 2 ( z − x 2 ) f ( x 2 ) + g T 3 ( z − x 2 ) f ( z ) Figure 1: At x 1 , the convex function f is differentiable, and g 1 (which is the derivative of f at x 1 ) is the unique subgradient at x 1 . At the point x 2 , f is not differentiable. At this point, f has many subgradients: two subgradients, g 2 and g 3 , are shown. epi f ( g, − 1) Figure 2: A vector g ∈ R n is a subgradient of f at x if and only if ( g, − 1) defines a supporting hyperplane to epi f at ( x, f ( x )). f ( z ) = | z | ∂f ( x ) z x 1 − 1 Figure 3: The absolute value function (left), and its subdifferential ∂f ( x ) as a function of x (right). 2 2.1 Existence of subgradients If f is convex and x ∈ intdom f , then ∂f ( x ) is nonempty and bounded. To establish that ∂f ( x ) negationslash = ∅ , we apply the supporting hyperplane theorem to the convex set epi f at the boundary point ( x, f ( x )), to conclude the existence of a ∈ R n and b ∈ R , not both zero, such that bracketleftBigg a b bracketrightBigg T parenleftBiggbracketleftBigg z t bracketrightBigg − bracketleftBigg x f ( x ) bracketrightBiggparenrightBigg...
View Full Document

This note was uploaded on 04/09/2010 for the course EE 364B at Stanford.

Page1 / 7

01-subgradients_notes - Subgradients S Boyd and L...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online