This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Subgradients S. Boyd and L. Vandenberghe Notes for EE364b, Stanford University, Winter 200607 April 13, 2008 1 Definition We say a vector g ∈ R n is a subgradient of f : R n → R at x ∈ dom f if for all z ∈ dom f , f ( z ) ≥ f ( x ) + g T ( z − x ) . (1) If f is convex and differentiable, then its gradient at x is a subgradient. But a subgradient can exist even when f is not differentiable at x , as illustrated in figure 1. The same example shows that there can be more than one subgradient of a function f at a point x . There are several ways to interpret a subgradient. A vector g is a subgradient of f at x if the affine function (of z ) f ( x ) + g T ( z − x ) is a global underestimator of f . Geometrically, g is a subgradient of f at x if ( g, − 1) supports epi f at ( x, f ( x )), as illustrated in figure 2. A function f is called subdifferentiable at x if there exists at least one subgradient at x . The set of subgradients of f at the point x is called the subdifferential of f at x , and is denoted ∂f ( x ). A function f is called subdifferentiable if it is subdifferentiable at all x ∈ dom f . Example. Absolute value. Consider f ( z ) =  z  . For x < 0 the subgradient is unique: ∂f ( x ) = {− 1 } . Similarly, for x > 0 we have ∂f ( x ) = { 1 } . At x = 0 the subdifferential is defined by the inequality  z  ≥ gz for all z , which is satisfied if and only if g ∈ [ − 1 , 1]. Therefore we have ∂f (0) = [ − 1 , 1]. This is illustrated in figure 3. 2 Basic properties The subdifferential ∂f ( x ) is always a closed convex set, even if f is not convex. This follows from the fact that it is the intersection of an infinite set of halfspaces: ∂f ( x ) = intersectiondisplay z ∈ dom f { g  f ( z ) ≥ f ( x ) + g T ( z − x ) } . 1 x 1 x 2 f ( x 1 ) + g T 1 ( z − x 1 ) f ( x 2 ) + g T 2 ( z − x 2 ) f ( x 2 ) + g T 3 ( z − x 2 ) f ( z ) Figure 1: At x 1 , the convex function f is differentiable, and g 1 (which is the derivative of f at x 1 ) is the unique subgradient at x 1 . At the point x 2 , f is not differentiable. At this point, f has many subgradients: two subgradients, g 2 and g 3 , are shown. epi f ( g, − 1) Figure 2: A vector g ∈ R n is a subgradient of f at x if and only if ( g, − 1) defines a supporting hyperplane to epi f at ( x, f ( x )). f ( z ) =  z  ∂f ( x ) z x 1 − 1 Figure 3: The absolute value function (left), and its subdifferential ∂f ( x ) as a function of x (right). 2 2.1 Existence of subgradients If f is convex and x ∈ intdom f , then ∂f ( x ) is nonempty and bounded. To establish that ∂f ( x ) negationslash = ∅ , we apply the supporting hyperplane theorem to the convex set epi f at the boundary point ( x, f ( x )), to conclude the existence of a ∈ R n and b ∈ R , not both zero, such that bracketleftBigg a b bracketrightBigg T parenleftBiggbracketleftBigg z t bracketrightBigg − bracketleftBigg x f ( x ) bracketrightBiggparenrightBigg...
View
Full
Document
This note was uploaded on 04/09/2010 for the course EE 364B at Stanford.
 '09
 BOYD,S

Click to edit the document details