L20_sgdopt

L20_sgdopt - Lecture 20: Subgradients and Optimality...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture 20: Subgradients and Optimality Conditions April 4, 2007
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Lecture 20 Outline Subgradients for Concave Problems Computing Subgradients in Dual Problems Optimality Conditions Cutting-Plane Methods Convex Optimization 1
Background image of page 2
Lecture 20 Subgradients of a Concave Function For a concave function h ( y ) : the vector s is a subgradient of h at ˆ y if and only if - s is a subgradient of the (convex) function - h ( y ) at ˆ y Def. A vector s is a subgradient of a concave function h at ˆ y dom ( h ) when h ( y ) h y ) + s T ( y - ˆ y ) for all y dom h For a concave function: a subgradient provides a linearization that over-estimates the function over its domain A subdifferential ∂h y ) is nonempty, convex, and compact for every ˆ y int ( dom h ) Subdifferential operations: scaling , sum , and composition with affine mapping are the same as for the subdifferentials of a convex function Min-type (Inf-type) concave functions: using convex - h , the rules reduce to Max-type (Sup-type) convex functions Convex Optimization 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Lecture 20 Computing Subgradients in Dual Problems Consider minimize f ( x ) subject to g j ( x ) 0 , j = 1 , . . . , m x X The dual function q ( μ ) = inf x X { f ( x ) + μ T g ( x ) } is concave Let ˆ μ be such that q μ ) is finite [i.e., ˆ μ dom q ] Let x ˆ μ X be a vector such that f ( x ˆ μ ) + ˆ μ T g ( x ˆ μ ) = inf x X { f ( x ) + ˆ μ T g ( x ) } = q μ ) Then the vector g ( x ˆ μ ) is a subgradient of q at ˆ μ To see this, note: q ( μ ) f ( x ˆ μ ) + μ T g ( x ˆ μ ) = q μ ) + ( μ - ˆ μ ) T g ( x ˆ μ ) for all μ dom q Convex Optimization 3
Background image of page 4
Lecture 20 Thus, for the dual function q : For a vector μ such that q ( μ ) is finite Let X μ be the set of vectors in X attaining the infimum in q ( μ ) : X μ = ± ˜ x X | f x ) + μ T g x ) = inf x X { f ( x ) + μ T g ( x ) } ² For the subdifferential ∂q ( μ ) , we have conv ( { g ( x ) | x
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 17

L20_sgdopt - Lecture 20: Subgradients and Optimality...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online