L24_kiwbun

L24_kiwbun - Lecture 24: Steepest Descent Subgradient...

Info iconThis preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Lecture 24: Steepest Descent Subgradient Methods April 18, 2007 Lecture 24 Outline -Subgradients and-Subdifferentals Bundle-Type Method using Cutting-Plane Idea Convex Optimization 1 Lecture 24-Subgradient and-Subdifferential An-subgradient is also referred to as approximate subgradient An-subgradient provides a linearization underestimating perturbed function f + instead of f Def. For a given > , a vector s is an-subgradient of f at x when f ( x ) + s T ( z- x ) f ( z ) + for all z dom f The-subdifferential of f at x is the set of all-subgradients of f at x , denoted by f ( x ) Convex Optimization 2 Lecture 24-Subgradients and-Optimality Theorem Let f be convex with dom f = R n . A vector x minimizes f approximately with error > over R n [i.e., f * f ( x ) f * + ] if and only if f ( x ) Proof : Let f ( x ) . Then, by the definition of-subgradient, we have f ( x ) f ( z ) + for all z R n By taking infimum over z R n , we obtain f ( x ) f * + . Let x be such that f ( x ) f * + . Then f ( x ) f ( z ) + for all z R n implying that f ( x ) . Convex Optimization 3 Lecture 24 Bundle Methods Based on Cutting-Plane Idea The basic idea is to form a convex piece-wise linear approximation of the function f The algorithm moves along directions resulting in Either sufficient descent Or improved bundle [of points and-subgradients] providing a better approximation of the function History : The first algorithm of such form was proposed and analyzed by Kiwiel 1985 [Generalized Cutting-plane] Subsequently, further developed by Kiwiel 1990, 1995 [Proximal Bundle] In parallel, a similar version was proposed by Schramm and Zowe 1990 [Bundle Trust Region] Convex Optimization 4 Lecture 24 Basic Steps in Bundle Method Consider unconstrained minimization of convex f with dom f = R n At a typical iteration, we have Current iterate x k and current trial point y k Current bundle B k =...
View Full Document

Page1 / 17

L24_kiwbun - Lecture 24: Steepest Descent Subgradient...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online