This preview shows pages 1–7. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Lecture 24: Steepest Descent Subgradient Methods April 18, 2007 Lecture 24 Outline Subgradients andSubdifferentals BundleType Method using CuttingPlane Idea Convex Optimization 1 Lecture 24Subgradient andSubdifferential Ansubgradient is also referred to as approximate subgradient Ansubgradient provides a linearization underestimating perturbed function f + instead of f Def. For a given > , a vector s is ansubgradient of f at x when f ( x ) + s T ( z x ) f ( z ) + for all z dom f Thesubdifferential of f at x is the set of allsubgradients of f at x , denoted by f ( x ) Convex Optimization 2 Lecture 24Subgradients andOptimality Theorem Let f be convex with dom f = R n . A vector x minimizes f approximately with error > over R n [i.e., f * f ( x ) f * + ] if and only if f ( x ) Proof : Let f ( x ) . Then, by the definition ofsubgradient, we have f ( x ) f ( z ) + for all z R n By taking infimum over z R n , we obtain f ( x ) f * + . Let x be such that f ( x ) f * + . Then f ( x ) f ( z ) + for all z R n implying that f ( x ) . Convex Optimization 3 Lecture 24 Bundle Methods Based on CuttingPlane Idea The basic idea is to form a convex piecewise linear approximation of the function f The algorithm moves along directions resulting in Either sufficient descent Or improved bundle [of points andsubgradients] providing a better approximation of the function History : The first algorithm of such form was proposed and analyzed by Kiwiel 1985 [Generalized Cuttingplane] Subsequently, further developed by Kiwiel 1990, 1995 [Proximal Bundle] In parallel, a similar version was proposed by Schramm and Zowe 1990 [Bundle Trust Region] Convex Optimization 4 Lecture 24 Basic Steps in Bundle Method Consider unconstrained minimization of convex f with dom f = R n At a typical iteration, we have Current iterate x k and current trial point y k Current bundle B k =...
View
Full
Document
 Spring '07
 AngeliaNedich

Click to edit the document details