lecture1 notes

Interior point methods subgradientincremental methods

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: hms − Applications • New problem paradigms with rich applications • Duality-based decomposition − Large-scale resource allocation − Lagrangian relaxation, discrete optimization − Stochastic programming • Conic programming − Robust optimization − Semidefinite programming • Machine learning − Support vector machines − l1 regularization/Robust regression/Compressed sensing 15 METHODOLOGICAL TRENDS • New methods, renewed interest in old methods. − Interior point methods − Subgradient/incremental methods − Polyhedral approximation/cutting plane methods − Regularization/proximal methods − Incremental methods • Renewed emphasis on complexity analysis − Nesterov, Nemirovski, and others ... − “Optimal algorithms” (e.g., extrapolated gradient methods) • Emphasis on interesting (often duality-related) large-scale special structures 16 COURSE OUTLINE • We will follow closely the textbook − Bertsekas, “Convex Optimization Theory,” Athena Scientific, 2009, including the on-line Chapter 6 and supplementary material at http://www.athenasc.com/convexduality.html • Additional book references: − Rockafellar, “Convex Analysis,” 1970. − Boyd and Vanderbergue, “Convex Optimization,” Cambridge U. Press, 2004. (On-line at http://www.stanford.edu/~boyd/cvxbook/) − Bertsekas, Nedic, and Ozdaglar, “Convex Analysis and Optimization,” Ath. Scientific, 2003. • Topics (the text’s design is modular, and the following sequence involves no loss of continuity): − Basic Convexity Concepts: Sect. 1.1-1.4. − Convexity and Optimization: Ch. 3. − Hyperplanes & Conjugacy: Sect. 1.5, 1.6. − Polyhedral Convexity: Ch. 2. − Geometric Duality Framework: Ch. 4. − Duality Theory: Sect. 5.1-5.3. − Subgradients: Sect. 5.4. − Algorithms: Ch. 6. 17 WHAT TO EXPECT FROM THIS COURSE • Requirements: Homework (25%), midterm (25%), and a term paper (50%) • We aim: − To develop insight and deep understanding of a fundamental optimization topic − To treat with mathematical rigor an important branch of methodological research, and to provide an account of the state of the art in the field − To get an understanding of the merits, limitations, and characteristics of the rich set of available algorithms • Mathematical level: − Prerequisites are linear algebra (preferably abstract) and real analysis (a course in each) − Proofs will matter ... but the rich geometry of the subject helps guide the mathematics • Applications: − They are many and pervasive ... but don’t expect much in this course. The book by Boyd and Vandenberghe describes a lot of practical convex optimization models − You can do your term paper on an application area 18 A NOTE ON THESE SLIDES • These slides are a teaching aid, not a text • Don’t expect a rigorous mathematical development • The statements of theorems are fairly precise, but the proofs are not • Many proofs have been omitted or greatly abbreviated • Figures are meant to convey and enhance understanding of ideas, not to express them precisely • The omitted proofs and a fuller discussion can be found in the “Convex Optimization Theory” textbook and its supplementary material 19...
View Full Document

This document was uploaded on 03/19/2014 for the course EECS 6.253 at MIT.

Ask a homework question - tutors are online