This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: MTH 432/532 LECTURE NOTES Spring 2009 INTRODUCTION TO LINEAR AND NONLINEAR PROGRAMMING S. Wright Department of Mathematics & Statistics Miami University, Oxford, OH c 2008, 2009 Table of Contents 1. INTRODUCTION 1 1.1 Terminology 2 1.2 Some Facts about Calculus, Vectors, and Matrices 5 1.3 Norms and Neighborhoods 7 1.4 Convergence, Continuity, Closedness and Boundedness 10 2. UNCONSTRAINED OPTIMIZATION 13 2.1 FirstOrder Necessary Conditions for Optimality 13 2.2 Sufficient Conditions for Optimality — a First Look 15 2.3 Definiteness and Quadratic Forms 19 2.4 Back to Optimization — Sufficient Conditions 26 3. EXISTENCE OF GLOBAL OPTIMIZERS 29 3.1 Extreme Value Theory 29 3.2 Verifying Coercivity 32 4. CONVEXITY AND OPTIMIZATION 37 4.1 Convex Sets and Functions 37 4.2 Practical Tests for Convexity 42 4.3 Further Properties of Convex Sets and Functions 47 5. ITERATIVE METHODS FOR OPTIMIZATION 50 5.1 Descent Methods 50 5.2 Classic Examples of Descent Methods 52 5.3 Solving Systems of Nonlinear Equations 58 6. LINEAR LEAST SQUARES AND RELATED PROBLEMS 60 6.1 A Motivating Application — CurveFitting 60 6.2 Linear Least Squares 61 6.3 A Very Brief Look at Nonlinear Least Squares 64 6.4 NearestPoint Projections and Complementary Subspaces 65 6.5 Optimality Conditions for Linear Least Squares 67 6.6 Nonnegative Linear Least Squares 70 i 7. GENERAL NONLINEAR PROGRAMMING 73 7.1 EqualityConstrained Problems — Lagrange Multipliers 73 7.2 EqualityConstrained Problems — Examples 76 7.3 InequalityConstrained Problems — Necessary Conditions 80 7.4 InequalityConstrained Problems — Examples 83 7.5 Sufficient Conditions 86 8. LINEAR PROGRAMMING 89 8.1 Linear Programming Formulations 89 8.2 Optimality Conditions — The Dual of a Linear Program 91 8.3 Basic Solutions of Linear Programs 96 8.4 The Simplex Method — A Motivating Example 101 8.5 The Simplex Method — Formal Algorithm and Theory 104 8.6 The Simplex Method — More Examples 108 8.7 PrimalDual Interior Point Methods for Linear Programs 117 A. APPENDIX — Additional Theory and Proofs 121 A.1 Positive Definite Matrices 121 A.2 Coercivity 125 A.3 Spectral Theorem for Symmetric Matrices 129 A.4 Matrix View of the Simplex Method 131 INDEX 133 ii 1. INTRODUCTION Optimization is the search for “extreme” values: minimum cost, greatest distance, least effort, best fit, and so on. Mathematical Programming is a field within mathematics com prising the theory and methods for solving optimization problems. (Here “programming” means “planning.”) Optimization problems are classified according to the different types of objects or decisions under consideration. Here’s a brief list of sample categories: • Linear and Nonlinear Programming — optimization involving vectors of real numbers and a finite list of constraints....
View
Full Document
 Spring '12
 DouglasWard
 Statistics, Linear Programming, Derivative, Continuous function

Click to edit the document details