Lecture 19 Notes: Necessary Optimality Conditions
1
Introduction
Recall that a constrained optimization problem is a problem of the form
(P) minx f (x)
s.t.
g(x) 0
h(x) = 0
x X,
where X is an open set and g(x) = (g1 (x), . . . , gm (x) : n m , h(x) =
(h1
Lecture 6 Notes: Constrained Optimization Part 1
1
Introduction
Recall that a constrained optimization problem is a problem of the form
(P) minx f (x)
s.t.
g(x) 0
h(x) = 0
x X,
where X is an open set and g(x) = (g1 (x), . . . , gm (x) : n m , h(x) =
(h1 (
Lecture 1 Notes: Unconstrained Optimization
1
Preliminaries
1.1
Types of optimization problems
Unconstrained Optimization Problem:
f (x)
(P) minx
s.t.
x X,
where x = (x1 , . . . , xn ) n , f (x) : n , and X is an open set (usually
X = n ).
We say that x i
Lecture 9 Notes: Projection Methods
1
Review of Steepest Descent
Suppose we want to solve
P : minimize f (x)
s.t.
x n ,
where f (x) is dierentiable. At the point x = x, f (x) can be
approximated by its linear expansion
f (x + d) f (x) +
f (x)T d
for d sma
Lecture 3 Notes: Newmans Methods
1
Newtons Method
Suppose we want to solve:
(P:)
min f (x)
x n .
At x = x, f (x) can be approximated by:
f (x) h(x) := f (x) +
1
f (x)T (x ) + (x )t H()(x x),
which is the quadratic Taylor expansion of f (x) at x = . Here
f
Lecture 4 Notes: Quadratic Forms
1
Quadratic Optimization
A quadratic optimization problem is an optimization problem of the form:
(QP) : minimize f (x) := 2 xT Qx + cT x
s.t.
x n.
Problems of the form QP are natural models that arise in a variety
of sett
Lecture 5 Notes: Steepest Descent Method
1
The Algorithm
The problem we are interested in solving is:
P : minimize f (x)
s.t.
x n,
where f (x) is dierentiable. If x = x is a given point, f (x) can be
approxi-mated by its linear expansion
f (x + d) f (x) +
Lecture 10 Notes: Projections
1
Introduction
Consider the constrained optimization problem P :
P : minimize f (x)
x
s.t.
gi (x) 0, i = 1, . . . , m
hi (x) = 0, i = 1, . . . , k
x n ,
whose feasible region we denote by
F := cfw_x n | gi (x) 0, i = 1, . . .
Lecture 14 Notes: Interior-Point Method
1
The Problem
The logarithmic barrier approach to solving a linear program dates back to
the work of Fiacco and McCormick in 1967 in their book Sequential Unconstrained Minimization Techniqu,esalso known simply as S
Lecture 12 Notes: Barrier Method
1
The Conditional-Gradient Method for Constrained
Optimization (Frank-Wolfe Method)
We now consider the following optimization problem:
P : minimizex
s.t.
f (x)
xC .
We assume that f (x) is a convex function, and that C is
Lecture 18 Notes: Duality
1
Overview
The Practical Importance of Duality
Denition of the Dual Problem
Steps in the Construction of the Dual Problem
Examples of Dual Constructions
The Column Geometry of the Primal and Dual Problems
The Dual is a Conc