.
LECTURES
ON
MODERN CONVEX OPTIMIZATION
Aharon Ben-Tal and Arkadi Nemirovski
The William Davidson Faculty of Industrial Engineering & Management,
Technion Israel Institute of Technology, [email protected]
http:/ie.technion.ac.il/Home/Users/morbt0
IE 8534
1
Lecture 4. The Dual Cone and Dual Problem
Shuzhong Zhang
IE 8534
2
For a convex cone K, its dual cone is dened as
K = cfw_y | x, y 0, x K.
The inner-product can be replaced by xT y if the coordinates of the
vectors are given. Clearly, K is alway
IE 8534
1
Lecture 3. Conic Optimization: What and Why?
Shuzhong Zhang
IE 8534
2
This course is about convex conic optimization models and the
solution methods for such models.
A few key words are worth noting here: convex, conic, and
optimization.
The las
Convex Optimization Boyd & Vandenberghe
2. Convex sets
ane and convex sets
some important examples
operations that preserve convexity
generalized inequalities
separating and supporting hyperplanes
dual cones and generalized inequalities
21
Ane set
l
IE 8534
1
IE 8534: Conic Optimization and Applications
Professor Shuzhong Zhang
Department of Industrial and Systems Engineering
College of Science and Engineering
University of Minnesota
Shuzhong Zhang
IE 8534
2
Lecture Hours:
Monday 2:30 - 5:50 pm
Quest
IE 8534
1
Lecture 2. The KKT Condition and Exchange Economy
Shuzhong Zhang
IE 8534
2
The KKT optimality condition has wide applications. In this lecture,
we will just illustrate one application in economics theory.
The so-called exchange economy introduce
IE 8534
1
Preliminaries: Basic Facts
Sets and set operations S T , S T and S \ T .
Vector space X :
- If x X and R then x X .
- If x, y X then x + y X .
Set of all real numbers
R and the Euclidean space Rn :
x1
.
. xi R, i = 1, ., n .
.
xn
Shuzhong Zhang
IE 8534
1
Supplement: Homans Error Bounds
Shuzhong Zhang
IE 8534
2
In Lecture 1 we learned that linear program
(P ) min
s.t.
cT x
Ax = b
x 0,
and its dual problem
max
bT y
s.t.
(D)
AT y + s = c
s0
under the Slater condition, admits the analytical central
IE 8534
1
Supplement: Error Bounds for Conic Optimization
Shuzhong Zhang
IE 8534
2
Let us now discuss the projection onto a convex cone. First note the
following important result.
Theorem 1 (Moreau) Suppose that K Rn is a closed convex cone.
Let x Rn be n
IE 8534
1
Lecture 5. Theorems of Alternatives and Self-Dual Embedding
Shuzhong Zhang
IE 8534
2
A system of linear equations may not have a solution.
It is well known that either
Ax = c
has a solution, or
A T y = 0 , c T y = 1
has a solution (but not both
IE 8534
1
Lecture 6. Some Graph Problems and Their SDP Relaxation
Shuzhong Zhang
IE 8534
2
For a given graph G = (V, E ; W ), where the weight on the edge (i, j ) is
wij , the max-cut problem can be simply written as
maximize
subject to
n
n
i=1
j =1 wij
1
Convex Optimization Boyd & Vandenberghe
1. Introduction
mathematical optimization
least-squares and linear programming
convex optimization
example
course goals and topics
nonlinear optimization
brief history of convex optimization
11
Mathematical o
IE 8534
1
Supplement: Universal Self-Concordant Barrier Functions
Shuzhong Zhang
IE 8534
2
Recall that a self-concordant barrier function for K is a barrier
function satisfying
|3 F (x)[h, h, h]|
2(2 F (x)[h, h])3/2 ,
|F (x)[h]|
(2 F (x)[h, h])1/2 ,
for
IE 8534
1
Lecture 12. The Self-Concordant Barrier and Path Following
Shuzhong Zhang
IE 8534
2
A natural method for solving constrained optimization, which has
been around for more than a half century, is the so-called barrier
function method. The method i
IE 8534
1
Supplement: Duality and the Minimax Theorem
Shuzhong Zhang
IE 8534
2
Duality plays an important role in convex analysis and optimization.
We have seen dual objects such as polar sets, dual cones, and
conjugate functions. They are of course all d
IE 8534
1
Supplement: The Lwner-John Ellipsoids
o
Shuzhong Zhang
IE 8534
2
An important result in convex analysis is the so-called Lwner-John
o
theorem, which says the following. For any compact convex set in Rn ,
let E be the minimum volume ellipsoid con
IE 8534
1
Lecture 10. Primal-Dual Interior Point Method for LP
Shuzhong Zhang
IE 8534
2
Consider a linear program
cT x
(P ) minimize
subject to
Ax = b
x0
and its dual
maximize
bT y
subject to
(D )
AT y + s = c
s 0.
Let FP be the feasible set for (P ).
Let
IE 8534
1
Lecture 8. SDP and the Matrix Decomposition
Shuzhong Zhang
IE 8534
2
Let A be any n n matrix, the eld of values of A is given by
F (A) := cfw_z H Az | z H z = 1 C.
This set, like the spectral values, contains much information about the
matrix A.
IE 8534
1
Lecture 7. SDP and Non-Convex Quadratic Optimization
Shuzhong Zhang
IE 8534
2
The idea of applying SDP relaxation can be used to solve broader
classes of non-convex quadratic models. In this lecture we shall discuss
two kinds of such optimizatio
IE 8534
1
Lecture 7: Nesterovs First Order Method
Shuzhong Zhang
IE 8534
2
Optimization is always a matter of weighing over dierent concerns.
There is no optimal method in optimization!
Even for convex optimization, especially very large size problems it
IE 8534
1
Lecture 6: Optimality Conditions for Nonlinear Programming
Shuzhong Zhang
IE 8534
2
In this lecture we will discuss a very important topic: how to
characterize and solve a nonlinear program with explicit equality and
inequality constraints? That
IE 8534
1
Lecture 5: Constrained Optimization
Shuzhong Zhang
IE 8534
2
Let us consider a general form of a constrained problem:
(P )
minimize
f (x)
subject to x X.
Certainly the most popular type of constrained optimization problem
is linear programming,
IE 8534
1
Lecture 6. Some Graph Problems and Their SDP Relaxation
Shuzhong Zhang
IE 8534
2
For a given graph G = (V, E ; W ), where the weight on the edge (i, j ) is
wij , the max-cut problem can be simply written as
maximize
subject to
n
n
i=1
j =1 wij
1
IE 8534
1
Lecture 5. Theorems of Alternatives and Self-Dual Embedding
Shuzhong Zhang
IE 8534
2
A system of linear equations may not have a solution.
It is well known that either
Ax = c
has a solution, or
A T y = 0 , c T y = 1
has a solution (but not both
IE 8534
1
Lecture 4. The Dual Cone and Dual Problem
Shuzhong Zhang
IE 8534
2
For a convex cone K, its dual cone is dened as
K = cfw_y | x, y 0, x K.
The inner-product can be replaced by xT y if the coordinates of the
vectors are given. Clearly, K is alway
IE 8534
1
Lecture 3. Conic Optimization: What and Why?
Shuzhong Zhang
IE 8534
2
This course is about convex conic optimization models and the
solution methods for such models.
A few key words are worth noting here: convex, conic, and
optimization.
The las
IE 8534
1
Lecture 2. The KKT Condition and Exchange Economy
Shuzhong Zhang
IE 8534
2
The KKT optimality condition has wide applications. In this lecture,
we will just illustrate one application to the economics theory.
The so-called exchange economy intro
IE 8534
1
IE 8534: Conic Optimization and Applications
Professor Shuzhong Zhang
Department of Industrial and Systems Engineering
College of Science and Engineering
University of Minnesota
Shuzhong Zhang
IE 8534
2
Lecture Hours:
Monday 2:30 - 6:30 pm
Quest
Convex Optimization Boyd & Vandenberghe
2. Convex sets
ane and convex sets
some important examples
operations that preserve convexity
generalized inequalities
separating and supporting hyperplanes
dual cones and generalized inequalities
21
Ane set
l
IE 8534
1
Review: Some Basic Facts
Sets and set operations S T , S T and S \ T .
Vector space X :
- If x X and R then x X .
- If x, y X then x + y X .
Set of all real numbers
R and the Euclidean space Rn :
x1
.
. xi R, i = 1, ., n .
.
xn
Shuzhong Zhang
IE