EE236C (Spring 2008-09)
3. Subgradient method
subgradient method convergence analysis optimal step size when f is known alternating projections optimality
31
Subgradient method
to minimize a nondierentiable convex function f : choose x(0) and repeat x(k)
EE236C (Spring 2008-09)
5. Gradient projection
projected gradient examples convergence analysis dual gradient methods
51
(Sub-)gradient projection
minimize f (x) subject to x C f convex with dom f = Rn C a closed convex set; we denote by PC (or just P )
EE236C (Spring 2008-09)
6. Smoothing techniques
motivation smoothing via conjugate examples
61
First-order convex optimization methods
complexity of nding -suboptimal point of f f dierentiable O( L/) iterations
with fast gradient method (L is Lipschitz c
EE236C (Spring 2008-09)
7. Gradient methods with generalized distances
Bregman distances variant of Nesterovs method example
71
Gradient method and extension
basic gradient method for minimizing f (lecture 1) x+ = argmin f (x) + f (x)T (z x) +
z
1 zx 2t
EE236C (Spring 2008-09)
8. Localization and cutting-plane methods
cutting-plane oracle nding cutting-planes localization algorithms specic cutting-plane methods epigraph cutting-plane method
81
Localization methods
based on idea of localizing desired po
EE236C (Spring 2008-09)
9. Analytic center cutting-plane method
analytic center cutting-plane method computing the analytic center pruning constraints lower bound and stopping criterion
91
Analytic center and ACCPM
analytic center of a set of inequalitie
EE236C (Spring 2008-09)
10. Ellipsoid method
ellipsoid method convergence proof inequality constraints
101
Ellipsoid method
history developed by Shor, Nemirovski, Yudin in 1970s used in 1979 by Khachian to show polynomial solvability of LPs properties ea
EE236C (Spring 2008-09)
13. Dual methods
dual of convex problem with linear constraints dierentiability of dual function dual decomposition rate control
131
Convex problem with linear constraints
minimize f (x) subject to Gx h Ax = b (G Rmn, A Rpn) dual
EE236C (Spring 2008-09)
14. Dual methods II
single commodity network ow augmented Lagrangian method
141
Single commodity network ow
network connected, directed graph with n links, p nodes node incidence matrix A Rpn is
1 arc j enters i 1 arc j leaves no
EE236C (Spring 2008-09)
15. Saddle-point problems
denition convex-concave games primal-dual decomposition
151
Min-max inequality
the inequalities inf f (x, y ) f (, y ) sup f (, y ) x x
xX y Y
hold for any function f , any sets X , Y , any point (, y ) X
EE236C (Spring 2008-09)
16. Variational inequalities
variational inequality monotonicity examples linear complementarity problem analytic center cutting-plane method extragradient method
161
Variational inequality
given closed convex set C , mapping F :
EE236C (Spring 2008-09)
17. Primal-dual interior-point methods
cone programming logarithmic barrier function and central path symmetrization Nesterov-Todd scaling path-following algorithm quadratic cone program
171
Cone program
primal problem minimize cT
EE236C (Spring 2008-09)
18. Primal-dual interior-point methods II
self-dual embedding path-following algorithm
181
Initialization and infeasibility detection
barrier method (EE236B) assumes problem is primal and dual feasible requires phase I to nd initi