This preview shows pages 1–7. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Lecture 17: InteriorPoint Method March 26, 2007 Lecture 17 Outline • Review of Selfconcordance • Overview of Netwon’s Methods for Equality Constrained Minimization • Examples • InteriorPoint Method Convex Optimization 1 Lecture 17 SelfConcordance • Overcomes the shortcoming of classical analysis of Newton’s method, where the convergence rate depends on constants m , M , and c : k∇ 2 f ( x ) ∇ 2 f ( y ) k ≤ c k x y k 2 mI ∇ 2 f ( x ) MI and m , M , and c change with the choice of the norm k · k • Selfconcordance introduced by Nesterov and Nemirovski [1994] ∇ 3 f ( x )[ u, u, u ] ≤ M f k u k 3 / 2 ∇ 2 f ( x ) = M f u T ∇ 2 f ( x ) u 3 / 2 Def. A function f : R 7→ R is selfconcordant when f is convex and  f 000 ( x )  ≤ 2 f 00 ( x ) 3 / 2 and for all x ∈ dom f The rate of change in curvature of f is bounded by the curvature Def. A function f : R n 7→ R is selfconcordant when its restriction to any line is selfconcordant. Note : One can use a constant κ other than 2 in the definition Convex Optimization 2 Lecture 17 Equality Constrained Minimization minimize f ( x ) subject to Ax = b KKT Optimality Conditions imply that x * is optimal if and only if there exists a λ * such that Ax * = b, ∇ f ( x * ) + A T λ * = 0 • Newton’s Method solves KKT conditions " ∇ 2 f ( x k ) A T A #" d k w k # = " ∇ f ( x k ) h k # where • Feasible point method uses h k = 0 • Infeasible point method uses h k = Ax k b with w k being dual optimal for the minimization of the quadratic approximation of f at x k Convex Optimization 3 Lecture 17 Equality Constrained Analytic Centering minimize f ( x ) = ∑ n i =1 ln x i subject to Ax = b Feasible point Newton’s method : g = ∇ f ( x ) , H = ∇ 2 f ( x ) " H A T A #" d w # = " g # , g =  1 x 1 . . . 1 x n , H = diag " 1 x 2 1 , . . . , 1 x 2 n # • The Hessian is positive definite • KKT matrix first row: Hd + A T w = g ⇒ d = H 1 ( g + A T w ) (1) • KKT matrix second row, Ad = 0 , and Eq. (1) ⇒ AH 1 ( g + A T w ) = 0 • The matrix A has full row rank, thus AH 1 A T is invertible, hence w = AH 1 A T 1 AH 1 g, H 1 = diag h x 2 1 , . . . , x 2 n i • The matrix AH 1 A T is known as Schur complement of H (any H ) Convex Optimization 4 Lecture 17 Network Flow Optimization minimize ∑ n l =1 φ l ( x l ) subject to Ax = b • Directed graph with n arcs and p + 1 nodes • Variable x l : flow through arc...
View
Full
Document
This note was uploaded on 08/22/2008 for the course GE 498 AN taught by Professor Angelianedich during the Spring '07 term at University of Illinois at Urbana–Champaign.
 Spring '07
 AngeliaNedich

Click to edit the document details