L16_self - Lecture 16 Self-Concordance and Equality...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
Lecture 16: Self-Concordance and Equality Constrained Minimization March 14, 2007
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Lecture 16 Outline Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications for Newton’s Method Equality Constrained Minimization Convex Optimization 1
Background image of page 2
Lecture 16 Classical Convergence Analysis of Newton’s Method Given a level set L 0 = { x | f ( x ) f ( x 0 ) } , it requires for all x, y L 0 : k∇ 2 f ( x ) - ∇ 2 f ( y ) k ≤ c k x - y k 2 mI ± ∇ 2 f ( x ) ± MI for some constants c > 0 , m > 0 , and M > 0 Given a desired error level ± , we are interested in an ± -solution of the problem i.e., a vector ˜ x such that f x ) f * + ± An upper bound on the number of iterations to generate an ± -solution is given by f ( x 0 ) - f * γ + log 2 log 2 ± ± 0 ± ² where γ = σβη 2 m M 2 , η (0 , m 2 /c ) , and ± 0 = 2 m 3 /c 2 The bound is conceptually informative, but not practical Furthermore, the constants c , m , and M change with affine transforma- tions of the space, while Newton’s method is affine invariant Can a bound be obtained in terms of problem data that is affine invariant and, moreover, practically verifiable??? Convex Optimization 2
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Lecture 16 Self-concordance Nesterov and Nemirovski introduced a notion of self-concordance and a class of self-concordant functions Importance of the self-concordance: Possesses affine invariant property Provides a new tool for analyzing Newton’s method that exploits the affine invariance of the method Results in a practical upper bound on the Newton’s iterations Plays a crucial role in performance analysis of interior point method Def. A function f : R 7→ R is self-concordant when f is convex and | f 000 ( x ) | ≤ 2 f 00 ( x ) 3 / 2 and for all x dom f The rate of change in curvature of f is bounded by the curvature Note : One can use a constant κ other than 2 in the definition Convex Optimization 3
Background image of page 4
Lecture 16 Examples Linear and quadratic functions are self-concordant [ f 000 ( x ) = 0 for all x ] Negative logarithm f ( x ) = - ln x , x > 0 is self-concordant: f 00 ( x ) = 1 x 2 , f 000 ( x ) = - 2 x 3 , | f 000 ( x ) | f 00 ( x ) 3 / 2 = 2 for all x > 0 Exponential function e x is not self-concordant: f 00 ( x ) = f 000 ( x ) = e x , | f 000 ( x ) | f 00 ( x ) 3 / 2 = e x e 3 x/ 2 = e - x/ 2 , | f 000 ( x ) | f
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 08/22/2008 for the course GE 498 AN taught by Professor Angelianedich during the Spring '07 term at University of Illinois at Urbana–Champaign.

Page1 / 18

L16_self - Lecture 16 Self-Concordance and Equality...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online