# L16_self - Lecture 16 Self-Concordance and Equality...

This preview shows pages 1–6. Sign up to view the full content.

Lecture 16: Self-Concordance and Equality Constrained Minimization March 14, 2007

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Lecture 16 Outline Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications for Newton’s Method Equality Constrained Minimization Convex Optimization 1
Lecture 16 Classical Convergence Analysis of Newton’s Method Given a level set L 0 = { x | f ( x ) f ( x 0 ) } , it requires for all x, y L 0 : k∇ 2 f ( x ) - ∇ 2 f ( y ) k ≤ c k x - y k 2 mI ± ∇ 2 f ( x ) ± MI for some constants c > 0 , m > 0 , and M > 0 Given a desired error level ± , we are interested in an ± -solution of the problem i.e., a vector ˜ x such that f x ) f * + ± An upper bound on the number of iterations to generate an ± -solution is given by f ( x 0 ) - f * γ + log 2 log 2 ± ± 0 ± ² where γ = σβη 2 m M 2 , η (0 , m 2 /c ) , and ± 0 = 2 m 3 /c 2 The bound is conceptually informative, but not practical Furthermore, the constants c , m , and M change with aﬃne transforma- tions of the space, while Newton’s method is aﬃne invariant Can a bound be obtained in terms of problem data that is aﬃne invariant and, moreover, practically veriﬁable??? Convex Optimization 2

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Lecture 16 Self-concordance Nesterov and Nemirovski introduced a notion of self-concordance and a class of self-concordant functions Importance of the self-concordance: Possesses aﬃne invariant property Provides a new tool for analyzing Newton’s method that exploits the aﬃne invariance of the method Results in a practical upper bound on the Newton’s iterations Plays a crucial role in performance analysis of interior point method Def. A function f : R 7→ R is self-concordant when f is convex and | f 000 ( x ) | ≤ 2 f 00 ( x ) 3 / 2 and for all x dom f The rate of change in curvature of f is bounded by the curvature Note : One can use a constant κ other than 2 in the deﬁnition Convex Optimization 3
Lecture 16 Examples Linear and quadratic functions are self-concordant [ f 000 ( x ) = 0 for all x ] Negative logarithm f ( x ) = - ln x , x > 0 is self-concordant: f 00 ( x ) = 1 x 2 , f 000 ( x ) = - 2 x 3 , | f 000 ( x ) | f 00 ( x ) 3 / 2 = 2 for all x > 0 Exponential function e x is not self-concordant: f 00 ( x ) = f 000 ( x ) = e x , | f 000 ( x ) | f 00 ( x ) 3 / 2 = e x e 3 x/ 2 = e - x/ 2 , | f 000 ( x ) | f

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 08/22/2008 for the course GE 498 AN taught by Professor Angelianedich during the Spring '07 term at University of Illinois at Urbana–Champaign.

### Page1 / 18

L16_self - Lecture 16 Self-Concordance and Equality...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online