This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE236A (Fall 200708) Lecture 12 The barrier method • brief history of interiorpoint methods • Newton’s method for smooth unconstrained minimization • logarithmic barrier function • central points, the central path • the barrier method 12–1 The ellipsoid method • 1972: ellipsoid method for (nonlinear) convex nondifferentiable optimization (Nemirovsky, Yudin, Shor) • 1979: Khachiyan proves that the ellipsoid method applied to LP has polynomial worstcase complexity • much slower in practice than simplex • very different approach from simplex method; extends gracefully to nonlinear convex problems • solved important open theoretical problem (polynomialtime algorithm for LP) The barrier method 12–2 Interiorpoint methods early methods (1950s–1960s) • methods for solving convex optimization problems via sequence of smooth unconstrained problems • logarithmic barrier method (Frisch), sequential unconstrained minimization (Fiacco & McCormick), affine scaling method (Dikin), method of centers (Huard & Lieu) • no worstcase complexity theory; (often) worked well in practice • fell out of favor in 1970s new methods (1984–) • 1984 Karmarkar: new polynomialtime method for LP (projective algorithm) • later recognized as closely related to earlier interiorpoint methods • many variations since 1984; widely believed to be faster than simplex for very large problems (over 10,000 variables/constraints) The barrier method 12–3 Gradient and Hessian differentiable function f : R n → R gradient and Hessian (evaluated at x ): ∇ f ( x ) = ∂f ( x ) ∂x 1 ∂f ( x ) ∂x 2 . . . ∂f ( x ) ∂x n , ∇ 2 f ( x ) = ∂ 2 f ( x ) ∂x 2 1 ∂ 2 f ( x ) ∂x 1 ∂x 2 ··· ∂ 2 f ( x ) ∂x 1 ∂x n ∂ 2 f ( x ) ∂x 2 ∂x 1 ∂ 2 f ( x ) ∂x 2 2 ··· ∂ 2 f ( x ) ∂x 2 ∂x n . . . . . . . . . . . . ∂ 2 f ( x ) ∂x n ∂x 1 ∂ 2 f ( x ) ∂x n x 2 ··· ∂ 2 f ( x ) ∂x 2 n 2nd order Taylor series expansion around x : f ( y ) ≃ f ( x ) + ∇ f ( x ) T ( y − x ) + 1 2 ( y − x ) T ∇ 2 f ( x )( y − x ) The barrier method 12–4 Positive semidefinite matrices a quadratic form is a function f : R n → R with f ( x ) = x T Ax = n summationdisplay i,j =1 A ij x i x j may as well assume A = A T since x T Ax = x T (( A + A T ) / 2) x A = A T is positive semidefinite if x T Ax ≥ for all x A = A...
View
Full
Document
 Spring '07
 Dr.Vandenber

Click to edit the document details