This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: CO 355 Lecture 14 Fundamentals of Nonlinear Optimization Vris Cheung (Substitute class) October 28, 2010 Vris Cheung (University of Waterloo) CO 355 2010 1 / 20 Unconstrained optimization Outline 1 Extremum: unconstrained optimization 2 Constrained optimization 3 Optimality conditions: Lagrangian Vris Cheung (University of Waterloo) CO 355 2010 2 / 20 Unconstrained optimization Extremum: unconstrained optimization + Given f : S ⊆ R n → R . + x * is a (global) minimizer of f : f ( x * ) 6 f ( x ) for all x ∈ S . + x * is a local minimizer of f : for some nbd. N x * of x * , f ( x * ) 6 f ( x ) for all x ∈ N x * ∩ S . Vris Cheung (University of Waterloo) CO 355 2010 3 / 20 Unconstrained optimization Extremum: unconstrained optimization Necessary optimality conditions for local minimizer: 1st order cond.: ∇ f ( x * ) = 2st order cond.: ∇ 2 f ( x * ) Su ffi cient optimality condition for local minimizer: 1st order cond.: ∇ f ( x * ) = AND 2st order cond.: ∇ 2 f ( x * ) If ∇ f ( x ) = 0, x is said to be a saddle point / critical point . Vris Cheung (University of Waterloo) CO 355 2010 4 / 20 Unconstrained optimization Extremum: unconstrained optimization When we try to solve min x ∈ S f ( x ) , using only 1st order opt. cond. 1 , we might end up + a global minimizer — great! + or... a local minimizer , + or... a saddle point .... (or maybe *nothing*...) But if f is convex , we don’t have such a problem: Theorem If f : S ⊆ R n → R is convex (with S convex) and ∇ f ( x * ) = , then x * is a global minimizer of f. 1 We usually don’t have 2nd order information because it is too expensive to obtain in practice. Vris Cheung (University of Waterloo) CO 355 2010 5 / 20 Unconstrained optimization Extremum: convex case Theorem If f : S ⊆ R n → R is convex (with S convex) and ∇ f ( x * ) = , then x * is a global minimizer of f. Proof. For any x ∈ S , by convexity, f ( x ) > f ( x * ) + ∇ f ( x * ) > ( x x * ) = f ( x * ) . Vris Cheung (University of Waterloo) CO 355 2010 6 / 20 Unconstrained optimization Existence of global minimizer Sadly, even in convex case, it could happen that a minimizer simply does not exist: consider 2 f ( x ) := e x ( x ∈ R ) . ! 1 1 2 3 0.5 1.0 1.5 2.0 2.5 inf x ∈ R e x = is finite, but not attained by any x ∈ R . 2 Graphics source: Wolfram Alpha . Vris Cheung (University of Waterloo) CO 355 2010 7 / 20 Unconstrained optimization Existence of global minimizer So when would a minimizer exist?...
View
Full Document
 Winter '10
 Harvey
 Optimization, Vris Cheung

Click to edit the document details