{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Lecture14

# Lecture14 - CO 355 Lecture 14 Fundamentals of Nonlinear...

This preview shows pages 1–9. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: CO 355 Lecture 14 Fundamentals of Nonlinear Optimization Vris Cheung (Substitute class) October 28, 2010 Vris Cheung (University of Waterloo) CO 355 2010 1 / 20 Unconstrained optimization Outline 1 Extremum: unconstrained optimization 2 Constrained optimization 3 Optimality conditions: Lagrangian Vris Cheung (University of Waterloo) CO 355 2010 2 / 20 Unconstrained optimization Extremum: unconstrained optimization + Given f : S ⊆ R n → R . + x * is a (global) minimizer of f : f ( x * ) 6 f ( x ) for all x ∈ S . + x * is a local minimizer of f : for some nbd. N x * of x * , f ( x * ) 6 f ( x ) for all x ∈ N x * ∩ S . Vris Cheung (University of Waterloo) CO 355 2010 3 / 20 Unconstrained optimization Extremum: unconstrained optimization Necessary optimality conditions for local minimizer: 1st order cond.: ∇ f ( x * ) = 2st order cond.: ∇ 2 f ( x * ) Su ffi cient optimality condition for local minimizer: 1st order cond.: ∇ f ( x * ) = AND 2st order cond.: ∇ 2 f ( x * ) If ∇ f ( x ) = 0, x is said to be a saddle point / critical point . Vris Cheung (University of Waterloo) CO 355 2010 4 / 20 Unconstrained optimization Extremum: unconstrained optimization When we try to solve min x ∈ S f ( x ) , using only 1st order opt. cond. 1 , we might end up + a global minimizer — great! + or... a local minimizer , + or... a saddle point .... (or maybe *nothing*...) But if f is convex , we don’t have such a problem: Theorem If f : S ⊆ R n → R is convex (with S convex) and ∇ f ( x * ) = , then x * is a global minimizer of f. 1 We usually don’t have 2nd order information because it is too expensive to obtain in practice. Vris Cheung (University of Waterloo) CO 355 2010 5 / 20 Unconstrained optimization Extremum: convex case Theorem If f : S ⊆ R n → R is convex (with S convex) and ∇ f ( x * ) = , then x * is a global minimizer of f. Proof. For any x ∈ S , by convexity, f ( x ) > f ( x * ) + ∇ f ( x * ) > ( x- x * ) = f ( x * ) . Vris Cheung (University of Waterloo) CO 355 2010 6 / 20 Unconstrained optimization Existence of global minimizer Sadly, even in convex case, it could happen that a minimizer simply does not exist: consider 2 f ( x ) := e- x ( x ∈ R ) . ! 1 1 2 3 0.5 1.0 1.5 2.0 2.5 inf x ∈ R e- x = is finite, but not attained by any x ∈ R . 2 Graphics source: Wolfram Alpha . Vris Cheung (University of Waterloo) CO 355 2010 7 / 20 Unconstrained optimization Existence of global minimizer So when would a minimizer exist?...
View Full Document

{[ snackBarMessage ]}

### Page1 / 26

Lecture14 - CO 355 Lecture 14 Fundamentals of Nonlinear...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online