KKTgeometry

KKTgeometry - MS&E 211 The Principles and Geometries of...

Info iconThis preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
1 MS&E 211 December 1, 2005 The Principles and Geometries of KKT and Optimization X ) ( ) ( 2 1 2 1 s.t. min x ,..., x , x x ,..., x , x f n n
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 MS&E 211 December 1, 2005 Geometries of KKT: Unconstrained Problem: Minimize f(x), where x is a vector that could have any values, positive or negative First Order Necessary Condition (min or max) : 2200 f(x) = 0 (∂f/∂x i = 0 for all i) is the first order necessary condition for optimization Second Order Necessary Condition : 2200 2 f(x) is positive semidefinite (PSD) • [ x • 2 f(x) • x ≥ 0 for all x ] Second Order Sufficient Condition (Given FONC satisfied) 2200 2 f(x) is positive definite (PD) [ x • 2 f(x) • x > 0 for all x ] ∂f/∂x i = 0 x i f 2 f/∂x i 2 > 0
Background image of page 2
3 MS&E 211 December 1, 2005 Geometries of KKT: Equality Constrained (one constraint) Problem: Minimize f(x), where x is a vector Subject to: h(x) = b First Order Necessary Condition for minimum (or for maximum): 2200 f(x) = λ h(x) for some λ free ( λ is a scalar) Two surfaces must be tangent h(x) = b and -h(x) = -b are the same; there is no sign restriction on λ h(x) = b
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 MS&E 211 December 1, 2005 Geometries of KKT: Equality Constrained (one constraint) First Order Necessary Condition: 2200 f(x) = λ h (x) for some λ Lagrangian: L (x, λ ) = f(x) - λ [h(x)-b] , Minimize L (x, λ ) over x and Maximize L (x, λ ) over λ . Use principles of unconstrained optimization 2200 L (x, λ ) = 0: 2200 x L (x, λ ) = f(x) - λ h (x) = 0 2200 λ L (x, λ ) = h (x)-b = 0
Background image of page 4
5 MS&E 211 December 1, 2005 Geometries of KKT: Equality Constrained (multiple constraints) Problem: Minimize f(x), where x is a vector • Such that: h i (x) = b i for i = 1,2,…,m KKT Conditions (Necessary Conditions) : Exist λ i ,i = 1,2,…,m, such that 2200 f(x) = Σ i=1 n λ i h i (x) • h i (x) = b i for i = 1,2,…,m Such a point (x, λ ) is called a KKT point , and λ is called the Dual Vector or the Lagrange Multipliers . Furthermore, these conditions are sufficient if f(x) is convex and h i (x), i = 1,2, …,m, are linear.
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
6 MS&E 211 December 1, 2005 Geometries of KKT: Unconstrained, Except Non-Negativity Condition Problem: Minimize f(x), where x is a vector, x > 0 First Order Necessary Condition : • ∂f/∂x i = 0 if x i > 0 • ∂f/∂x i ≥ 0 if x i = 0 • Thus: [∂f/∂x i ]x i = 0 for all x i , or f(x) • x = 0, f(x) ≥ 0 If interior point (x > 0), then f(x) = 0 Nothing changes if the constraint is not binding ∂f/∂x i = 0 x i f ∂f/∂x i > 0
Background image of page 6
7 MS&E 211 December 1, 2005
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

Page1 / 26

KKTgeometry - MS&E 211 The Principles and Geometries of...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online