KKTgeometry

# KKTgeometry - MS&E 211 The Principles and Geometries of KKT...

This preview shows pages 1–8. Sign up to view the full content.

1 MS&E 211 December 1, 2005 The Principles and Geometries of KKT and Optimization X ) ( ) ( 2 1 2 1 s.t. min x ,..., x , x x ,..., x , x f n n

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 MS&E 211 December 1, 2005 Geometries of KKT: Unconstrained Problem: Minimize f(x), where x is a vector that could have any values, positive or negative First Order Necessary Condition (min or max) : 2200 f(x) = 0 (∂f/∂x i = 0 for all i) is the first order necessary condition for optimization Second Order Necessary Condition : 2200 2 f(x) is positive semidefinite (PSD) • [ x • 2 f(x) • x ≥ 0 for all x ] Second Order Sufficient Condition (Given FONC satisfied) 2200 2 f(x) is positive definite (PD) [ x • 2 f(x) • x > 0 for all x ] ∂f/∂x i = 0 x i f 2 f/∂x i 2 > 0
3 MS&E 211 December 1, 2005 Geometries of KKT: Equality Constrained (one constraint) Problem: Minimize f(x), where x is a vector Subject to: h(x) = b First Order Necessary Condition for minimum (or for maximum): 2200 f(x) = λ h(x) for some λ free ( λ is a scalar) Two surfaces must be tangent h(x) = b and -h(x) = -b are the same; there is no sign restriction on λ h(x) = b

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4 MS&E 211 December 1, 2005 Geometries of KKT: Equality Constrained (one constraint) First Order Necessary Condition: 2200 f(x) = λ h (x) for some λ Lagrangian: L (x, λ ) = f(x) - λ [h(x)-b] , Minimize L (x, λ ) over x and Maximize L (x, λ ) over λ . Use principles of unconstrained optimization 2200 L (x, λ ) = 0: 2200 x L (x, λ ) = f(x) - λ h (x) = 0 2200 λ L (x, λ ) = h (x)-b = 0
5 MS&E 211 December 1, 2005 Geometries of KKT: Equality Constrained (multiple constraints) Problem: Minimize f(x), where x is a vector • Such that: h i (x) = b i for i = 1,2,…,m KKT Conditions (Necessary Conditions) : Exist λ i ,i = 1,2,…,m, such that 2200 f(x) = Σ i=1 n λ i h i (x) • h i (x) = b i for i = 1,2,…,m Such a point (x, λ ) is called a KKT point , and λ is called the Dual Vector or the Lagrange Multipliers . Furthermore, these conditions are sufficient if f(x) is convex and h i (x), i = 1,2, …,m, are linear.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
6 MS&E 211 December 1, 2005 Geometries of KKT: Unconstrained, Except Non-Negativity Condition Problem: Minimize f(x), where x is a vector, x > 0 First Order Necessary Condition : • ∂f/∂x i = 0 if x i > 0 • ∂f/∂x i ≥ 0 if x i = 0 • Thus: [∂f/∂x i ]x i = 0 for all x i , or f(x) • x = 0, f(x) ≥ 0 If interior point (x > 0), then f(x) = 0 Nothing changes if the constraint is not binding ∂f/∂x i = 0 x i f ∂f/∂x i > 0
7 MS&E 211 December 1, 2005

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 06/16/2010 for the course MS&E 211 taught by Professor Yinyuye during the Fall '07 term at Stanford.

### Page1 / 26

KKTgeometry - MS&E 211 The Principles and Geometries of KKT...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online