Optimization

# i mi 0 i 1 i mi gi x 0 i 1 i for

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: i (x * ) £ 0, i = 1,..., I mi ³ 0, i = 1,..., I mi gi (x * ) = 0, i = 1,..., I for vectors l Î RJ and m Î RI RISK AND PORTFOLIO MANAGEMENT WITH ECONOMETRICS, VER. 11/21/2012. © P. KOLM. 26 Comments: Both vectors l Î RJ and m Î RI are called Lagrange multipliers Any point that satisfies the KKT conditions is called a KKT point It can be shown that if x * is an optimal solution of the nonlinear programming problem then it must be a KKT point o However, the converse is not true in general. In other words, the KKT conditions are necessary for all nonlinear programming problems, but not sufficient o For the subclass of convex nonlinear programs the KKT conditions are also sufficient Observe that the Karush-Kuhn-Tucker conditions for general nonlinear program take the form of a system of nonlinear equations. Many optimization algorithms are based on solving this set of nonlinear equations RISK AND PORTFOLIO MANAGEMENT WITH ECONOMETRICS, VER. 11/21/2012. © P. KOLM. 27 Example max ln(x 1 + 1) + x 2 x1 ,x 2 subject to 2x 1 + x 2 £ 3 , x 1 ³ 0, x 2 ³ 0 Solution: Define the Lagrangian L º L(x 1, x 2 ) = - ln(x 1 + 1) - x 2 + l1(2x 1 + x 2 - 3) - l2x 1 - l3x 2 FOCs: ¶L 1 =+ 2l1 - l2 = 0 ¶x 1 x1 + 1 ¶L = -1 + l1 - l3 = 0 ¶x 2 2x 1 + x 2 - 3 £ 0, (1) (2) - x 1 £ 0, - x 2 £ 0, li ³ 0 (i = 1,2, 3) (3) l1(2x 1 + x 2 - 3) = 0 (4) l2x 1 = 0 (5) RISK AND PORTFOLIO MANAGEMENT WITH ECONOMETRICS, VER. 11/21/2012. © P. KOLM. 28 l3x 2 = 0 (6) l3 ³ 0 and (2) imply l1 ³ 1. Therefore (4) implies, 2x 1 + x 2 - 3 = 0 Now assume l2 = 0 , then (1) gives us (7) 1 = 2l1 . Let us look at the left- and x1 + 1 right-hand side (LH/RH) of this equality: LH: x 1 ³ 0 1 £1 x1 + 1 RH: l1 ³ 1 2l1 ³ 2 This is a contradiction, so we conclude that l2 > 0 . From (5) we infer x 1 = 0 . Now (7) yields x 2 = 3 . From (6) we obtain l3 = 0 , and hence (1) and (2) imply l1 = l3 + 1 = 1 and l2 = 2l1 - 1 = 2 - 1 = 1 . The solution is (x 1, x 2 ) = (0, 3) x1 + 1 and (l1, l2, l3 ) = (1,1, 0) . RISK AND PORTFOLIO MANAGEMENT WITH ECONOMETRICS, VER. 11/21/2012. © P. KOLM. 29 How Do Optimization Algorithms Work? Optimization packages are built upon rather sophisticated algorithms, know- how and heuristics It can be hard for the optimization non-expert to understand in detail how particular algorithms work It is not necessary to fully understand all the intricacies of optimization theory in order to make use of optimization software efficiently. However, a basic understanding is very useful RISK AND PORTFOLIO MANAGEMENT WITH ECONOMETRICS, VER. 11/21/2012. © P. KOLM. 30 Optimization Algorithms Are Of Iterative Nature An optimization solver generates a sequence of approximate solutions x 0, x 1, x 2,... that gets closer and closer to the true solution x * We say that the sequence of approximate solutions converge to the true solution if x k - x * 0 as k ¥ Since the true solution is not known we use termination or convergence criteria. Example: “Stop when no longer any progress is being made in improving the solution,” that is when x k - x k +1 < TOL where TOL is a user-def...
View Full Document

## This document was uploaded on 02/17/2014 for the course COURANT G63.2751.0 at NYU.

Ask a homework question - tutors are online