This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Chapter 2 Optimality Conditions 2.1 Global and Local Minima for Unconstrained Problems When a minimization problem does not have any constraints, the problem is to find the minimum of the objective function. We distinguish between two kinds of minima. The global minimum is the minimum of the function over the entire domain of interest, while a local minimum is the minimum over a smaller subdomain. The design point where the objective function f reaches a minimum is called the minimizer and is denoted by an asterisk as x * . The global minimizer is the point x * which satisfies f ( x * ) ≤ f ( x ) . (2.1.1) A point x * is a local minimizer if you for some r > f ( x * ) ≤ f ( x ) . if  x x *  < r . (2.1.2) That is, x * is a local minimizer, if you can find a sphere around it in which it is the minimizer. 2.2 Taylor series Expansion To find conditions for a point to be a minimizer, we use the Taylor series expansion around a presumed minimizer x * f ( x ) = f ( x * )+ n X i =1 ( x i x * i ) ∂f ∂x i ( x * )+ 1 2 n X i =1 n X k =1 ( x i x * i )( x k x * k ) ∂ 2 f ∂x i ∂x k ( x * )+higher order terms (2.2.1) When x is very close to x * , we can neglect even the second derivative terms. Furthermore, if x and x * are identical except for the j th component, then Eq. 2.2.1 becomes f ( x ) ≈ f ( x * ) + ( x j x * j ) ∂f ∂x j ( x * ) . (2.2.2) Then, for Eq. 2.1.2 to hold for both positive and negative values of ( x j x * j ), we must have the familiar first order condition ∂f ∂x j ( x * ) = 0 , ,j = 1 ,...,n (2.2.3) 15 CHAPTER 2. OPTIMALITY CONDITIONS Equation 2.2.3 is not applicable only to a minimum; the same condition is derived in the same manner for a maximum. It is called a stationarity condition, and a point satisfying it is called a stationary point. To obtain conditions specific to a minimizer, we need to use the second derivative terms in the Taylor expansion. However, to facilitate working with these terms, we would like to rewrite this expansion in matrix notation, which is more compact. We define the gradient vector to be the vector whose components are the first derivatives, and denote it as ∇ f . That is, ∇ f = ∂f ∂x 1 ∂f ∂x 2 . . . ∂f ∂x n (2.2.4) Similarly, we define the Hessian (after the German mathematician) Otto Ludwig Hesse (1811 1874)) matrix H , to be the symmetric matrix of second derivatives. That is, h ij , the element of the matrix H at the i th row and j th column is ∂ 2 ∂x i x j , H = ∂ 2 f ∂x 2 1 ∂ 2 f ∂x 1 x 2 ... ∂ 2 f ∂x 1 x n ∂ 2 f ∂x 2 x 1 ∂ 2 f ∂x 2 2 ......
View
Full Document
 Spring '08
 Staff
 Critical Point, Optimization

Click to edit the document details