{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

284textpart2

# 284textpart2 - 2 Unconstrained descent-type methods 2.1...

This preview shows pages 1–4. Sign up to view the full content.

2. Unconstrained descent-type methods 2.1 Multivariate Analysis: a review We will briefly recall some basic concepts related to the study of multi-variate function before describing the optimizations techniques to find optima of these function. We consider here functions that are continuous, at least, have first and second derivatives. Tangent plane Tangent z=c 1 z=c 2 z=F(x 1 ,x 2 )=F(x ) F(x 1 ,x 2 )=c 1 F(x 1 ,x 2 )=c 2 x * ) ( x F Figure 1: definitions associated with a multi-variate function Consider the following n -dimensional multivariate function for which we are interested to find an optimum ( 29 x F ( 29 T n x x x x ___ , , 2 1 = n : F The gradient is defined as the following vector 13

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
( 29 ( 29 = n x F x F x g x F 1 Any plane (constant gradient) in n (a hyperplane) can be written as ( 29 α + = x c x F T ( 29 c x F = The matrix of the nxn partial derivatives is termed the Hessian ( 29 ( 29 - - - - = 2 2 1 2 1 2 2 1 2 2 | | n n n x F x x F x x F x F x G x F Note that this matrix is always symmetric. If the Hessian of F(x ) does not depend on x , we have a quadratic function. ( 29 + + = x c x G x x F T T 2 1 Next to gradients we can also calculate directional derivatives. A direction in n is given by the unit vector s 1 1 2 = = = = n i i T s s s s A line in n can be defined as ( 29 s x x + = ' Then define the scalar function ( 29 ( 29 ( 29 ( 29 s x F x F F + ' its derivate in a direction s is a directional derivative ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 s x F x F s x x F s d dx x x F d dF T T n i i i i n i i = = = α α = α α = = 1 1 The second derivative in that direction is then given by 14
( 29 ( 29 ( 29 ( 29 ( 29 ∑∑ = = = = = = = n i n j x F G T j i j i n i i i s x F s s x x x F s x x f s d d d F d 1 1 of curvature 2 2 1 2 2 e e e e e ee e e e α Note ( 29 ( 29 θ = cos x F s x F s T θ is the angle between s and ( 29 x F Of importance to understand some of the techniques is the Taylor expansion of multi- variate functions which are developed for a certain direction s ( 29 ( 29 ( 29 ( 29 ... ' 2 1 ' ' ' 2 + + + = + s x G s s x g x F s x F T T ( 29 ( 29 ( 29 ( 29 + + + = ... 0 " 2 1 0 ' 0 : 1 2 f f f f function D for recall Let’s now consider the conditions for a minimum for these n-dimensional functions. (1) Necessary condition : if ( 29 n x F : has continuous first derivatives and x * is a local minimum of F, then ( 29 0 * = x F (stationary point) (2) Sufficient condition : if ( 29 n x F : has continuous second derivatives and ( 29 0 * = x F and ( 29 ( 29 * * 2 x F x G = is positive definite , then x * is an isolated local minimum. (for a maximum we require that the Hessian is the maximum is negative definite) note (1) does not guarantee an isolated minimum. It could be a maximum, minimum or saddle-point.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 25

284textpart2 - 2 Unconstrained descent-type methods 2.1...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online