284textpart2

284textpart2 - 2. Unconstrained descent-type methods 2.1...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
2. Unconstrained descent-type methods 2.1 Multivariate Analysis: a review We will briefly recall some basic concepts related to the study of multi-variate function before describing the optimizations techniques to find optima of these function. We consider here functions that are continuous, at least, have first and second derivatives. Tangent plane Tangent z=c 1 z=c 2 z=F(x 1 ,x 2 )=F(x ) F(x 1 ,x 2 )=c 1 F(x 1 ,x 2 )=c 2 x * ) ( x F Figure 1: definitions associated with a multi-variate function Consider the following n -dimensional multivariate function for which we are interested to find an optimum ( 29 x F ( 29 T n x x x x ___ , , 2 1 = n : F The gradient is defined as the following vector 13
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
( 29 ( 29 = n x F x F x g x F 1 Any plane (constant gradient) in n (a hyperplane) can be written as ( 29 α + = x c x F T ( 29 c x F = The matrix of the nxn partial derivatives is termed the Hessian ( 29 ( 29 - - - - = 2 2 1 2 1 2 2 1 2 2 | | n n n x F x x F x x F x F x G x F Note that this matrix is always symmetric. If the Hessian of F(x ) does not depend on x , we have a quadratic function. ( 29 + + = x c x G x x F T T 2 1 Next to gradients we can also calculate directional derivatives. A direction in n is given by the unit vector s 1 1 2 = = = = n i i T s s s s A line in n can be defined as ( 29 s x x + = ' Then define the scalar function ( 29 ( 29 ( 29 ( 29 s x F x F F + ' its derivate in a direction s is a directional derivative ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 ( 29 s x F x F s x x F s d dx x x F d dF T T n i i i i n i i = = = α α = α α = = 1 1 The second derivative in that direction is then given by 14
Background image of page 2
( 29 ( 29 ( 29 ( 29 ( 29 ∑∑ = = = = = = = n i n j x F G T j i j i n i i i s x F s s x x x F s x x f s d d d F d 1 1 of curvature 2 2 1 2 2 e e e e e ee e e e α Note ( 29 ( 29 θ = cos x F s x F s T θ is the angle between s and ( 29 x F Of importance to understand some of the techniques is the Taylor expansion of multi- variate functions which are developed for a certain direction s ( 29 ( 29 ( 29 ( 29 ... ' 2 1 ' ' ' 2 + + + = + s x G s s x g x F s x F T T ( 29 ( 29 ( 29 ( 29 + + + = ... 0 " 2 1 0 ' 0 : 1 2 f f f f function D for recall Let’s now consider the conditions for a minimum for these n-dimensional functions. (1) Necessary condition : if ( 29 n x F : has continuous first derivatives and x * is a local minimum of F, then ( 29 0 * = x F (stationary point) (2) Sufficient condition : if ( 29 n x F : has continuous second derivatives and ( 29 0 * = x F and ( 29 ( 29 * * 2 x F x G = is positive definite , then x * is an isolated local minimum. (for a maximum we require that the Hessian is the maximum is negative definite) note (1) does not guarantee an isolated minimum. It could be a maximum, minimum or saddle-point.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 01/24/2011 for the course ERE 284 taught by Professor . during the Spring '10 term at Stanford.

Page1 / 25

284textpart2 - 2. Unconstrained descent-type methods 2.1...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online