hw3sol_fa09

hw3sol_fa09 - UC Berkeley Department of Electrical...

Info iconThis preview shows pages 1–3. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 3 Fall 2009 Solution 3.1 (a) We have f ( x ) = bardbl x bardbl 2 x and 2 f ( x ) = ( 2) bardbl x bardbl 4 xx T + bardbl x bardbl 2 I , so 2 f ( x ) x = ( 2) bardbl x bardbl 4 xx T x + bardbl x bardbl 2 x = ( 2) bardbl x bardbl 2 x + bardbl x bardbl 2 x = ( 1) bardbl x bardbl 2 x = ( 1) f ( x ) Thus ( 2 f ( x )) 1 f ( x ) = x/ ( 1). Alternatively, we could use Sherman-Morrison formula to find the inverse of the Hessian ( A + CBC T ) 1 = A 1 A 1 C ( B 1 + C T A 1 C ) 1 C T A 1 The pure Newtons method has the form x k +1 = x k 1 1 x k = 2 1 x k bardbl x k +1 bardbl = vextendsingle vextendsingle vextendsingle vextendsingle 2 1 vextendsingle vextendsingle vextendsingle vextendsingle bardbl x k bardbl Hence If > 3 2 then vextendsingle vextendsingle vextendsingle 2 1 vextendsingle vextendsingle vextendsingle < 1 and the method converges to 0 for any initial point. If = 3 2 , we have x k +1 = x k , k so it does not converges for any initial point x negationslash = 0. For 1 < < 3 2 , we have vextendsingle vextendsingle vextendsingle 2 1 vextendsingle vextendsingle vextendsingle > 1 and the diverges for all x negationslash = 0. When 1, we have ( 2 f ( x )) 1 = 1 bardbl x bardbl 2 parenleftbigg I 2 1 xx T bardbl x bardbl 2 parenrightbigg so ( 2 f ( x )) 1 does not exist If < 1 then vextendsingle vextendsingle vextendsingle 2 1 vextendsingle vextendsingle vextendsingle > 1 and the method diverges for any initial point x negationslash = 0 (b) With the Armijo rule we have bardbl x k +1 bardbl = vextendsingle vextendsingle vextendsingle vextendsingle 1 k 1 vextendsingle vextendsingle vextendsingle vextendsingle bardbl x k bardbl At each step, Armijo rule sets the stepsize k = a m k s where s is the initial stepsize, a is the reduction factor, and m k is the smallest nonnegative integer such that f ( x k +1 ) f ( x k ) a m k s f ( x k ) T d k 1 which is equivalent to bardbl x k bardbl parenleftBigg 1 vextendsingle vextendsingle vextendsingle vextendsingle 1 k 1 vextendsingle vextendsingle vextendsingle vextendsingle parenrightBigg k bardbl x k bardbl 1 1 vextendsingle vextendsingle vextendsingle vextendsingle 1 k 1 vextendsingle vextendsingle vextendsingle vextendsingle k 1 > Therefore, the stepsize is the same at each iteration k = , k...
View Full Document

Page1 / 6

hw3sol_fa09 - UC Berkeley Department of Electrical...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online