hw3sol_fa09

# hw3sol_fa09 - UC Berkeley Department of Electrical...

This preview shows pages 1–3. Sign up to view the full content.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: UC Berkeley Department of Electrical Engineering and Computer Science EECS 227A Nonlinear and Convex Optimization Solutions 3 Fall 2009 Solution 3.1 (a) We have ∇ f ( x ) = β bardbl x bardbl β − 2 x and ∇ 2 f ( x ) = β ( β − 2) bardbl x bardbl β − 4 xx T + β bardbl x bardbl β − 2 I , so ∇ 2 f ( x ) x = β ( β − 2) bardbl x bardbl β − 4 xx T x + β bardbl x bardbl β − 2 x = β ( β − 2) bardbl x bardbl β − 2 x + β bardbl x bardbl β − 2 x = ( β − 1) β bardbl x bardbl β − 2 x = ( β − 1) ∇ f ( x ) Thus ( ∇ 2 f ( x )) − 1 ∇ f ( x ) = x/ ( β − 1). Alternatively, we could use Sherman-Morrison formula to find the inverse of the Hessian ( A + CBC T ) − 1 = A − 1 − A − 1 C ( B − 1 + C T A − 1 C ) − 1 C T A − 1 The pure Newton’s method has the form x k +1 = x k − 1 β − 1 x k = β − 2 β − 1 x k ⇒ bardbl x k +1 bardbl = vextendsingle vextendsingle vextendsingle vextendsingle β − 2 β − 1 vextendsingle vextendsingle vextendsingle vextendsingle bardbl x k bardbl Hence • If β > 3 2 then vextendsingle vextendsingle vextendsingle β − 2 β − 1 vextendsingle vextendsingle vextendsingle < 1 and the method converges to 0 for any initial point. • If β = 3 2 , we have x k +1 = − x k , ∀ k so it does not converges for any initial point x negationslash = 0. • For 1 < β < 3 2 , we have vextendsingle vextendsingle vextendsingle β − 2 β − 1 vextendsingle vextendsingle vextendsingle > 1 and the diverges for all x negationslash = 0. • When β ≤ 1, we have ( ∇ 2 f ( x )) − 1 = 1 β bardbl x bardbl β − 2 parenleftbigg I − β − 2 β − 1 xx T bardbl x bardbl 2 parenrightbigg so ( ∇ 2 f ( x )) − 1 does not exist • If β < 1 then vextendsingle vextendsingle vextendsingle β − 2 β − 1 vextendsingle vextendsingle vextendsingle > 1 and the method diverges for any initial point x negationslash = 0 (b) With the Armijo rule we have bardbl x k +1 bardbl = vextendsingle vextendsingle vextendsingle vextendsingle 1 − α k β − 1 vextendsingle vextendsingle vextendsingle vextendsingle bardbl x k bardbl At each step, Armijo rule sets the stepsize α k = a m k s where s is the initial stepsize, a is the reduction factor, and m k is the smallest nonnegative integer such that f ( x k +1 ) − f ( x k ) ≤ σa m k s ∇ f ( x k ) T d k 1 which is equivalent to bardbl x k bardbl parenleftBigg 1 − vextendsingle vextendsingle vextendsingle vextendsingle 1 − α k β − 1 vextendsingle vextendsingle vextendsingle vextendsingle β parenrightBigg ≥ σα k β bardbl x k bardbl β β − 1 ⇔ 1 − vextendsingle vextendsingle vextendsingle vextendsingle 1 − α k β − 1 vextendsingle vextendsingle vextendsingle vextendsingle β ≥ σα k β β − 1 > Therefore, the stepsize is the same at each iteration α k = α, ∀ k...
View Full Document

{[ snackBarMessage ]}

### Page1 / 6

hw3sol_fa09 - UC Berkeley Department of Electrical...

This preview shows document pages 1 - 3. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online