slide8_handout-1

Find dk f xk k d k d f x d xk 1 xk k dk 2

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: R. We want to solve minx∈Rn f (x) ￿ ￿ ￿ ￿ Looking for x∗ at which ∇f (x∗ ) = 0 At k th step, choose search direction dk = −∇f (xk ) xk +1 = xk + αk dk where αk ∈ arg min f (xk + αdk ) Specifically, select αk such that d k d α f (x + α dk ) = 0 30 / 45 Steepest Gradient Descent Given: ￿ ￿ ￿ Continuously differentiable f : Rn → R ￿>0 Starting point x0 with k = 0 Algorithm: 1. Find dk = −∇f (xk ) k d k d α f (x + α d ) = xk +1 = xk + αk dk 2. Solve 3. Let 0 to find αk 4. If ￿xk +1 − xk ￿2 < ￿ return xk +1 . Otherwise, return to step 1. 31 / 45 Steepest Gradient Descent Example 2 2 Let f (x) = x1 + 2x2 + x1 x2 + 3x2 with x0 = (0, 0) and ￿ = .001. Minimixe f (x) 32 / 45 Steepest Gradient Descent Example 2 2 Let f (x) = x1 + 2x2 + x1 x2 + 3x2 with x0 = (0, 0) and ￿ = .01. Minimize f (x) First Iteration ￿ ￿ ∇ f ( x) = ￿ ￿ ￿ 2x 1 + x 2 4x 2 + x 1 + 3 ￿ ￿￿ ￿ 2(0) + (0) 0 d0 = −∇f (x0 ) = − = 4(0) + 0 + 3 −3 ￿ ...
View Full Document

This note was uploaded on 12/10/2013 for the course MS&E 211 taught by Professor Yinyuye during the Fall '07 term at Stanford.

Ask a homework question - tutors are online