Lecture-6 - Lecture-6 Convergence and order of convergence...

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon
1 Lecture-6 Convergence and order of convergence Line Search Methods k k k k p x x a + + 1 k k k f B p - - 1 Steepest descent is and identity matrix Newton is a Hessian matrix Quasi-Newton is approximation to the Hessian matrix k B k B k B
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
2 Line Search Methods k k k k p x x a + + 1 j i 0 2200 = j T i Ap p Conjugate gradient Important Questions • What are the conditions under which, the method converges? • What is the rate of convergence?
Background image of page 2
3 Conditions of convergence • Steepest Descent: Wolf’s conditions • Newton and Quasi-Newton: In addition to Wolfe’s conditions, PD Hessian, and bounded condition number • Conjugate Gradient: subsequence of direction cosines is bounded away from zero. k q cos Convergence Rate • Steepest descent: Linear • Quasi-Newton: Super-linear • Newton: Quadratic • Conjugate Gradient: n steps
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
4 Convergence of Line Search Methods • The steepest descent method is globally convergent • For other algorithms how far p k can deviate from the steepest descent direction and still gives rise to globally convergent iteration. Convergence of Line Search Methods (Theorem 3.2) || ||| || cos k k k T k k p f p f - = q < 0 2 2 || || cos k k k f q The angle between p k and steepest descent direction T k f - We will show (Theorem 3.2):
Background image of page 4
5 Convergence of Line Search Methods k T k k T k k p f c p f f - - + ) 1 ( ) ( 2 1 2 1 1 1 || || ) ( || |||| || || || || ) ( || ) ( k k k T k k k k k k T k k k T k k p L p f f p p L p f f p f f a a - - - + + + k k k k p x x a + = + 1 Curvature condition Iteration scheme Therefore ) 1 , ( , ) ( ) ( 1 2 2 c c p x f c p p x f k k T k k T k k + a Lipschitz continuous || ~ || || ) ~ ( ) ( || x
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 6
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 06/12/2011 for the course COT 6505 taught by Professor Shah during the Spring '07 term at University of Central Florida.

Page1 / 13

Lecture-6 - Lecture-6 Convergence and order of convergence...

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online