Lect04

Lect04 - EE553 LECTURE 4 LECTURE OUTLINE • Principal...

Info iconThis preview shows pages 1–5. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE553 LECTURE 4 LECTURE OUTLINE • Principal Gradient Methods • Gradient Methods - Choices of Direction • Gradient Methods - Choice of Stepsize • Gradient Methods - Convergence Issues CONVERGENCE RESULTS CONSTANT STEPSIZE Let { x k } be a sequence generated by a gradient method x k +1 = x k + α k d k , where { d k } is gradient related. Assume that for some constant L > , we have k∇ f ( x )- ∇ f ( y ) k ≤ L k x- y k , ∀ x,y ∈ < n , Also, assume that there exists a scalar such that for all k < ≤ α k ≤ (2- ) |∇ f ( x k ) d k | L k d k k 2 . Then either f ( x k ) → -∞ or else { f ( x k ) } converges and every limit point of { x k } is stationary. MAIN PROOF IDEA α α∇ f(x k )'d k + (1/2) α 2 L||d k || 2 × α∇ f(x k )'d k α = | ∇ f(x k )'d k | L||d k || |2 f(x k + α d k ) - f(x k ) The idea of the convergence proof for a constant stepsize. Given x k and the descent direction d k , the cost difference f ( x k + αd k )- f ( x k ) is ma- jorized by α ∇ f ( x k ) d k + 1 2 α 2 L k d k k 2 (based on the Lipschitz assumption; see descent lemma). Mini- mization of this function over α yields the stepsize α = |∇ f ( x k ) d k | L k d k k 2 This stepsize reduces the cost function f as well. PRINCIPAL GRADIENT METHODS x k +1 = x k + α k d k , k = 0 , 1 ,......
View Full Document

This note was uploaded on 02/27/2008 for the course EE 553 taught by Professor Safonov during the Spring '08 term at USC.

Page1 / 15

Lect04 - EE553 LECTURE 4 LECTURE OUTLINE • Principal...

This preview shows document pages 1 - 5. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online