{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

3sgmethod - EE236C(Spring 2008-09 3 Subgradient method...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
EE236C (Spring 2008-09) 3. Subgradient method subgradient method convergence analysis optimal step size when f is known alternating projections optimality 3–1 Subgradient method to minimize a nondifferentiable convex function f : choose x (0) and repeat x ( k ) = x ( k 1) t k g ( k 1) , k = 1 , 2 , . . . g ( k 1) is any subgradient of f at x ( k 1) step size rules fixed step: t k constant fixed length: t k bardbl g ( k 1) bardbl 2 constant ( i.e. , bardbl x ( k ) x ( k 1) bardbl 2 constant) diminishing: t k 0 , k =1 t k = Subgradient method 3–2
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Assumptions f has finite optimal value f , minimizer x f is convex, dom f = R n f is Lipschitz continuous with constant G > 0 : | f ( x ) f ( y ) | ≤ G bardbl x y bardbl 2 x, y this is equivalent to bardbl g bardbl 2 G for all g ∂f ( x ) , all x Subgradient method 3–3 Analysis the subgradient method is not a descent method the key quantity in the analysis is the distance to the optimal set bardbl x ( i ) x bardbl 2 2 = vextenddouble vextenddouble vextenddouble x ( i 1) t i g ( i 1) x vextenddouble vextenddouble vextenddouble 2 2 = bardbl x ( i 1) x bardbl 2 2 2 t i g ( i 1) T ( x ( i 1) x ) + t 2 i bardbl g ( i 1) bardbl 2 2 bardbl x ( i 1) x bardbl 2 2 2 t i parenleftBig f ( x ( i 1) ) f parenrightBig + t 2 i bardbl g ( i 1) bardbl 2 2 define f ( k ) best = min 0 i<k f ( x ( i ) ) , and combine inequalities for i = 1 , . . . , k : 2( k summationdisplay i =1 t i ) parenleftBig f ( k ) best f parenrightBig bardbl x (0) x bardbl 2 2 − bardbl x ( k ) x bardbl 2 2 + k summationdisplay i =1 t 2 i bardbl g ( i 1) bardbl 2 2 bardbl x (0) x bardbl 2 2 + k summationdisplay i =1 t 2 i bardbl g ( i 1) bardbl 2 2 Subgradient method 3–4
Background image of page 2
fixed step size t i = t f ( k ) best f bardbl x (0) x bardbl 2 2 + kt 2 G 2 2 kt does not guarantee convergence of f ( k ) best for large k , f ( k ) best
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}