6smoothing

6smoothing - EE236C (Spring 2008-09) 6. Smoothing...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: EE236C (Spring 2008-09) 6. Smoothing techniques motivation smoothing via conjugate examples 61 First-order convex optimization methods complexity of finding -suboptimal point of f f differentiable O ( radicalbig L/ ) iterations with fast gradient method ( L is Lipschitz constant for f ) f = g + h with g differentiable, h a simple nondifferentiable function O ( radicalbig L/ ) iterations with fast gradient method ( L is Lipschitz constant for g ) f general nondifferentiable O ( G 2 / 2 ) iterations with subgradient method ( G is Lipschitz constant for f ) Smoothing techniques 62 Nondifferentiable optimization by smoothing make a differentiable approximation f and minimize by gradient method complexity: O ( radicalbig L/ ) iterations L is Lipschitz constant of f (smaller L means more smoothing) is accuracy for smooth problem; needs to be smaller than to account for approximation error trade-off in amount of smoothing (choice of L ) large L gives more accurate approximation small L gives faster convergence, but requires smaller is the overall complexity better than O (1 / 2 ) ? Smoothing techniques 63 Example: 1-norm approximation minimize f ( x ) = bardbl Ax b bardbl 1 = m summationdisplay i =1 | a T i x b i | Huber penalty as smoothed absolute value h ( z ) = braceleftbigg z 2 / (2 ) | z | | z | / 2 | z | controls accuracy and smoothness h ( z ) 1 / h ( z ) | z | h ( z ) + / 2 / 2 / 2 z h ( z ) Smoothing techniques 64 1-norm approximation by smoothing: take = /m and solve minimize f ( x ) = m summationdisplay i =1 h ( a T i x b i ) f is smooth with 2 f ( x ) precedesequal LI , L = max ( A T A ) = m bardbl A bardbl 2 if f ( x ) f / 2 , then f ( x ) f f ( x ) + m/ 2 f bound on #iterations to reach f ( x ( k ) ) f by fast gradient method: O ( radicalbig 2 L/ ) = O ( 2 m bardbl A bardbl ) cf. the O (1 / 2 ) bound for subgradient method Smoothing techniques 65 Strongly convex functions f is strongly convex with parameter > if dom...
View Full Document

This note was uploaded on 01/25/2010 for the course EE 236 taught by Professor Staff during the Spring '08 term at UCLA.

Page1 / 11

6smoothing - EE236C (Spring 2008-09) 6. Smoothing...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online