This preview shows pages 1–4. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: EE236C (Spring 200809) 6. Smoothing techniques motivation smoothing via conjugate examples 61 Firstorder convex optimization methods complexity of finding suboptimal point of f f differentiable O ( radicalbig L/ ) iterations with fast gradient method ( L is Lipschitz constant for f ) f = g + h with g differentiable, h a simple nondifferentiable function O ( radicalbig L/ ) iterations with fast gradient method ( L is Lipschitz constant for g ) f general nondifferentiable O ( G 2 / 2 ) iterations with subgradient method ( G is Lipschitz constant for f ) Smoothing techniques 62 Nondifferentiable optimization by smoothing make a differentiable approximation f and minimize by gradient method complexity: O ( radicalbig L/ ) iterations L is Lipschitz constant of f (smaller L means more smoothing) is accuracy for smooth problem; needs to be smaller than to account for approximation error tradeoff in amount of smoothing (choice of L ) large L gives more accurate approximation small L gives faster convergence, but requires smaller is the overall complexity better than O (1 / 2 ) ? Smoothing techniques 63 Example: 1norm approximation minimize f ( x ) = bardbl Ax b bardbl 1 = m summationdisplay i =1  a T i x b i  Huber penalty as smoothed absolute value h ( z ) = braceleftbigg z 2 / (2 )  z   z  / 2  z  controls accuracy and smoothness h ( z ) 1 / h ( z )  z  h ( z ) + / 2 / 2 / 2 z h ( z ) Smoothing techniques 64 1norm approximation by smoothing: take = /m and solve minimize f ( x ) = m summationdisplay i =1 h ( a T i x b i ) f is smooth with 2 f ( x ) precedesequal LI , L = max ( A T A ) = m bardbl A bardbl 2 if f ( x ) f / 2 , then f ( x ) f f ( x ) + m/ 2 f bound on #iterations to reach f ( x ( k ) ) f by fast gradient method: O ( radicalbig 2 L/ ) = O ( 2 m bardbl A bardbl ) cf. the O (1 / 2 ) bound for subgradient method Smoothing techniques 65 Strongly convex functions f is strongly convex with parameter &gt; if dom...
View
Full
Document
This note was uploaded on 01/25/2010 for the course EE 236 taught by Professor Staff during the Spring '08 term at UCLA.
 Spring '08
 staff
 Gate

Click to edit the document details