{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

6smoothing - EE236C(Spring 2008-09 6 Smoothing techniques...

Info icon This preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
EE236C (Spring 2008-09) 6. Smoothing techniques motivation smoothing via conjugate examples 6–1 First-order convex optimization methods complexity of finding ǫ -suboptimal point of f f differentiable O ( radicalbig L/ǫ ) iterations with fast gradient method ( L is Lipschitz constant for f ) f = g + h with g differentiable, h a ‘simple’ nondifferentiable function O ( radicalbig L/ǫ ) iterations with fast gradient method ( L is Lipschitz constant for g ) f general nondifferentiable O ( G 2 2 ) iterations with subgradient method ( G is Lipschitz constant for f ) Smoothing techniques 6–2
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Nondifferentiable optimization by smoothing make a differentiable approximation ˜ f and minimize by gradient method complexity: O ( radicalbig L/ ˜ ǫ ) iterations L is Lipschitz constant of ˜ f (smaller L means more smoothing) ˜ ǫ is accuracy for smooth problem; needs to be smaller than ǫ to account for approximation error trade-off in amount of smoothing (choice of L ) large L gives more accurate approximation small L gives faster convergence, but requires smaller ˜ ǫ is the overall complexity better than O (1 2 ) ? Smoothing techniques 6–3 Example: 1-norm approximation minimize f ( x ) = bardbl Ax b bardbl 1 = m summationdisplay i =1 | a T i x b i | Huber penalty as smoothed absolute value h μ ( z ) = braceleftbigg z 2 / (2 μ ) | z | ≤ μ | z | − μ/ 2 | z | ≥ μ μ controls accuracy and smoothness h ′′ μ ( z ) 1 h μ ( z ) ≤ | z | ≤ h μ ( z ) + μ/ 2 μ/ 2 μ/ 2 z h μ ( z ) Smoothing techniques 6–4
Image of page 2
1-norm approximation by smoothing: take μ = ǫ/m and solve minimize ˜ f ( x ) = m summationdisplay i =1 h μ ( a T i x b i ) ˜ f is smooth with 2 ˜ f ( x ) precedesequal LI , L = λ max ( A T A ) μ = m bardbl A bardbl 2 ǫ if ˜ f ( x ) ˜ f ǫ/ 2 , then f ( x ) f ˜ f ( x ) + mμ/ 2 ˜ f ǫ bound on #iterations to reach f ( x ( k ) ) f ǫ by fast gradient method: O ( radicalbig 2 L/ǫ ) = O ( 2 m bardbl A bardbl ǫ ) cf. the O (1 2 ) bound for subgradient method Smoothing techniques 6–5 Strongly convex functions f is strongly convex with parameter μ > 0 if dom f is convex and (1 θ ) f ( x ) + θf ( y ) f ((1 θ ) x + θy ) + μ θ (1 θ ) 2 bardbl x y bardbl 2 2 for all x, y dom f
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern