lecture14 - Yinyu Ye MS&E Stanford MS&E211 Lecture Note#14...

Info icon This preview shows pages 1–8. Sign up to view the full content.

View Full Document Right Arrow Icon
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 1 Nonlinear Optimization Algorithms II Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/˜yyye
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 2 Nonlinear Optimization: Primal Methods ( NLP ) minimize f ( x ) subject to g i ( x ) 0 i = 1 , ..., m. Primal Methods are methods that work on the original problem directly by searching through the feasible region of the problem . For simplicity, assume that the constrains are all linear g i ( x ) := a i x b i .
Image of page 2
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 3 Feasible Direction Methods Search along a feasible direction that is also descent at the same time. Let the active constraint set be A ( x k ) := { i : a i x k b i = 0 } at a feasible iterative point x k . Then we solve: ( FDM ) minimize f ( x k ) d subject to a i d 0 , i ∈ A ( x k ) d 1 . Suppose the minimizer is d k . If the minimal value is 0 , then x k is already a local minimizer ; otherwise x k +1 = x k + α k d k where α k is the step size to make x k +1 stay feasible and reduce the objective function the most.
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 4 Active Set Methods Guess which constraints would be active at the minimizer, say in a working set W k . Then we solve: ( ASM ) minimize f ( x ) subject to a i x b i = 0 , i W k . The minimizer can be computed from the KKT conditions for equality constrained problem (which is generally easier than inequality problem to solve), and let it be x k . If a i x k b i < 0 for some i W k , then ADD this violated constraint into W k and resolve (ASM); else if one of the Lagrange multipliers λ i is negative, then DROP the associate constraint from W k and resolve (ASM). If there is no need to ADD or DROP , x k is a KKT point for the original inequality constrained problem.
Image of page 4
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 5 Other Primal Methods The gradient projection method : project the gradient onto the feasible direction space. The reduced gradient method : like the simplex method by changing non-basic variables. The ellipsoid method : The basic ideas of the ellipsoid method stem from research done in the nineteen sixties and seventies mainly in the Soviet Union (as it was then called). The idea in a nutshell is to enclose the region of interest in each member of a sequence of ellipsoids whose size is decreasing, resembling the bisection method.
Image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 6 Linear Feasibility Problem The method discussed here is really aimed at finding an element of a solution set X given by a system of linear inequalities. X = { x R n : a i x b i 0 , i = 1 , . . . m } Finding an element of X can be thought of as being equivalent to solving a linear programming problem, though this requires a bit of discussion.
Image of page 6
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 7 Two technical assumptions (A1) X is contained by a ball centered at the origin with a radius R > 0 .
Image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 8
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern