lecture14

# lecture14 - Yinyu Ye MS&E Stanford MS&E211 Lecture Note#14...

This preview shows pages 1–8. Sign up to view the full content.

Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 1 Nonlinear Optimization Algorithms II Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/˜yyye

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 2 Nonlinear Optimization: Primal Methods ( NLP ) minimize f ( x ) subject to g i ( x ) 0 i = 1 , ..., m. Primal Methods are methods that work on the original problem directly by searching through the feasible region of the problem . For simplicity, assume that the constrains are all linear g i ( x ) := a i x b i .
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 3 Feasible Direction Methods Search along a feasible direction that is also descent at the same time. Let the active constraint set be A ( x k ) := { i : a i x k b i = 0 } at a feasible iterative point x k . Then we solve: ( FDM ) minimize f ( x k ) d subject to a i d 0 , i ∈ A ( x k ) d 1 . Suppose the minimizer is d k . If the minimal value is 0 , then x k is already a local minimizer ; otherwise x k +1 = x k + α k d k where α k is the step size to make x k +1 stay feasible and reduce the objective function the most.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 4 Active Set Methods Guess which constraints would be active at the minimizer, say in a working set W k . Then we solve: ( ASM ) minimize f ( x ) subject to a i x b i = 0 , i W k . The minimizer can be computed from the KKT conditions for equality constrained problem (which is generally easier than inequality problem to solve), and let it be x k . If a i x k b i < 0 for some i W k , then ADD this violated constraint into W k and resolve (ASM); else if one of the Lagrange multipliers λ i is negative, then DROP the associate constraint from W k and resolve (ASM). If there is no need to ADD or DROP , x k is a KKT point for the original inequality constrained problem.
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 5 Other Primal Methods The gradient projection method : project the gradient onto the feasible direction space. The reduced gradient method : like the simplex method by changing non-basic variables. The ellipsoid method : The basic ideas of the ellipsoid method stem from research done in the nineteen sixties and seventies mainly in the Soviet Union (as it was then called). The idea in a nutshell is to enclose the region of interest in each member of a sequence of ellipsoids whose size is decreasing, resembling the bisection method.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 6 Linear Feasibility Problem The method discussed here is really aimed at finding an element of a solution set X given by a system of linear inequalities. X = { x R n : a i x b i 0 , i = 1 , . . . m } Finding an element of X can be thought of as being equivalent to solving a linear programming problem, though this requires a bit of discussion.
Yinyu Ye, MS&E, Stanford MS&E211 Lecture Note #14 7 Two technical assumptions (A1) X is contained by a ball centered at the origin with a radius R > 0 .

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern