Coursehero >> Pennsylvania >> Lehigh >> IE 417

Course Hero has millions of student submitted documents similar to the one below including study guides, homework solutions, papers, exam answer keys and textbook solutions.

Outline Today's IE417: Nonlinear Programming: Lecture 8 Trust Regions Jeff Linderoth Department of Industrial and Systems Engineering Lehigh University Dogleg Example Convergence Proofs Homework for Next Time 9th February 2006 lehigh-logo lehigh-logo Jeff Linderoth IE417:Lecture 8 Jeff Linderoth IE417:Lecture 8 Stuff We Learned Last Time Trust Regions, use model function, mk (d): Trust Region Algorithm 1 mk (d) = f (xk ) + d Trust Region Subproblem (TRS) def Solve TRS to get dk Evaluate k Adjust TR If k < 1/4, Shrink: k+1 = 1/4 dk Else If k > 3/4 and dk = k , Grow: k+1 = 2k Else k+1 = k T f (xk ) + 1/2d Bk d T 2 3 dk arg min {mk (d)} d k 4 Move? [0, 1/4) If k > , Step: xk+1 = xk + dk Else Stay: xk+1 = xk Control of Trust Region depends on parameter k : k = def f (xk ) - f (xk + dk ) Predicted Reduction = mk (0) - mk (dk ) Actual Reduction lehigh-logo 5 Go to 1. lehigh-logo Jeff Linderoth IE417:Lecture 8 Jeff Linderoth IE417:Lecture 8 Solving TRS Finding the Cauchy Point We need not solve TRS exactly. Most algorithms are based on finding an improvement from the Cauchy Point Cauchy Point: The point that minimizes the model in the steepest descent direction, subject to the trust region constraint Cauchy Direction: s = k Cauchy "Step": Minimizer of min{mk ( s) | s k } >0 f (xk ) f (xk ) lehigh-logo lehigh-logo Jeff Linderoth IE417:Lecture 8 Jeff Linderoth IE417:Lecture 8 Finding the Cauchy Point Cauchy Step mk ( dc ) = - k mk ( dc ) = f (xk )- k T Case 1: dc Bk dc k k c ) is Then mk ( dk f (xk ) f (xk ) -1 k f (xk )T f (xk ) f (xk )T Bk f (xk ) f (xk ) -1 k f (xk )T f (xk )+1/2 2 dk Bk dc k 0) cT +(1/2) 2 2 k mk ( dc ) = 0 when k -2 < 0. (Bk decreasing function of , minimum at = 1 0. = k f (xk ) 3 f (xk )Bk f (xk ) Case 2: T dc Bk dc k k mk ( dc ) is convex quadratic function of . (Why?) k So gives us a closed form for the Cauchy "step length" ( ). Cauchy Step: dc = - k lehigh-logo f (xk ) -1 k f (xk ) lehigh-logo Jeff Linderoth IE417:Lecture 8 Jeff Linderoth IE417:Lecture 8 The Dogleg Method we Here switch notation like the book: f (xk ) = g, Bk = B, dC dN = - k g -B -1 g -1 g Theorem = Want to take some combination of these two steps Want to favor dC for small steps Want to favor dN for big steps It Only Crosses Once! Let B be positive definite, and let 2 1 2 0, then d(1 ) d(2 ) m(d(1 )) m(d(2 )) Dog Leg function (trajectory), d : R Rn d( ) = dC , 0 1 dC + ( - 1)(dN - dC ), 1 2 lehigh-logo lehigh-logo Jeff Linderoth IE417:Lecture 8 Jeff Linderoth IE417:Lecture 8 Beale What if B is not positive definite 2D Subspace Minimization min m(d) = {f + g T d + 1/2 dT Bd | d , d S} d def S = span(g, B -1 g) d span(d1 , d2 ) d = d1 + d2 for some , R Problem 4.9 How do you solve this problem? When B has negative eigenvalues, then change space of directions to def S = span(g, (B + I)-1 g) where > -1 (the most negative eigenvalue of B) lehigh-logo def Problem 4.10: Show that (B + I) is positive definite Jeff Linderoth IE417:Lecture 8 lehigh-logo Jeff Linderoth IE417:Lecture 8 Global Convergence Global Convergence Key idea: Subproblem solutions to DogLeg or 2D subspace minimization satisfy: mk (0) - mk (dk ) c1 f (xk ) min k , f (xk ) Bk (1) If min term is k : then this is like the Wolfe condition Model reduction is proportional to the norm of the gradient Theorem Let (0, 1/4) in Trust Region Algorithm. (i.e. we do make a reduction in the model function). Suppose that functions and model functions are "nice", and all steps dk satisfy , for c1 > 0, then lim f (xk ) = 0 k Cauchy Point dC satisfies (14) with c1 = 1/2 Dogleg/2D subspace minimization, also satisfy , since they are at least as good for the model function as the Cauchy Point. lehigh-logo lehigh-logo Jeff Linderoth IE417:Lecture 8 Jeff Linderoth IE417:Lecture 8 Scaling (Additional?) Homework Scaling trust region can help: min mk (d) = {f + g d + 1/2d Bd | Dd k d def T T Optional, Implement Dogleg method. (4.2). Can use it to run on CUTEr instances. Read 5.1, 8.1 We've only done 2, but you can do other norms as well: lehigh-logo lehigh-logo Jeff Linderoth IE417:Lecture 8 Jeff Linderoth IE417:Lecture 8

Find millions of documents here - Study Guides, Homework Solutions, Papers, Exam Answer Keys and more. Course Hero has millions of course related materials that will enable you to learn better, faster and get an A in all your courses.
Below is a small sample set of documents: