466/666: CO Continuous Optimization Winter 2008 Problem Set 4 S. Vavasis Handed out: 2008-Feb-11. Due: 2008-Feb-25 in lecture.
1. Consider the trust region subproblem in which the linear term is absent: min m(p) = pT Bp/2 s.t. p . Give a simple characterization of the global minimizer. [Hint: You may need more than one case. The theorem given in lecture about maximizing and minimizing xT Ax subject to x = 1 may help.] 2. The textbook states that (4.44) on p. 87 can be derived from (4.43) by elementary manipulation. Carry out this derivation yourself. [Hints: To get started, you will need to gure out a formula for p (), where p() is dened in the middle of p. 84 of the text. One way to obtain p () uses a Taylor series to estimate (B + ( + h)I)1 for small h. The relevant Taylor series, which you may use without proof, is that (I E)1 = I +E +E 2 +E 3 + , provided that E < 1. At one step in my argument, I needed the following equality: gT R1 RT R1 RT R1 RT g = RT R1 RT g 2 .] 3. Implement the trust region method in one dimension. Use exact Hessian and exact solution of the TRS, which is fairly easy in one dimension since it boils down to consideration of a few cases and doesnt require solution of a nonlinear equation. Terminate when the absolute value of the derivative is less than a tolerance and the second derivative is at least as big as the negative of that tolerance. Test your algorithm on an exact convex quadratic see to if the behavior predicted in PS3 actually occurs. Test your algorithm on nice function that is not quadratic: the function in PS3, Q2. What happens to the trust region radius? Test your algorithm on cos(x) starting with x=0 to make sure that it doesnt terminate at a point that satises the rst but not second order necessary conditions. Finally, for three extra bonus points, see if you can nd a pathological C 1 function so that the trust region method gets stuck (e.g., the radius shrinks to a very small number at a point not close to a local minimizer). My example is a fairly complicated C 1 convex function involving sinint(1/x), but maybe yours will be more straightforward. Hand in: listings of all functions, printouts of test runs. 4. (For grad students.) On p. 76 of the text, the two-dimensional subspace approach to solving the TRS ...