This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: By applying the GaussNewton formula with the starting value 2, we can get the first and the second iterative value, x 1 and x 2 . Recall the GaussNewton formula is x n +1 = x n f ( x n ) f 00 ( x n ) . With the condition x = 2, we have x 1 = x f ( x ) f 00 ( x ) = 2 f (2) f 00 (2) = 0 . Next we plug in x 1 = 0 to the formula to get x 2 = 0 f (0) f 00 (0) = 4 / 5 . (b) (Ans) This time we chose a different starting value. Similarly we can get x 1 and x 2 from the iteration, now with x = 3. x 1 = x f ( x ) f 00 ( x ) = 3 f (3) f 00 (3) = 5 x 2 = x 1 f ( x 1 ) f 00 ( x 1 ) = 5 f (5) f 00 (5) = 21 / 5 . (c) (Ans) Under some regularity conditions, we expect that x n will approach to the point x * at which the local minimum (or maximum) is obtained. To be precise, x n converges to x * where f ( x * ) = 0. To explain this, let us take the limit on each side of the GaussNewton formula. Then we have the equation, lim x n +1 = lim x n lim f ( x n ) f 00 ( x n ) . Assuming the limit of the sequence { x n } exists (which we will denote as x * ), and f ( x ) and f 00 ( x ) are continuous. Then the above equation can be written as x * = x * f ( x * ) f 00 ( x * ) . 2 Here we have used the fact that lim x n +1 = lim x n = x * , and lim g ( x n ) = g (lim x n ) for any continuous function g ( x ). Therefore, we conclude that f ( x * ) is zero. In this example, the GaussNewton optimization process pushes the sequence towards two different limits depending on starting value. The limit of x n is 1 with the initial value x = 2 however, x n converges to 4 with the initial value 3. The limit of { x n } can be sensitive to initial values especially when there are multiple solutions of x in the equation f ( x ) = 0 . We can tell the direction which { x n } converges to by looking at the signs of f ( x n ) and f 00 ( x n ). Let’s go back to our example. From the form of derivatives f ( x ) and f 00 ( x ), you will see that f ( x n ) > 0 and f 00 ( x n ) < 0 when x n < 1 . Therefore, f ( x n ) f 00 ( x n ) < 0 and thus we know from the GaussNewton formula that the next value in the sequence is larger than x n , i.e.,, i....
View
Full Document
 Winter '08
 Stohs
 Linear Regression, Regression Analysis, Yi, Errors and residuals in statistics, Xn

Click to edit the document details