With the condition
x
0
= 2, we have
x
1
=
x
0

f
0
(
x
0
)
f
00
(
x
0
)
= 2

f
0
(2)
f
00
(2)
= 0
.
Next we plug in
x
1
= 0
to the formula to get
x
2
= 0

f
0
(0)
f
00
(0)
= 4
/
5
.
(b)
(Ans)
This time we chose a different starting value. Similarly we can get
x
1
and
x
2
from
the iteration, now with
x
0
= 3.
x
1
=
x
0

f
0
(
x
0
)
f
00
(
x
0
)
= 3

f
0
(3)
f
00
(3)
= 5
x
2
=
x
1

f
0
(
x
1
)
f
00
(
x
1
)
= 5

f
0
(5)
f
00
(5)
= 21
/
5
.
(c)
(Ans)
Under some regularity conditions, we expect that
x
n
will approach to the point
x
*
at which the local minimum (or maximum) is obtained. To be precise,
x
n
converges to
x
*
where
f
0
(
x
*
) = 0. To explain this, let us take the limit on each side of the GaussNewton formula. Then
we have the equation,
lim
x
n
+1
= lim
x
n

lim
f
0
(
x
n
)
f
00
(
x
n
)
.
Assuming the limit of the sequence
{
x
n
}
exists (which we will denote as
x
*
), and
f
0
(
x
) and
f
00
(
x
) are continuous. Then the above equation can be written as
x
*
=
x
*

f
0
(
x
*
)
f
00
(
x
*
)
.
2
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Here we have used the fact that lim
x
n
+1
= lim
x
n
=
x
*
, and lim
g
(
x
n
) =
g
(lim
x
n
) for any
continuous function
g
(
x
). Therefore, we conclude that
f
0
(
x
*
) is zero.
In this example, the GaussNewton optimization process pushes the sequence towards two
different limits depending on starting value. The limit of
x
n
is 1 with the initial value
x
0
= 2
however,
x
n
converges to 4 with the initial value 3. The limit of
{
x
n
}
can be sensitive to initial
values especially when there are multiple solutions of
x
in the equation
f
0
(
x
) = 0
.
We can tell the direction which
{
x
n
}
converges to by looking at the signs of
f
0
(
x
n
) and
f
00
(
x
n
).
Let’s go back to our example. From the form of derivatives
f
0
(
x
) and
f
00
(
x
), you will see that
f
0
(
x
n
)
>
0 and
f
00
(
x
n
)
<
0 when
x
n
<
1
.
Therefore,
f
0
(
x
n
)
f
00
(
x
n
)
<
0 and thus we know from the
GaussNewton formula that the next value in the sequence is larger than
x
n
, i.e.,
x
n
+1
> x
n
.
This implies that as we repeat the iteration, we will get larger value as long as the sequence moves
along the range in
x
n
<
1. Likewise, we can find where the sequence moves to for all
x
n
:
This is the end of the preview.
Sign up
to
access the rest of the document.
 Winter '08
 Stohs
 Linear Regression, Regression Analysis, Yi, Errors and residuals in statistics, Xn

Click to edit the document details