In this chapter we consider the problem of finding zeros of a continuous
function
f
, i.e.
solving
f
(
x
) = 0 for example
e
x

x
= 0 or a system of
nonlinear equations:
f
1
(
x
1
, x
2
,
· · ·
, x
n
) = 0
,
f
2
(
x
1
, x
2
,
· · ·
, x
n
) = 0
,
.
.
.
f
n
(
x
1
, x
2
,
· · ·
, x
n
) = 0
.
(12.1)
We are going to write this generic system in vector form as
f
(
x
) =
0
,
(12.2)
where
f
:
U
⊆
R
n
→
R
n
. Unless otherwise noted the function
f
is assumed
to be smooth in its domain
U
.
We are going to start with the scalar case,
n
= 1 and look a very simple
but robust method that relies only on the continuity of the function and the
existence of a zero.
12.2
Bisection
Suppose we are interested in solving a nonlinear equation in one unknown
f
(
x
) = 0
,
(12.3)
193
194
CHAPTER 12.
NONLINEAR EQUATIONS
where
f
is a continuous function on an interval [
a, b
] and has at least one
zero there.
Suppose that
f
has values of different sign at the end points of the interval,
i.e.
f
(
a
)
f
(
b
)
<
0
.
(12.4)
By the Intermediate Value Theorem,
f
has at least one zero in (
a, b
).
To
locate a zero we bisect the interval and check on which subinterval
f
changes
sign. We repeat the process until we bracket a zero within a desired accuracy.
The Bisection algorithm to find a zero
x
*
is shown below.
Algorithm 7
The Bisection Method
1:
Given
f
,
a
and
b
(
a < b
),
TOL
, and
N
max
, set
k
= 1 and do:
2:
while
(
b

a
)
> TOL
and
k
≤
N
max
do
3:
c
= (
a
+
b
)
/
2
4:
if
f
(
c
) == 0
then
5:
x
*
=
c
.
This is the solution
6:
stop
7:
end if
8:
if
sign(
f
(
c
)) == sign(
f
(
a
))
then
9:
a
←
c
10:
else
11:
b
←
c
12:
end if
13:
k
←
k
+ 1
14:
end while
15:
x
*
←
(
a
+
b
)
/
2
12.2.1
Convergence of the Bisection Method
With the bisection method we generate a sequence
c
k
=
a
k
+
b
k
2
,
k
= 1
,
2
, . . .
(12.5)
where each
a
k
and
b
k
are the endpoints of the subinterval we select at each
bisection step (because
f
changes sign there). Since
b
k

a
k
=
b

a
2
k

1
,
k
= 1
,
2
, . . .
(12.6)
12.3.
RATE OF CONVERGENCE
195
and
c
k
=
a
k
+
b
k
2
is the midpoint of the interval then

c
k

x
*
 ≤
1
2
(
b
k

a
k
) =
b

a
2
k
(12.7)
and consequently
c
k
→
x
*
, a zero of
f
in [
a, b
].
12.3
Rate of Convergence
We now define in precise terms the rate of convergence of a sequence of
approximations to a value
x
*
.
Definition 21.
Suppose a sequence
{
x
n
}
∞
n
=1
converges to
x
*
as
n
→ ∞
. We
say that
x
n
→
x
*
of order
p
(
p
≥
1
) if there is a positive integer
N
and a
constant
C
such that

x
n
+1

x
*
 ≤
C

x
n

x
*

p
,
for all
n
≥
N.
(12.8)
or equivalently
lim
n
→∞

x
n
+1

x
*


x
n

x
*

p
=
C.
(12.9)
Example 32.
The sequence generated by the bisection method converges lin
early to
x
*
because

c
n
+1

x
*


c
n

x
*

≤
b

a
2
n
+1
b

a
2
n
=
1
2
.
Let’s examine the significance of the rate of convergence. Consider first,
p
= 1, linear convergence. Suppose

x
n
+1

x
*
 ≈
C

x
n

x
*

,
n
≥
N.
(12.10)
Then

x
N
+1

x
*
 ≈
C

x
N

x
*

,

x
N
+2

x
*
 ≈
C

x
N
+1

x
*
 ≈
C
(
C

x
N

x
*

) =
C
2

x
N

x
*

.
Continuing this way we get

x
N
+
k

x
*
 ≈
C
k

x
N

x
*

,
k
= 0
,
1
, . . .
(12.11)
196
CHAPTER 12.
NONLINEAR EQUATIONS
and this is the reason of the requirement
C <
1 for
p
= 1. If the error at the
N
step,

x
N

x
*

, is small enough it will be reduced by a factor of
C
k
after
k
more steps. Setting
C
k
= 10

d
k
, then the error

x
N

x
*

will be reduced
approximately
d
k
=
b
log
10
1
C
c
k
(12.12)
digits.around
Let us now do a similar analysis for
p
= 2, quadratic convergence. We