AIMS Lecture Notes 2006
Peter J. Olver
7. Iterative Methods for Linear Systems
Linear iteration coincides with multiplication by successive powers of a matrix; con
vergence of the iterates depends on the magnitude of its eigenvalues. We discuss in some
detail a variety of convergence criteria based on the spectral radius, on matrix norms, and
on eigenvalue estimates provided by the Gerschgorin Circle Theorem.
We will then turn our attention to the three most important iterative schemes used
to accurately approximate the solutions to linear algebraic systems. The classical Jacobi
method is the simplest, while an evident serialization leads to the popular Gauss–Seidel
method.
Completely general convergence criteria are hard to formulate, although con
vergence is assured for the important class of diagonally dominant matrices that arise in
many applications. A simple modification of the Gauss–Seidel scheme, known as Succes
sive OverRelaxation (SOR), can dramatically speed up the convergence rate, and is the
method of choice in many modern applications. Finally, we introduce the method of conju
gate gradients, a powerful “semidirect” iterative scheme that, in contrast to the classical
iterative schemes, is guaranteed to eventually produce the exact solution.
7.1. Linear Iterative Systems.
We begin with the basic definition of an iterative system of linear equations.
Definition 7.1.
A
linear iterative system
takes the form
u
(
k
+1)
=
T
u
(
k
)
,
u
(0)
=
a
.
(7
.
1)
The
coefficient matrix
T
has size
n
×
n
. We will consider both real and complex sys
tems, and so the
iterates
†
u
(
k
)
are vectors either in
R
n
(which assumes that the coefficient
matrix
T
is also real) or in
C
n
. For
k
= 1
,
2
,
3
, . . .
, the solution
u
(
k
)
is uniquely determined
by the
initial conditions
u
(0)
=
a
.
Powers of Matrices
The solution to the general linear iterative system (7.1) is, at least at first glance,
immediate. Clearly,
u
(1)
=
T
u
(0)
=
T
a
,
u
(2)
=
T
u
(1)
=
T
2
a
,
u
(3)
=
T
u
(2)
=
T
3
a
,
†
Warning
: The superscripts on
u
(
k
)
refer to the iterate number, and should not be mistaken
for derivatives.
10/18/06
103
c
circlecopyrt
2006
Peter J. Olver
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
and, in general,
u
(
k
)
=
T
k
a
.
(7
.
2)
Thus, the iterates are simply determined by multiplying the initial vector
a
by the succes
sive powers of the coefficient matrix
T
. And so, unlike differential equations, proving the
existence and uniqueness of solutions to an iterative system is completely trivial.
However, unlike real or complex scalars, the general formulae and qualitative behavior
of the powers of a square matrix are not nearly so immediately apparent.
(Before con
tinuing, the reader is urged to experiment with simple 2
×
2 matrices, trying to detect
patterns.) To make progress, recall how we managed to solve linear systems of differential
equations by suitably adapting the known exponential solution from the scalar version.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '09
 Olver
 Linear Algebra, Multiplication, Linear Systems, Gauss–Seidel method, Jacobi method, Peter J. Olver

Click to edit the document details