EE364b
Prof. S. Boyd
EE364b Homework 6
1.
Conjugate gradient residuals.
Let
r
(
k
)
=
b
−
Ax
(
k
)
be the residual associated with the
k
th element of the Krylov sequence. Show that
r
(
j
)
T
r
(
k
)
= 0 for
j
n
=
k
. In other
words, the Krylov sequence residuals are mutually orthogonal. Do not use the explicit
algorithm to show this property; use the basic deFnition of the Krylov sequence,
i.e.
,
x
(
k
)
minimizes (1
/
2)
x
T
Ax
−
b
T
x
over
K
k
.
Solution.
Since
x
(
k
)
minimizes
f
(
x
) = (1
/
2)
x
T
Ax
−
b
T
x
over
K
k
= span
{
b,Ab,.
.. ,A
k

1
b
}
,
∇
f
(
x
(
k
)
)
T
y
= 0 for all
y
∈ K
k
. The residual
r
(
k
)
=
b
−
Ax
(
k
)
=
−∇
f
(
x
(
k
)
), and for any
j < k
the residual
r
(
j
)
∈ K
k
. Therefore
r
(
k
)
T
r
(
j
)
= 0 for all
j
n
=
k
.
2.
CG and PCG example.
In this problem you explore a variety of methods to solve
Ax
=
b
, where
A
∈
S
n
++
has block diagonal plus sparse structure:
A
=
A
blk
+
A
sp
,
where
A
blk
∈
S
n
++
is block diagonal and
A
sp
∈
S
n
is sparse. ±or simplicity we assume
A
blk
consists of
k
blocks of size
m
, so
n
=
mk
. The matrix
A
sp
has
N
nonzero elements.
(a) What is the approximate ²op count for solving
Ax
=
b
if we treat
A
as dense?
(b) What is the approximate ²op count for an iteration of CG? (Assume multiplication
by
A
blk
and
A
sp
are done exploiting their respective structures.) You can ignore
the handful of inner products that need to be computed.
(c) Now suppose that we use PCG, with preconditioner
M
=
A

1
blk
. What is the
approximate ²op count for computing the Cholesky factorization of
A
blk
? What
is the approximate ²op count per iteration of PCG, once a Cholesky factorization
of
A
blk
if found?
(d) Now consider the speciFc problem with
A
blk
,
A
, and
b
generated by
ex_blockprecond.m
.
Solve the problem using direct methods, treating
A
as dense, and also treating
A
as sparse. Run CG on the problem for a hundred iterations or so, and plot the
relative residual versus iteration number. Run PCG on the same problem, using
the blockdiagonal preconditioner
M
=
A

1
blk
. Give the solution times for dense
direct, sparse direct, CG (to relative residual 10

4
, say), and PCG (to relative
residual 10

8
, say). ±or PCG break out the time as time for initial Cholesky
factorization, and time for PCG iterations.
Hints.
•
To force Matlab to treat
A
as dense, use
full(A)
.
•
You do not need to implement the conjugate gradient algorithm; instead use the
pcg
function in Matlab.
•
To block precondition with
M
=
A

1
blk
, Frst Fnd the Cholesky factorization of
A
blk
,
i.e.
, lower triangular
L
with
LL
T
=
A
blk
. The Matlab code to implement block
preconditioning is
1