EE364b
Prof. S. Boyd
EE364b Homework 6
1.
Conjugate gradient residuals.
Let
r
(
k
)
=
b
−
Ax
(
k
)
be the residual associated with the
k
th element of the Krylov sequence.
Show that
r
(
j
)
T
r
(
k
)
= 0 for
j
negationslash
=
k
.
In other
words, the Krylov sequence residuals are mutually orthogonal. Do not use the explicit
algorithm to show this property; use the basic definition of the Krylov sequence,
i.e.
,
x
(
k
)
minimizes (1
/
2)
x
T
Ax
−
b
T
x
over
K
k
.
Solution.
Since
x
(
k
)
minimizes
f
(
x
) = (1
/
2)
x
T
Ax
−
b
T
x
over
K
k
= span
{
b, Ab, . . . , A
k

1
b
}
,
∇
f
(
x
(
k
)
)
T
y
= 0 for all
y
∈ K
k
. The residual
r
(
k
)
=
b
−
Ax
(
k
)
=
−∇
f
(
x
(
k
)
), and for any
j < k
the residual
r
(
j
)
∈ K
k
. Therefore
r
(
k
)
T
r
(
j
)
= 0 for all
j
negationslash
=
k
.
2.
CG and PCG example.
In this problem you explore a variety of methods to solve
Ax
=
b
, where
A
∈
S
n
++
has block diagonal plus sparse structure:
A
=
A
blk
+
A
sp
,
where
A
blk
∈
S
n
++
is block diagonal and
A
sp
∈
S
n
is sparse. For simplicity we assume
A
blk
consists of
k
blocks of size
m
, so
n
=
mk
. The matrix
A
sp
has
N
nonzero elements.
(a) What is the approximate flop count for solving
Ax
=
b
if we treat
A
as dense?
(b) What is the approximate flop count for an iteration of CG? (Assume multiplication
by
A
blk
and
A
sp
are done exploiting their respective structures.) You can ignore
the handful of inner products that need to be computed.
(c) Now suppose that we use PCG, with preconditioner
M
=
A

1
blk
.
What is the
approximate flop count for computing the Cholesky factorization of
A
blk
? What
is the approximate flop count per iteration of PCG, once a Cholesky factorization
of
A
blk
if found?
(d) Now consider the specific problem with
A
blk
,
A
, and
b
generated by
ex_blockprecond.m
.
Solve the problem using direct methods, treating
A
as dense, and also treating
A
as sparse. Run CG on the problem for a hundred iterations or so, and plot the
relative residual versus iteration number. Run PCG on the same problem, using
the blockdiagonal preconditioner
M
=
A

1
blk
. Give the solution times for dense
direct, sparse direct, CG (to relative residual 10

4
, say), and PCG (to relative
residual 10

8
, say).
For PCG break out the time as time for initial Cholesky
factorization, and time for PCG iterations.
Hints.
•
To force Matlab to treat
A
as dense, use
full(A)
.
•
You do not need to implement the conjugate gradient algorithm; instead use the
pcg
function in Matlab.
•
To block precondition with
M
=
A

1
blk
, first find the Cholesky factorization of
A
blk
,
i.e.
, lower triangular
L
with
LL
T
=
A
blk
. The Matlab code to implement block
preconditioning is
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
L = chol(A_blk)’;
[...] = pcg(A,b,tolerance,MAXITER,L,L’);
Matlab uses a sparse Cholesky to factor
A
blk
, which is less efficient than using a
dense Cholesky factorization on each block separately, but it is efficient enough
to make the point.
Solution.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '09
 StephenBoyd
 Gate, Optimization, lower bound, PCG, Cholesky factorization

Click to edit the document details