In this case,
L
will not have 1’s along the diagonal. Algorithms used
to compute Cholesky factorization are similar to those that compute
LU
factorizations. See the technical details at the end of these notes
for further detail.
The Cholesky decomposition is unique. It can actually be computed
slightly faster than a general
LU
decomposition, and is easier to
stabilize.
Example.
6 3 0
3 4 1
0 1 3
=
2
.
4495
0
0
1
.
2247 1
.
5811
0
0
0
.
6325 1
.
6125
2
.
4495 1
.
2247
0
0
1
.
5811 0
.
6325
0
0
1
.
6125
6
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019
Subscribe to view the full document.
QR
factorization
The
QR
decomposition factors
A
as
A
=
QR
,
where
Q
is orthogonal, and
R
is upper triangular. It can be com
puted by running (a stabilized version of) GramSchmidt on the
columns of
A
; its computational complexity is again
O
(
N
3
). Once
we have it in hand, solving
Ax
=
b
has the cost of solving an or
thogonal system (
O
(
N
2
)) and a triangular system (
O
(
N
2
)).
We will explore this connection more on the next homework.
Like all of the other decompositions in this section, computing a
QR
decomposition costs
O
(
N
3
), but once it is in place, we can solve
Ax
=
b
using
x
=
R

1
Q
T
b
in
O
(
N
2
) time.
Symmetric
QR
When
A
is symmetric, we can write
A
=
QT Q
T
,
where
Q
is orthonormal, and
T
is symmetric and
tridiagonal
:
T
=
t
11
t
12
0
0
· · ·
0
t
21
t
22
t
23
0
· · ·
0
0
t
32
t
33
t
34
· · ·
0
.
.
.
.
.
.
.
.
.
0
· · ·
0
t
NN

1
t
NN
.
7
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019
This is a handy fact, since in general, tridiagonal matrices are easier
to manipulate (invert, compute eigenvalues/eigenvectors of, etc) that
general symmetric matrices.
We call this “symmetric
QR
” since algorithms to compute this de
composition are very similar to those used to compute
A
=
QR
.
See the technical details at the end of these notes for an example of
such an algorithm.
SVD and eigenvalue decompositions
We are already familiar with the SVD for general matrices:
A
=
U
Σ
V
T
.
When
A
is square and invertible (rank(
A
) =
N
), then all of
U
,
Σ
,
and
V
are
N
×
N
and
UU
T
=
U
T
U
=
V V
T
=
V
T
V
=
I
. As
we have seen, we can solve
Ax
=
b
with
x
=
V
Σ

1
U
T
b
.
We can see that with the SVD in place, the cost of solving a system
is
O
(
N
2
).
For symmetric
A
, we can write
A
=
V
Λ
V
T
,
and then
Ax
=
b
is solved with
x
=
V
Λ

1
V
T
b
.
Both of these decompositions represent a matrix as orthogonaldiagonal
orthogonal.
Computing either costs
O
(
N
3
), and they are slightly
more expensive than the
QR
and
LU
decompositions above.
In
fact, computing a
QR
decomposition is often used as a stepping
stone to computing the SVD or eigenvalue decomposition.
8
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019
Subscribe to view the full document.
Computing eigenvalue decompositions of
symmetric matrices
For
N
×
N
symmetric positive semidefinite
A
, there are many ways
to compute the eigenvalue decomposition
A
=
V
Λ
V
T
. We discuss
here one particular technique, popular for its stability, flexibility, and
speed, based on
power iterations.
 Fall '08
 Staff