If we count up the operations above, we have to eliminate (
N

1)
N/
2
terms, each at a cost of
O
(
N
), so computing the
LU
factorization
is
O
(
N
3
) (same as using Gaussian elimination to solve
Ax
=
b
0
for
a particular
b
0
).
20
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019
Subscribe to view the full document.
Cholesky factorization
The basic idea in Cholesky factorization is that you can use elimi
nation to find an lowertriangular matrix
R
1
that eliminates all but
the first entry in the first column of
A
:
R
1
a
11
a
12
· · ·
a
1
N
a
21
a
22
· · ·
a
2
N
.
.
.
.
.
.
a
N
1
a
N
2
· · ·
a
NN
=
√
a
11
a
0
12
· · ·
a
0
1
N
0
a
0
22
· · ·
a
0
2
N
.
.
.
.
.
.
0
a
0
N
2
· · ·
a
0
NN
.
Since
A
is symmetric, we can do the same elimination on the columns
to get
R
1
AR
T
1
=
1
0
· · ·
0
0
a
00
22
· · ·
a
00
2
N
.
.
.
.
.
.
0
a
00
N
2
· · ·
a
00
NN
=
1
0
T
0
A
1
It is easy to see that
A
1
must also be symmetric (and positive defi
nite), so we continue by eliminating all but the first entry in its first
column using another lowertriangular matrix
R
2
, then doing the
same to the columns to get
R
2
R
1
AR
T
1
R
T
2
=
I
0
0
A
2
.
After
N
such iterations, we have
R
N
· · ·
R
1
AR
T
1
· · ·
R
T
N
=
RAR
T
=
I
.
Since the product of two lowertriangular matrices is again lower
triangular (show this on your own!),
R
is again lowertriangular.
Since the inverse of a lowertriangular matrix is again lower triangular
(show this on your own!), we can take
A
=
LL
T
,
with
L
=
R

1
.
21
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019
Symmetric
QR
factorization
We describe one algorithm for computing a symmetric
QR
factor
ization which is very efficient and fairly easy to understand, called
Household tridiagonalization
.
Let
A
be a symmetric matrix
A
=
a
11
a
12
· · ·
a
1
N
a
21
a
22
· · ·
a
2
N
.
.
.
.
.
.
.
.
.
a
N
1
a
N
2
· · ·
a
NN
,
and set
x
1
=
0
a
21
a
31
.
.
.
a
N
1
,
y
1
=
0
r
1
0
.
.
.
0
,
with
r
1
=
k
x
1
k
2
.
From
x
1
and
y
1
we create the
Householder matrix
H
1
=
I

2
u
1
u
T
1
,
u
1
=
x
1

y
1
k
x
1

y
1
k
2
.
Then by direct calculation, you can check that
A
2
=
H
1
AH
1
=
a
11
r
1
0
· · ·
0
r
1
˜
a
22
˜
a
23
· · ·
˜
a
2
N
0
˜
a
32
˜
a
33
· · ·
˜
a
3
N
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
0
˜
a
N
2
˜
a
N
3
· · ·
˜
a
NN
.
So applying
H
1
on either side eliminates all but two entries in both
the first row and first column of
A
.
22
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 13:22, November 13, 2019
Subscribe to view the full document.
Notice also that the
N
×
N
matrix
H
1
is orthnormal, as
H
T
1
H
1
= (
I

2
u
1
u
T
1
)(
I

2
u
1
u
T
1
)
=
I

4
u
1
u
T
1
+ 4
u
1
u
T
1
u
1
u
T
1
=
I
,
where the last equality follows since
u
T
1
u
1
=
k
u
1
k
2
2
= 1.
 Fall '08
 Staff