1
Math 1b Practical — March 7, 2011
Singular value decomposition; psuedoinverses
Recall that every symmetric matrix can be written as
UDU
>
where
D
is nonnegative
diagonal and
U
is orthogonal. Something similar can be done even if the matrix is not
symmetric, and perhaps not even square.
The topic of SVD deals with minimizing and approximating, and so it has many
applications.
Theorem 1.
Let
A
be a
m
×
n
real matrix. Then there exist orthogonal matrices
U
and
V
of order
m
and
n
, and a nonnegative ‘diagonal matrix’
D
,sothat
A
=
UDV
>
.
Here
D
is
m
×
n
and ‘diagonal’ means that all entries are zero except possibly the
(
i,i
)

entries where
i
≤
min
{
m,n
}
.
Proof:
We prove the theorem only in the case that
A
is (square and) nonsingular here.
Let
V
be any orthogonal matrix so that
V
>
(
A
>
A
)
V
is diagonal and positive deF
nite (positive diagonal entries). Let
D
be a p.d. diagonal square root of this matrix, so
V
>
(
A
>
A
)
V
=
D
2
.Le
t
U
=
AV D
−
1
. That was quick, but we must check that
A
=
>
and
U
>
U
=
I
. And these are easy:
>
=(
AV D
−
1
)(
DV
>
)=
AV V
>
=
A,
U
>
U
D
−
1
V
>
A
>
)(
AV D
−
1
D
−
1
(
V
>
(
A
>
A
)
V
)
D
=
D
−
1
D
2
D
−
1
=
I.
ut
The diagonal entries of
D
are the positive square roots of the eigenvalues of
A
>
A
and
are called the
singular values
of
A
. (Caution: this can be confusing because “singular” is
part of the term “singular values” and is not being used as in “singular matrix”.)
The SVD (singular value decomposition)
A
=
>
of an
m
×
n
matrix
A
displays
orthonormal bases for
R
n
and
R
m
,theco
lumns
e
1
,...,
e
n
of
V
and the columns
f
1
f
m
of
U
A
e
i
=
d
i
f
i
(and
A
e
i
=
0
if
i>m
)andfor
i
=1
,
2
min
{
}
,where
d
i
is the
i
th diagonal entry of
D
.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview.
Sign up
to
access the rest of the document.
 Winter '08
 Aschbacher
 Math, Singular value decomposition, Orthogonal matrix, Linear least squares, svd, Durer

Click to edit the document details