When
A
is full rank (
R
= min(
M, N
)), then we can calculate the
pseudoinverse without using the SVD. There are three cases:
•
When
A
is square and invertible (
R
=
M
=
N
), then
A
†
=
A

1
.
This is easy to check, as here
A
=
U
Σ
V
T
where both
U
,
V
are
N
×
N,
42
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 9:10, October 30, 2019
Subscribe to view the full document.
and since in this case
V V
T
=
V
T
V
=
I
and
UU
T
=
U
T
U
=
I
,
A
†
A
=
V
Σ

1
U
T
U
Σ
V
T
=
V
Σ

1
Σ
V
T
=
V V
T
=
I
.
Similarly,
AA
†
=
I
, and so
A
†
is both a left and right inverse
of
A
, and thus
A
†
=
A

1
.
•
When
A
more rows than columns and has full column rank
(
R
=
N
≤
M
), then
A
T
A
is invertible, and
A
†
= (
A
T
A
)

1
A
T
.
(7)
This type of
A
is “tall and skinny”
A
,
and its columns are linearly independent. To verify equation
(
7
), recall that
A
T
A
=
V
Σ
U
T
U
Σ
V
T
=
V
Σ
2
V
T
,
and so
(
A
T
A
)

1
A
T
=
V
Σ

2
V
T
V
Σ
U
T
=
V
Σ

1
U
T
,
which is exactly the content of (
6
).
43
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 9:10, October 30, 2019
•
When
A
has more columns than rows and has full row rank
(
R
=
M
≤
N
), then
AA
T
is invertible, and
A
†
=
A
T
(
AA
T
)

1
.
(8)
This occurs when
A
is “short and fat”
A
,
and its rows are linearly independent. To verify equation (
8
),
recall that
AA
T
=
U
Σ
V
T
V
Σ
U
T
=
U
Σ
2
U
T
,
and so
A
T
(
AA
T
)

1
=
V
Σ
U
T
U
Σ

2
U
T
=
V
Σ

1
U
T
,
which again is exactly (
6
).
A
†
is as close to an inverse of
A
as possible
As discussed in above, when
A
is square and invertible,
A
†
is exactly
the inverse of
A
. When
A
is not square, we can ask if there is a
better right or left inverse. We will argue that there is not.
Left inverse
Given
y
=
Ax
, we would like
A
†
y
=
A
†
Ax
=
x
for any
x
. That is, we would like
A
†
to be a
left inverse
of
A
:
A
†
A
=
I
. Of course, this is not always possible, especially
when
A
has more columns than rows,
M < N
. But we can
ask if any other matrix
H
comes closer to being a left inverse
44
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 9:10, October 30, 2019
Subscribe to view the full document.
than
A
†
. To find the “best” leftinverse, we look for the matrix
which minimizes
min
H
∈
R
N
×
M
k
HA

I
k
2
F
.
(9)
Here,
k · k
F
is the
Frobenius norm
, defined for an
N
×
M
matrix
Q
as the sum of the squares of the entries:
3
k
Q
k
2
F
=
M
X
n
=1
N
X
n
=1

Q
[
m, n
]

2
With (
9
), we are finding
H
such that
HA
is as close to the
identity as possible in the leastsquares sense.
The pseudoinverse
A
†
minimizes (
9
).
To see this, recognize
(see the exercise below) that the solution
ˆ
H
to (
9
) must obey
AA
T
ˆ
H
T
=
A
.
(10)
We can see that this is indeed true for
ˆ
H
=
A
†
:
AA
T
A
†
T
=
U
Σ
V
T
V
Σ
U
T
U
Σ

1
V
T
=
U
Σ
V
T
=
A
.
So there is no
N
×
M
matrix that is closer to being a left
inverse than
A
†
.
3
It is also true that
k
Q
k
2
F
is the sum of the squares of the singular values of
Q
:
k
Q
k
2
F
=
λ
2
1
+
· · ·
+
λ
2
p
. This is something that you will prove on the
next homework.
 Fall '08
 Staff