A simple way to avoid this is to simply truncate the sum (
4
), leaving
out the terms where
σ
r
is too small (1
/σ
r
is too big). Exactly how
many terms to keep depends a great deal on the application, as there
are competing interests. On the one hand, we want to ensure that
each of the
σ
r
we include has an inverse of reasonable size, on the
other, we want the reconstruction to be accurate (i.e. not to deviate
from the noiseless leastsquares solution by too much).
We form an approximation
A
0
to
A
by taking
A
0
=
R
0
X
r
=1
σ
r
u
r
v
T
r
,
for some
R
0
< R
. Again, our final answer will depend on which
R
0
we use, and choosing
R
0
is often times something of an art. It is clear
that the approximation
A
0
has rank
R
0
. Note that the pseudoinverse
of
A
0
is also a truncated sum
A
0†
=
R
0
X
r
=1
1
σ
r
v
r
u
T
r
.
Given noisy data
y
as in (
3
), we reconstruct
x
by applying the
truncated pseudoinverse to
y
:
ˆ
x
trunc
=
A
0†
y
=
R
0
X
r
=1
1
σ
r
h
y
,
u
r
i
v
r
.
54
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019
Subscribe to view the full document.
How good is this reconstruction? To answer this question, we will
compare it to the noiseless leastsquares reconstruction
x
pinv
=
A
†
y
clean
,
where
y
clean
=
Ax
are “noiseless” measurements of
x
. The differ
ence between these two is the reconstruction error (relative to
x
pinv
)
as
ˆ
x
trunc

x
pinv
=
A
0†
y

A
†
Ax
=
A
0†
Ax
+
A
0†
e

A
†
Ax
= (
A
0†

A
†
)
Ax
+
A
0†
e
.
Proceeding further, we can write the matrix
A
0†

A
†
as
A
0†

A
†
=
R
X
r
=
R
0
+1

1
σ
r
v
r
u
T
r
,
and so the first term in the reconstruction error can be written as
(
A
0†

A
†
)
Ax
=
R
X
r
=
R
0
+1

1
σ
r
h
Ax
,
u
r
i
v
r
=
R
X
r
=
R
0
+1

1
σ
r
*
R
X
j
=1
σ
j
h
x
,
v
j
i
u
j
,
u
r
+
v
k
=
R
X
r
=
R
0
+1

1
σ
r
R
X
j
=1
σ
j
h
x
,
v
j
ih
u
j
,
u
r
i
v
r
=
R
X
r
=
R
0
+1
h
x
,
v
r
i
v
r
(since
h
u
r
,
u
j
i
= 0 unless
j
=
r
)
.
The second term in the reconstruction error can also be expanded
against the
v
r
:
A
0†
e
=
R
0
X
r
=1
1
σ
r
h
e
,
u
r
i
v
r
.
55
Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019
Combining these expressions, the reconstruction error can be written
ˆ
x
trunc

x
pinv
=
R
0
X
r
=1
1
σ
r
h
e
,
u
r
i
v
r

{z
}
+
R
X
k
=
R
0
+1
h
x
,
v
k
i
v
k

{z
}
=
Noise error
+
Approximation error
.
Since the
v
r
are mutually orthogonal, and the two sums run over
disjoint index sets, the noise error and the approximation error will
be orthogonal. Also
k
ˆ
x
trunc

x
pinv
k
2
2
=
k
Noise error
k
2
2
+
k
Approximation error
k
2
2
=
R
0
X
r
=1
1
σ
2
r
h
e
,
u
r
i
2
+
R
X
r
=
R
0
+1
h
x
,
v
r
i
2
.
The reconstruction error, then, is signal dependent and will depend
on how much of the vector
x
is concentrated in the subspace spanned
by
v
R
0
+1
, . . . ,
v
R
.
We will lose everything in this subspace; if it
contains a significant part of
x
, then there is not much leastsquares
can do for you.
The worstcase noise error occurs when
e
is aligned with
u
R
0
:
k
Noise error
k
2
2
=
R
0
X
r
=1
1
σ
2
r
h
e
,
u
r
i
2
≤
1
σ
2
R
0
· k
e
k
2
2
.
Subscribe to view the full document.
 Fall '08
 Staff