A simple way to avoid this is to simply truncate the sum 4 leaving out the

# A simple way to avoid this is to simply truncate the

• Notes
• 18

This preview shows page 6 - 9 out of 18 pages.

A simple way to avoid this is to simply truncate the sum ( 4 ), leaving out the terms where σ r is too small (1 r is too big). Exactly how many terms to keep depends a great deal on the application, as there are competing interests. On the one hand, we want to ensure that each of the σ r we include has an inverse of reasonable size, on the other, we want the reconstruction to be accurate (i.e. not to deviate from the noiseless least-squares solution by too much). We form an approximation A 0 to A by taking A 0 = R 0 X r =1 σ r u r v T r , for some R 0 < R . Again, our final answer will depend on which R 0 we use, and choosing R 0 is often times something of an art. It is clear that the approximation A 0 has rank R 0 . Note that the pseudo-inverse of A 0 is also a truncated sum A 0† = R 0 X r =1 1 σ r v r u T r . Given noisy data y as in ( 3 ), we reconstruct x by applying the truncated pseudo-inverse to y : ˆ x trunc = A 0† y = R 0 X r =1 1 σ r h y , u r i v r . 54 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019

Subscribe to view the full document.

How good is this reconstruction? To answer this question, we will compare it to the noiseless least-squares reconstruction x pinv = A y clean , where y clean = Ax are “noiseless” measurements of x . The differ- ence between these two is the reconstruction error (relative to x pinv ) as ˆ x trunc - x pinv = A 0† y - A Ax = A 0† Ax + A 0† e - A Ax = ( A 0† - A ) Ax + A 0† e . Proceeding further, we can write the matrix A 0† - A as A 0† - A = R X r = R 0 +1 - 1 σ r v r u T r , and so the first term in the reconstruction error can be written as ( A 0† - A ) Ax = R X r = R 0 +1 - 1 σ r h Ax , u r i v r = R X r = R 0 +1 - 1 σ r * R X j =1 σ j h x , v j i u j , u r + v k = R X r = R 0 +1 - 1 σ r R X j =1 σ j h x , v j ih u j , u r i v r = R X r = R 0 +1 -h x , v r i v r (since h u r , u j i = 0 unless j = r ) . The second term in the reconstruction error can also be expanded against the v r : A 0† e = R 0 X r =1 1 σ r h e , u r i v r . 55 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019
Combining these expressions, the reconstruction error can be written ˆ x trunc - x pinv = R 0 X r =1 1 σ r h e , u r i v r | {z } + R X k = R 0 +1 -h x , v k i v k | {z } = Noise error + Approximation error . Since the v r are mutually orthogonal, and the two sums run over disjoint index sets, the noise error and the approximation error will be orthogonal. Also k ˆ x trunc - x pinv k 2 2 = k Noise error k 2 2 + k Approximation error k 2 2 = R 0 X r =1 1 σ 2 r |h e , u r i| 2 + R X r = R 0 +1 |h x , v r i| 2 . The reconstruction error, then, is signal dependent and will depend on how much of the vector x is concentrated in the subspace spanned by v R 0 +1 , . . . , v R . We will lose everything in this subspace; if it contains a significant part of x , then there is not much least-squares can do for you. The worst-case noise error occurs when e is aligned with u R 0 : k Noise error k 2 2 = R 0 X r =1 1 σ 2 r |h e , u r i| 2 1 σ 2 R 0 · k e k 2 2 .

Subscribe to view the full document.

• Fall '08
• Staff

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern