Since U is an M R matrix it is possible when R M that the reconstruction error

# Since u is an m r matrix it is possible when r m that

• Notes
• 18

This preview shows page 3 - 7 out of 18 pages.

Since U is an M × R matrix, it is possible when R < M that the reconstruction error is zero. This happens when e is orthogonal to every column of U , i.e. U T e = 0 . Putting this together with the work above means 0 1 σ 2 1 k U T e k 2 2 ≤ k ˆ x ls - x pinv k 2 2 1 σ 2 R k U T e k 2 2 1 σ 2 R k e k 2 2 . Notice that if σ R is small, the worst case reconstruction error can be very bad . 51 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019 Subscribe to view the full document.

We can also relate the “average case” error to the singular values. Say that e is additive Gaussian white noise, that is each entry e [ m ] is a random variable independent of all the other entries, and distributed e [ m ] Normal(0 , ν 2 ) . Then, as we have argued before, the average measurement error is E[ k e k 2 2 ] = 2 , and the average reconstruction error 1 is E h k A e k 2 2 i = ν 2 · trace( A T A ) = ν 2 · 1 σ 2 1 + 1 σ 2 2 + · · · + 1 σ 2 R = 1 M 1 σ 2 1 + 1 σ 2 2 + · · · + 1 σ 2 R · E[ k e k 2 2 ] . Again, if σ R is tiny, 1 2 R will dominate the sum above, and the average reconstruction error will be quite large. Exercise: Let D be a diagonal R × R matrix whose diagonal elements are positive. Show that the maximizer ˆ β to maximize β R R k k 2 2 subject to k β k 2 = 1 has a 1 in the entry corresponding to the largest diagonal element of D , and is 0 elsewhere. 1 We are using the fact that if e is vector of iid Gaussian random vari- ables, e Normal( 0 , ν 2 I ), then for any matrix M , E[ k Me k 2 2 ] = ν 2 trace( M T M ). We leave this as an exercise (or maybe a homework!) 52 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019 Stable Reconstruction with the Truncated SVD We have seen that if A has very small singular values and we apply the pseudo-inverse in the presence of noise, the results can be disas- trous. But it doesn’t have to be this way. There are several ways to stabilize the pseudo-inverse. We start be discussing the simplest one, where we simply “cut out” the part of the reconstruction which is causing the problems. As before, we are given noisy indirect observations of a vector x through a M × N matrix A : y = Ax + e . (3) The matrix A has SVD A = U Σ V T , and pseudo-inverse A = V Σ - 1 U T . We can rewrite A as a sum of rank-1 matrices: A = R X r =1 σ r u r v T r , where R is the rank of A , the σ r are the singular values, and u r R M and v r R N are columns of U and V , respectively. Similarly, we can write the pseudo-inverse as A = R X r =1 1 σ r v r u T r . Given y as above, we can write the least-squares estimate of x from the noisy measurements as ˆ x ls = A y = R X r =1 1 σ r h y , u r i v r . (4) 53 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 21:13, November 3, 2019 Subscribe to view the full document.

As we can see (and have seen before) if any one of the σ r are very small, the least-squares reconstruction can be a disaster.  • Fall '08
• Staff

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern