45 Georgia Tech ECE 6250 Fall 2019 Notes by J Romberg and M Davenport Last

# 45 georgia tech ece 6250 fall 2019 notes by j romberg

• Notes
• 14

This preview shows page 11 - 14 out of 14 pages.

45 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 9:10, October 30, 2019

Subscribe to view the full document.

Right inverse If we re-apply A to our solution ˆ x = A y , we would like it to be as close as possible to our observations y . That is, we would like AA to be as close to the identity as possible. Again, achieving this goal exactly is not always possible, espe- cially if A has more rows that columns. But we can attempt to find the “best” right inverse, in the least-squares sense, by solving minimize H R N × M k AH - I k 2 F . (11) The solution ˆ H to ( 11 ) (see the exercise below) must obey A T A ˆ H = A T . (12) Again, we show that A satisfies ( 12 ), and hence is a minimizer to ( 11 ): A T AA = V Σ 2 V T V Σ - 1 U T = V Σ U T = A T . Moral: A = V Σ - 1 U T is as close (in the least-squares sense) to an inverse of A as you could possibly have . Exercise: Show that the minimizer ˆ H to ( 9 ) must obey ( 10 ). Do this by using the fact that the derivative of the functional k HA - I k 2 F with respect to an entry H [ k, ‘ ] in H must obey k HA - I k 2 F ∂H [ k, ‘ ] = 0 , for all 1 k N, 1 M, to be a solution to ( 9 ). Do the same for ( 11 ) and ( 12 ). 46 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 9:10, October 30, 2019
Technical Details: Existence of the SVD In this section we will prove that any M × N matrix A with rank( A ) = R can be written as A = U Σ V T where U , Σ , V have the five properties listed at the beginning of the last section. Since A T A is symmetric positive semi-definite, we can write: A T A = N X n =1 λ n v n v T n , where the v n are orthonormal and the λ n are real and non-negative. Since rank( A ) = R , we also have rank( A T A ) = R , and so λ 1 , . . . , λ R are all strictly positive above, and λ R +1 = · · · = λ N = 0. Set u m = 1 λ m Av m , for m = 1 , . . . , R, U = u 1 · · · u R . Notice that these u m are orthonormal, as h u m , u i = 1 λ m λ v T A T Av m = s λ m λ v T v m = ( 1 , m = ‘, 0 , m 6 = ‘. These u m also happen to be eigenvectors of AA T , as AA T u m = 1 λ m AA T Av m = p λ m Av m = λ m u m . Now let u R +1 , . . . , u M be an orthobasis for the null space of U T concatenating these two sets into u 1 , . . . , u M forms an orthobasis for all of R M . 47 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 9:10, October 30, 2019

Subscribe to view the full document.

Let V = v 1 v 2 · · · v R . In addition, let V 0 = v R +1 v R +2 · · · v N , V full = V V 0 and U 0 = u R +1 u R +2 · · · u M , U full = U U 0 . It should be clear that V full is an N × N orthonormal matrix and U full is a M × M orthonormal matrix. Consider the M × N matrix U T full AV full — the entry in the m th rows and n th column of this matrix is ( U T full AV full )[ m, n ] = u T m Av n = ( λ n u T m u n n = 1 , . . . , R 0 , n = R + 1 , . . . , N. = ( λ n , m = n = 1 , . . . , R 0 , otherwise . Thus U T full AV full = Σ full where Σ full [ m, n ] = ( λ n , m = n = 1 , . . . , R 0 , otherwise .
• Fall '08
• Staff

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern