When A is full rank R min M N then we can calculate the pseudo inverse without

# When a is full rank r min m n then we can calculate

• Notes
• 14

This preview shows page 8 - 12 out of 14 pages.

When A is full rank ( R = min( M, N )), then we can calculate the pseudo-inverse without using the SVD. There are three cases: When A is square and invertible ( R = M = N ), then A = A - 1 . This is easy to check, as here A = U Σ V T where both U , V are N × N, 42 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 9:10, October 30, 2019 Subscribe to view the full document.

and since in this case V V T = V T V = I and UU T = U T U = I , A A = V Σ - 1 U T U Σ V T = V Σ - 1 Σ V T = V V T = I . Similarly, AA = I , and so A is both a left and right inverse of A , and thus A = A - 1 . When A more rows than columns and has full column rank ( R = N M ), then A T A is invertible, and A = ( A T A ) - 1 A T . (7) This type of A is “tall and skinny” A , and its columns are linearly independent. To verify equation ( 7 ), recall that A T A = V Σ U T U Σ V T = V Σ 2 V T , and so ( A T A ) - 1 A T = V Σ - 2 V T V Σ U T = V Σ - 1 U T , which is exactly the content of ( 6 ). 43 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 9:10, October 30, 2019 When A has more columns than rows and has full row rank ( R = M N ), then AA T is invertible, and A = A T ( AA T ) - 1 . (8) This occurs when A is “short and fat” A , and its rows are linearly independent. To verify equation ( 8 ), recall that AA T = U Σ V T V Σ U T = U Σ 2 U T , and so A T ( AA T ) - 1 = V Σ U T U Σ - 2 U T = V Σ - 1 U T , which again is exactly ( 6 ). A is as close to an inverse of A as possible As discussed in above, when A is square and invertible, A is exactly the inverse of A . When A is not square, we can ask if there is a better right or left inverse. We will argue that there is not. Left inverse Given y = Ax , we would like A y = A Ax = x for any x . That is, we would like A to be a left inverse of A : A A = I . Of course, this is not always possible, especially when A has more columns than rows, M < N . But we can ask if any other matrix H comes closer to being a left inverse 44 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 9:10, October 30, 2019 Subscribe to view the full document.

than A . To find the “best” left-inverse, we look for the matrix which minimizes min H R N × M k HA - I k 2 F . (9) Here, k · k F is the Frobenius norm , defined for an N × M matrix Q as the sum of the squares of the entries: 3 k Q k 2 F = M X n =1 N X n =1 | Q [ m, n ] | 2 With ( 9 ), we are finding H such that HA is as close to the identity as possible in the least-squares sense. The pseudo-inverse A minimizes ( 9 ). To see this, recognize (see the exercise below) that the solution ˆ H to ( 9 ) must obey AA T ˆ H T = A . (10) We can see that this is indeed true for ˆ H = A : AA T A T = U Σ V T V Σ U T U Σ - 1 V T = U Σ V T = A . So there is no N × M matrix that is closer to being a left inverse than A . 3 It is also true that k Q k 2 F is the sum of the squares of the singular values of Q : k Q k 2 F = λ 2 1 + · · · + λ 2 p . This is something that you will prove on the next homework.  • Fall '08
• Staff

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern