The key piece of mathematical technology we need is the matrix inversion lemma

The key piece of mathematical technology we need is

This preview shows page 2 - 5 out of 7 pages.

The key piece of mathematical technology we need is the matrix inversion lemma . The Matrix Inversion Lemma The matrix inversion lemma shows us how the solution to a system of equations can be efficiently updated. Here we state a slightly simplified version of the result: If W is an N × N invertible matrix and X is an R × N matrix, then the following identity holds: ( W + X T X ) - 1 = W - 1 - W - 1 X T ( I + XW - 1 X T ) - 1 XW - 1 This is a special case of the Sherman-Morrison-Woodbury identity, and is straightforward to prove (see the Technical Details at the end of these notes). The point is that if W - 1 has already been calculated, then finding a solution to ( W + X T X ) w = v costs O ( N 2 R ) + O ( NR 2 ) + O ( R 3 ) instead of O ( N 3 ). If R is very small compared to N , this can be a significant savings. 48 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 3:33, November 20, 2019
Image of page 2

Subscribe to view the full document.

Updating least-squares solutions We can apply this fact to efficiently update the solution to least- squares problems as new measurements become available. Suppose we have observed y 0 = A 0 x * + e 0 and have formed the least-squares estimate b x 0 = ( A T 0 A 0 ) - 1 A T 0 y 0 . Now we observe y 1 = A 1 x * + e 1 , where A 1 is an M 1 × N matrix with M 1 N . Given y 0 and y 1 , the full least-squares estimate is formed from the system of equations y 0 y 1 = A 0 A 1 x * + e 0 e 1 , resulting in b x 1 = A T 0 A 0 + A T 1 A 1 - 1 ( A T 0 y 0 + A T 1 y 1 ) . Now let P k be the inverse of the aggregated system we would like to solve at each step: P 0 = ( A T 0 A 0 ) - 1 P 1 = ( A T 0 A 0 + A T 1 A 1 ) - 1 . Then using the matrix inversion lemma with W = A T 0 A 0 = P - 1 0 and X = A 1 gives us the update P 1 = P 0 - P 0 A T 1 ( I + A 1 P 0 A T 1 ) - 1 A 1 P 0 . 49 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 3:33, November 20, 2019
Image of page 3
When the number of new measurements (rows in A 1 ) is small, then the system of equations I + A 1 P 0 A T 1 can be much easier to handle than A T 0 A 0 + A T 1 A 1 = P - 1 1 . For example, suppose we see just one new measurement, so the matrix A 1 has just one row: A 1 = a T 1
Image of page 4

Subscribe to view the full document.

Image of page 5
  • Fall '08
  • Staff

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes