The key piece of mathematical technology we need is the matrix inversion lemma

# The key piece of mathematical technology we need is

• Notes
• 7

This preview shows page 2 - 5 out of 7 pages.

The key piece of mathematical technology we need is the matrix inversion lemma . The Matrix Inversion Lemma The matrix inversion lemma shows us how the solution to a system of equations can be efficiently updated. Here we state a slightly simplified version of the result: If W is an N × N invertible matrix and X is an R × N matrix, then the following identity holds: ( W + X T X ) - 1 = W - 1 - W - 1 X T ( I + XW - 1 X T ) - 1 XW - 1 This is a special case of the Sherman-Morrison-Woodbury identity, and is straightforward to prove (see the Technical Details at the end of these notes). The point is that if W - 1 has already been calculated, then finding a solution to ( W + X T X ) w = v costs O ( N 2 R ) + O ( NR 2 ) + O ( R 3 ) instead of O ( N 3 ). If R is very small compared to N , this can be a significant savings. 48 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 3:33, November 20, 2019

Subscribe to view the full document.

Updating least-squares solutions We can apply this fact to efficiently update the solution to least- squares problems as new measurements become available. Suppose we have observed y 0 = A 0 x * + e 0 and have formed the least-squares estimate b x 0 = ( A T 0 A 0 ) - 1 A T 0 y 0 . Now we observe y 1 = A 1 x * + e 1 , where A 1 is an M 1 × N matrix with M 1 N . Given y 0 and y 1 , the full least-squares estimate is formed from the system of equations y 0 y 1 = A 0 A 1 x * + e 0 e 1 , resulting in b x 1 = A T 0 A 0 + A T 1 A 1 - 1 ( A T 0 y 0 + A T 1 y 1 ) . Now let P k be the inverse of the aggregated system we would like to solve at each step: P 0 = ( A T 0 A 0 ) - 1 P 1 = ( A T 0 A 0 + A T 1 A 1 ) - 1 . Then using the matrix inversion lemma with W = A T 0 A 0 = P - 1 0 and X = A 1 gives us the update P 1 = P 0 - P 0 A T 1 ( I + A 1 P 0 A T 1 ) - 1 A 1 P 0 . 49 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 3:33, November 20, 2019
When the number of new measurements (rows in A 1 ) is small, then the system of equations I + A 1 P 0 A T 1 can be much easier to handle than A T 0 A 0 + A T 1 A 1 = P - 1 1 . For example, suppose we see just one new measurement, so the matrix A 1 has just one row: A 1 = a T 1

Subscribe to view the full document.

• Fall '08
• Staff

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern