We want to find Δ A Δ y x such that y Δ y A Δ A x for Δ y Δ A of minimal size

We want to find δ a δ y x such that y δ y a δ a x

This preview shows page 4 - 8 out of 18 pages.

We want to find Δ A , Δ y , x such that ( y + Δ y ) = ( A + Δ A ) x , for Δ y , Δ A of minimal size. Rewrite this as ( A + Δ A ) x - ( y + Δ y ) = 0 A + Δ A y + Δ y x - 1 = 0 ( C + Δ ) x - 1 = 0 where C = A y , Δ = Δ A Δ y . Note that both C and Δ are M × ( N + 1) matrices. The result of the progression of equations above says that we are looking for a Δ (of minimal size) such that there is a vector x - 1 in the nullspace of C + Δ . Since v Null( C + Δ ) α v Null( C + Δ ) for all α R , and x in arbitrary, we are really just asking that C + Δ has a nullspace; as long as there is at least one vector in the nullspace 70 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 23:01, November 5, 2019
Image of page 4

Subscribe to view the full document.

whose last entry is nonzero, we can find a vector of the required form just by normalizing. In short, this means that our task is to find Δ such that the M × ( N + 1) matrix C + Δ is rank deficient , that is rank( C + Δ ) < N + 1. Put another way, we want to solve the optimization program minimize Δ k Δ k 2 F subject to rank( C + Δ ) = N. Making the substitution X = C + Δ , this is equivalent to solving minimize X k C - X k 2 F subject to rank( X ) = N, and then taking c Δ = c X - C . This is a low-rank approximation problem. 1 We now consider how to solve problems of this form (which arise in many other important contexts). The SVD and Matrix Approximation Let A be an M × N matrix with rank R . We are often interested in determining the closest matrix to A that has rank r 2 ? More precisely, we would like to solve minimize X k A - X k 2 F subject to rank( X ) = r. (1) The functional above is standard least-squares, but the constraint set (the set of rank- r matrices) is a complicated entity. Neverthe- less, as with many things in this class, the SVD reveals the solution immediately. 1 Or at least a “lower rank” approximation problem. 2 We will assume that r < R , as for r = R the answer is easy, and for R < r min( M, N ) the question is not well-posed. 71 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 23:01, November 5, 2019
Image of page 5
Low-rank approximation. Let A be a matrix with SVD A = U Σ V T = R X p =1 σ p u p v T p . Then ( 1 ) is solved simply by truncating the SVD: c X = r X p =1 σ p u p v T p = U r Σ r V T r , where U r contains the first r columns of U , V r contains the first r columns of V , and Σ r is the first r columns and r rows of Σ . The framed result above, known as the Eckart-Young theorem, is an immediate consequence of the following lemma. Subspace Approximation Lemma. For fixed A with SVD A = U Σ V T , the optimization program minimize Q : M × r Θ : r × N k A - Q Θ k 2 F subject to Q T Q = I , (2) has solution b Q = U r , b Θ = U T r A , where U r = u 1 u 2 · · · u r contains the first r columns of U . We prove this lemma in the technical details section at the end of the notes. To see how it implies the Eckart-Young theorem, we can 72 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 23:01, November 5, 2019
Image of page 6

Subscribe to view the full document.

interpret the search over M × r matrices Q with orthonormal columns as a search over all possible column spaces of dimension r . Then the search over Θ
Image of page 7
Image of page 8
  • Fall '08
  • Staff

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask You can ask ( soon) You can ask (will expire )
Answers in as fast as 15 minutes