We want to find Δ A Δ y x such that y Δ y A Δ A x for Δ y Δ A of minimal size

# We want to find δ a δ y x such that y δ y a δ a x

• Notes
• 18

This preview shows page 4 - 8 out of 18 pages.

We want to find Δ A , Δ y , x such that ( y + Δ y ) = ( A + Δ A ) x , for Δ y , Δ A of minimal size. Rewrite this as ( A + Δ A ) x - ( y + Δ y ) = 0 A + Δ A y + Δ y x - 1 = 0 ( C + Δ ) x - 1 = 0 where C = A y , Δ = Δ A Δ y . Note that both C and Δ are M × ( N + 1) matrices. The result of the progression of equations above says that we are looking for a Δ (of minimal size) such that there is a vector x - 1 in the nullspace of C + Δ . Since v Null( C + Δ ) α v Null( C + Δ ) for all α R , and x in arbitrary, we are really just asking that C + Δ has a nullspace; as long as there is at least one vector in the nullspace 70 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 23:01, November 5, 2019

Subscribe to view the full document.

whose last entry is nonzero, we can find a vector of the required form just by normalizing. In short, this means that our task is to find Δ such that the M × ( N + 1) matrix C + Δ is rank deficient , that is rank( C + Δ ) < N + 1. Put another way, we want to solve the optimization program minimize Δ k Δ k 2 F subject to rank( C + Δ ) = N. Making the substitution X = C + Δ , this is equivalent to solving minimize X k C - X k 2 F subject to rank( X ) = N, and then taking c Δ = c X - C . This is a low-rank approximation problem. 1 We now consider how to solve problems of this form (which arise in many other important contexts). The SVD and Matrix Approximation Let A be an M × N matrix with rank R . We are often interested in determining the closest matrix to A that has rank r 2 ? More precisely, we would like to solve minimize X k A - X k 2 F subject to rank( X ) = r. (1) The functional above is standard least-squares, but the constraint set (the set of rank- r matrices) is a complicated entity. Neverthe- less, as with many things in this class, the SVD reveals the solution immediately. 1 Or at least a “lower rank” approximation problem. 2 We will assume that r < R , as for r = R the answer is easy, and for R < r min( M, N ) the question is not well-posed. 71 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 23:01, November 5, 2019
Low-rank approximation. Let A be a matrix with SVD A = U Σ V T = R X p =1 σ p u p v T p . Then ( 1 ) is solved simply by truncating the SVD: c X = r X p =1 σ p u p v T p = U r Σ r V T r , where U r contains the first r columns of U , V r contains the first r columns of V , and Σ r is the first r columns and r rows of Σ . The framed result above, known as the Eckart-Young theorem, is an immediate consequence of the following lemma. Subspace Approximation Lemma. For fixed A with SVD A = U Σ V T , the optimization program minimize Q : M × r Θ : r × N k A - Q Θ k 2 F subject to Q T Q = I , (2) has solution b Q = U r , b Θ = U T r A , where U r = u 1 u 2 · · · u r contains the first r columns of U . We prove this lemma in the technical details section at the end of the notes. To see how it implies the Eckart-Young theorem, we can 72 Georgia Tech ECE 6250 Fall 2019; Notes by J. Romberg and M. Davenport. Last updated 23:01, November 5, 2019

Subscribe to view the full document.

interpret the search over M × r matrices Q with orthonormal columns as a search over all possible column spaces of dimension r . Then the search over Θ
• Fall '08
• Staff

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern