Lecture 2 - 2 Singular Value Decomposition The singular...

Info icon This preview shows pages 1–3. Sign up to view the full content.

2 Singular Value Decomposition The singular value decomposition (SVD) allows us to transform a matrix A C m × n to diagonal form using unitary matrices, i.e., A = ˆ U ˆ Σ V * . (4) Here ˆ U C m × n has orthonormal columns, ˆ Σ C n × n is diagonal, and V C n × n is unitary. This is the practical version of the SVD also known as the reduced SVD . We will discuss the full SVD later. It is of the form A = U Σ V * with unitary matrices U and V and Σ C m × n . Before we worry about how to find the matrix factors of A we give a geometric interpretation. First note that since V is unitary (i.e., V * = V - 1 ) we have the equiva- lence A = ˆ U ˆ Σ V * ⇐⇒ AV = ˆ U ˆ Σ . Considering each column of V separately the latter is the same as A v j = σ j u j , j = 1 , . . . , n. (5) Thus, the unit vectors of an orthogonal coordinate system { v 1 , . . . , v n } are mapped under A onto a new “scaled” orthogonal coordinate system { σ 1 u 1 , . . . , σ n u n } . In other words, the unit sphere with respect to the matrix 2-norm (which is a perfectly round sphere in the v -system) is transformed to an ellipsoid with semi-axes σ j u j (see Fig- ure 2). We will see below that, depending on the rank of A , some of the σ j may be zero. Therefore, yet another geometrical interpretation of the SVD is: Any m × n matrix A maps the 2-norm unit sphere in R n to an ellipsoid in R r ( r min( m, n )). 1 v 2 v –1 –0.5 0.5 1 –1 –0.5 0.5 1 2 v 2 σ 1 u 1 σ –2 –1 0 1 2 –2 –1 1 2 Figure 2: Geometrical interpretation of singular value decomposition. In (5) we refer to the σ j as singular values of A (the diagonal entries of ˆ Σ). They are usually ordered such that σ 1 σ 2 . . . σ n . The orthonormal vectors u j (the columns of ˆ U ) are called the left singular vectors of A , and the orthonormal vectors v j (the columns of V ) are called the right singular vectors of A ). Remark For most practical purposes it suffices to compute the reduced SVD (4). We will give examples of its use, and explain how to compute it later. 23
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

QR Compression ratio 0.2000, 25 columns used 50 100 150 200 250 300 20 40 60 80 100 120 140 160 180 200 Compression ratio 0.2000, relative error 0.0320, 25 singular values used 50 100 150 200 250 300 20 40 60 80 100 120 140 160 180 200 Figure 3: Image compressed using QR factorization (left) and SVD (right). Besides applications to inconsistent and underdetermined linear systems and least squares problems, the SVD has important applications in image and data compression (see our discussion of low-rank approximation below). Figure 3 shows the difference between using the SVD and the QR factorization (to be introduced later) for com- pression of the same image. In both cases the same amount (20%) of information was retained. Clearly, the SVD does a much better job in picking out what information is “important”. We will also see below that a number of theoretical facts about the matrix A can be obtained via the SVD.
Image of page 2
Image of page 3
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern