Lecture_12 - Lecture Note 13: Nov 28 - Dec 1, 2006 Dr. Jeff...

Info iconThis preview shows pages 1–9. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Lecture Note 13: Nov 28 - Dec 1, 2006 Dr. Jeff Chak-Fu WONG Department of Mathematics Chinese University of Hong Kong jwong@math.cuhk.edu.hk MAT 2310 Linear Algebra and Its Applications Fall, 2006 Produced by Jeff Chak-Fu WONG 1 S INGULAR V ALUE D ECOMPOSITION SINGULAR VALUE DECOMPOSITION 2 The diagonalization theorem Lecture Note 11-1 and 11-2 play a part in many interesting applications. Unfortunately, as we know, not all matrices can be factored as A = PDP- 1 with D diagonal. However, a factorization A = QDP- 1 is possible for any m × n matrix A ! A special factorization of this type, the singular value decomposition , is one of the most useful matrix factorizations in applied linear algebra. SINGULAR VALUE DECOMPOSITION 3 Our aim is to factor any m × n matrix A in the form A = U Σ V T where U is m × m , V is n × n , and they are both orthogonal. Also Σ is an m × n matrix with a diagonal upper left block of positive entries of decreasing magnitude and the remaining entries 0. Thus Σ = D . . . ... ... ... . . . , where D = σ 1 ... . . . . . . . . . ... σ r and σ 1 ≥ σ 2 ≥ ... ≥ σ r > , r ≤ m,n A m × n = U m × m Σ m × n V T n × n SINGULAR VALUE DECOMPOSITION 4 The singular value decomposition is based on the following property of the ordinary diagonalization that can be imitated for rectangular matrices: The absolute values of the eigenvalue of a symmetric matrix A measure the amounts that A stretches or shrinks certain vectors (the eigenvectors) . If A x = λ x and k x k = 1 , then k A x k = k λ x k = | λ |k x k = | λ | (1) If λ i is the eigenvalue with the greatest magnitude, then a corresponding unit eigenvector v 1 identifies a direction in which the stretching effect of A is greatest. That is, the length of A x is maximized when x = v 1 and k A v 1 k = | λ 1 | , by (1). This description of v 1 and | λ 1 | has an analogue for rectangular matrices that will lead to the singular value decomposition. SINGULAR VALUE DECOMPOSITION 5 T HE S INGULAR V ALUES OF AN m × n M ATRIX THE SINGULAR VALUES OF AN m × n MATRIX 6 Let A be an m × n matrix. Then A T A is symmetric and can be orthogonally diagonalized . Let { v 1 ,..., v n } be an orthonormal basis for R n consisting of eigenvectors of A T A . Here, V is simply as V = h v 1 v 2 ··· v n i n × n Let λ 1 ,...,λ n be the associated eigenvalues of A T A . Then for 1 ≤ i ≤ n , ≤ k A v i k 2 = ( A v i ) T A v i = v T i A T A v i = v T i ( λ i v i ) Since v i is a eigenvector of A T A = λ i k v i k 2 = λ i Since v i is a unit vector (2) So the eigenvalue of A T A are all nonnegative (so A T A is positive semi definite). By renumbering, if necessary, we may assume that the eigenvalues are arranged so that λ 1 ≥ λ 2 ≥ ... ≥ λ n ≥ THE SINGULAR VALUES OF AN m × n MATRIX 7 The singular values of A are the square roots of the eigenvalues of A T A , denoted by σ 1...
View Full Document

Page1 / 35

Lecture_12 - Lecture Note 13: Nov 28 - Dec 1, 2006 Dr. Jeff...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online