This preview shows pages 1–9. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Lecture Note 13: Nov 28  Dec 1, 2006 Dr. Jeff ChakFu WONG Department of Mathematics Chinese University of Hong Kong jwong@math.cuhk.edu.hk MAT 2310 Linear Algebra and Its Applications Fall, 2006 Produced by Jeff ChakFu WONG 1 S INGULAR V ALUE D ECOMPOSITION SINGULAR VALUE DECOMPOSITION 2 The diagonalization theorem Lecture Note 111 and 112 play a part in many interesting applications. Unfortunately, as we know, not all matrices can be factored as A = PDP 1 with D diagonal. However, a factorization A = QDP 1 is possible for any m × n matrix A ! A special factorization of this type, the singular value decomposition , is one of the most useful matrix factorizations in applied linear algebra. SINGULAR VALUE DECOMPOSITION 3 Our aim is to factor any m × n matrix A in the form A = U Σ V T where U is m × m , V is n × n , and they are both orthogonal. Also Σ is an m × n matrix with a diagonal upper left block of positive entries of decreasing magnitude and the remaining entries 0. Thus Σ = D . . . ... ... ... . . . , where D = σ 1 ... . . . . . . . . . ... σ r and σ 1 ≥ σ 2 ≥ ... ≥ σ r > , r ≤ m,n A m × n = U m × m Σ m × n V T n × n SINGULAR VALUE DECOMPOSITION 4 The singular value decomposition is based on the following property of the ordinary diagonalization that can be imitated for rectangular matrices: The absolute values of the eigenvalue of a symmetric matrix A measure the amounts that A stretches or shrinks certain vectors (the eigenvectors) . If A x = λ x and k x k = 1 , then k A x k = k λ x k =  λ k x k =  λ  (1) If λ i is the eigenvalue with the greatest magnitude, then a corresponding unit eigenvector v 1 identifies a direction in which the stretching effect of A is greatest. That is, the length of A x is maximized when x = v 1 and k A v 1 k =  λ 1  , by (1). This description of v 1 and  λ 1  has an analogue for rectangular matrices that will lead to the singular value decomposition. SINGULAR VALUE DECOMPOSITION 5 T HE S INGULAR V ALUES OF AN m × n M ATRIX THE SINGULAR VALUES OF AN m × n MATRIX 6 Let A be an m × n matrix. Then A T A is symmetric and can be orthogonally diagonalized . Let { v 1 ,..., v n } be an orthonormal basis for R n consisting of eigenvectors of A T A . Here, V is simply as V = h v 1 v 2 ··· v n i n × n Let λ 1 ,...,λ n be the associated eigenvalues of A T A . Then for 1 ≤ i ≤ n , ≤ k A v i k 2 = ( A v i ) T A v i = v T i A T A v i = v T i ( λ i v i ) Since v i is a eigenvector of A T A = λ i k v i k 2 = λ i Since v i is a unit vector (2) So the eigenvalue of A T A are all nonnegative (so A T A is positive semi definite). By renumbering, if necessary, we may assume that the eigenvalues are arranged so that λ 1 ≥ λ 2 ≥ ... ≥ λ n ≥ THE SINGULAR VALUES OF AN m × n MATRIX 7 The singular values of A are the square roots of the eigenvalues of A T A , denoted by σ 1...
View Full
Document
 Spring '09
 JeffWong
 Linear Algebra, Algebra

Click to edit the document details