MIT6_241JS11_lec04

MIT6_241JS11_lec04 - 6.241 Dynamic Systems and Control...

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 6.241 Dynamic Systems and Control Lecture 4: Singular Values Readings: DDV, Chapter 4 Emilio Frazzoli Aeronautics and Astronautics Massachusetts Institute of Technology February 14, 2011 E. Frazzoli (MIT) Lecture 4: Singular Values Feb 14, 2011 1/9 Outline 1 Singular Values 2 Norm computations through singular values E. Frazzoli (MIT) Lecture 4: Singular Values Feb 14, 2011 2/9 Unitary Matrices A square matrix U ∈ Cn×n is unitary if U � U = UU � = I . A square matrix U ∈ Rn×n is orthogonal if U T U = UU T = I . Properties: If U is a unitary matrix, then �Ux �2 = �x �2 , for all x ∈ Cn . If S = S � is a Hermitian matrix, then there exists a unitary matrix U such that U � SU is a diagonal matrix. 1 For any matrix A ∈ Rm×n , both A� A ∈ Rn×n , AA� ∈ Rm×m are Hermitian ⇒ can be diagonalized by unitary matrices. For any matrix A, the eigenvalues of A� A and AA� are always real 2 and non-negative 3 (in other words, A� A and AA� are positive definite). = S � ⇔ �Sx , y � = �x , Sy �. Let v1 be an eigenvector of S , and let M1 = R(v1 )⊥ . If u ∈ M1 , then so is Su : �Su , v1 � = �u , Sv1 � = �u , λ1 v1 � = 0. All other eigenvectors must b e in M1 . Finite induction gets the result. 2 Assuming �v , v � = 1, λ = �Sv , v � = �v , Sv � = �Sv , v �� = λ� 11 1 11 1 1 11 1 3 0 < �Av , Av � = v � A� Av = λ v � v . 1 1 1 111 1 1S E. Frazzoli (MIT) Lecture 4: Singular Values Feb 14, 2011 3/9 Singular Value Decomposition Theorem (SVD) Any matrix A ∈ Cm×n can be decomposed as A = U ΣV , where U ∈ Cm×m and V ∈ Cn×n are unitary matrices. The matrix Σ ∈ Rm×n is “diagonal,” with non-negative elements on the main diagonal. The non-zero elements of Σ are √ called the singular values of A, and satisfy σi = i-th eigenvalue of A� A. Proof (assuming rank(A) = m): Since AA� is Hermitian, there exist a diagonal matrix Λ = diag(λ1 , λ2 , . . . , λm ) > 0 such that U ΛU � = AA� . 2 2 2 Write Λ = Σ2 = diag(σ1 , σ2 , . . . , σm ) 1 � � Define V1 := Σ−1 U � A ∈ Rm×n . Clearly, V1 V1 = Σ−1 U � AA� U Σ−1 = I m×m . 1 1 1 Construct V = [V1 , V2 ] ∈ Cn×n by choosing the columns in V2 so that V is unitary, and Σ = [Σ1 , 0] ∈ Rn×n , by padding with zeroes. � � Hence, ΣV � = Σ1 V1 + 0V2 = U � A, i.e., A = U ΣV � . E. Frazzoli (MIT) Lecture 4: Singular Values Feb 14, 2011 4/9 Singular Vectors If U and V are written as sequences of column vectors, i.e., � � � � U = u1 , u2 , . . . , um and V = v1 , v2 , . . . , vn , then A = U ΣV � = r � σi ui vi� i =1 The columns of U are called the left singular vectors, and the columns of V are called the right singular vectors. Note: Ax can be written as the weighted sum of the left singular vectors, where the weights are given by the projections of x onto the right singular vectors: Ax = r � σi ui (vi� x ), i =1 The range of A is given by the span of the first r vectors in U The rank of A is given by r ; The nullspace of A is given the span of the last (n − r ) vectors in V . E. Frazzoli (MIT) Lecture 4: Singular Values Feb 14, 2011 5/9 Outline 1 Singular Values 2 Norm computations through singular values E. Frazzoli (MIT) Lecture 4: Singular Values Feb 14, 2011 6/9 Induced 2-norm computation Theorem (Induced 2-norm) �A�2 = sup x �=0 �Ax �2 = σmax (A). �x �2 Proof: sup x =0 � �Ax �2 �U ΣV � x �2 �ΣV � x �2 = sup = sup = �x �2 �x �2 �x �2 x �=0 x �=0 ��n � 2 2 1/2 �Σy �2 �Σy �2 i = σ | yi | sup = sup = sup �� 1 i �1/2 ≤ σmax (A). n y =0 �y �2 y =0 y �=0 �Vy �2 � � |yi |2 i =1 Assuming σmax = σ1 , the supremum is attained for y = (1, 0, . . . , 0). This corresponds to x = v1 , and Av1 = σ u1 E. Frazzoli (MIT) Lecture 4: Singular Values Feb 14, 2011 7/9 Minimal amplification Theorem Given A ∈ Cm×n , with rank(A) = n, �Ax �2 = σn (A). x �=0 �x �2 inf Proof: �Ax �2 �U ΣV � x �2 �ΣV � x �2 = inf = inf = x �=0 �x �2 x �=0 x �=0 �x �2 �x �2 ��n � 2 2 1/2 �Σy �2 �Σy �2 i =1 σi |yi | inf = inf = inf �� �1/2 ≥ σmin (A). n y =0 �y �2 y �=0 y �=0 �Vy �2 � |yi |2 inf i =1 Assuming σmin = σn , the supremum is attained for y = (0, . . . , 0, 1). This corresponds to x = vn , and Avn = σ un E. Frazzoli (MIT) Lecture 4: Singular Values Feb 14, 2011 8/9 Frobenius norm computation Theorem �A�F = �r � �1/2 2 σi (A) i =1 Proof: ⎞1/2 ⎛ n m �� 1/2 1/2 |aij |2 ⎠ = (Trace(A� A)) = (Trace(V Σ� U � U ΣV � )) = �A�F = ⎝ j =1 i =1 � �1/2 � �1/2 Trace(V � V Σ2 ) = Trace(Σ2 ) = �r � �1/2 σi2 i =1 E. Frazzoli (MIT) Lecture 4: Singular Values Feb 14, 2011 9/9 MIT OpenCourseWare http://ocw.mit.edu 6.241J / 16.338J Dynamic Systems and Control Spring 2011 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms . ...
View Full Document

Ask a homework question - tutors are online