This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Unformatted text preview: Chapter 6 More about Eigenval— ues, Eigenvectors
Section 6.3 The Singular Value Decomposition Recall that if A is real symmetric, then
A = UDUT, where D is a diagonal matrix whose entries are the eigenvalues of A, and U is an orthogonal matrix,
UTU = UUT = I. There is an analogue of this for general (not necessarily square) matrices: DeﬁnitiOn Singular Value Decomposition (page 411)
Let A be a matrix. Suppose we can ﬁnd matrices V and U with VTV = I and
UT U = I, and a diagonal matrix D with nonnegative entries, such that A = VDUT. Then we call this a singular value decomposition of A. The diagonal entries of
D are called the singular values of A. Remarks
(a) Note that the singular values of a square matrix A are not necessarily the same as the eigenvalues of A. When A is symmetric, they are the absolute
values of the eigenvalues of A.
(b) The singular value decomposition is not unique, but always exists. The theory on how to ﬁnd a singular value decomposition
Assuming the decomposition, we have ATA = (VDUT)T(VDUT)
= (UT)TDTVTVTDUT
m UD(VTV)DUT
= UDIDUT
= UD2UT. This is a just our old diagonalisation of B = ATA. Note that B is symmetric,
so this diagonalisation exists. So we proceed as follows: (I) Compute B = ATA; (II) Find the eigenvalues of ATA, and its eigenvectors. Apply Gram—Schmidt
to these eigenvectors, to obtain an orthogonal matrix U such that ATA = UD2UT. 122 The singular values of A are the diagonal entries of D, and hence the square
roots of the eigenvalues of B.
(III) Assume all diagonal entries of D are nonzero. Form V = AUD‘I.
(Proof that this works when U is square: With this choice of V, VDUT
= (AUD4)DUT
= AUIUT
= AUUT=AI=AJ (IV) The singular values of A are the square roots of the eigenvalues of B. 4 Find its singular value decomposition.
Solution
Form Example
Let l—‘CDM [ONO
I_—l B = ATA=[ = The characteristic polynomial of B = ATA is O
2 HON 2 1
0 2 H HMO
I—J pB(t) = det(B—tI) = dell"? 82]) = w—nQ—a—4
=t14a+w
= u—me—n. So the eigenvalues of B = ATA are 9 and 4. Note that they are both positive. The singular values of A will be x/g = 3 and x/il = 2. Next, we ﬁnd the
eigenvectors of B. We solve (B—pI)v=0 123 by appiying row reduction to [B#I§0]= [ 5;“ 83“ 8] Eigenvectors of B for ,u = 9
Here [B#no]= [ 2 _1 0
Step 1 Row 2 +% Row 1
—4 2 0
0 0 0
Then :31 is pivotal, 3:2 is not, so set $2 = t.
Step 2 Back substitution —43:1 +2$2=0
=> —4sc1 = —2:t:2 = —2t
=>x1 =ét.
Then %t '1
[ H i ]
Set 15:2, and Wm. W—MM:[;E Step 1 Row 2 —2 Row 1 Eigenvectors of B for ,u = 4
Here 8] l 2 0 0 0 0
Then 51:1 is pivotal, $2 is not, so set :32 = 1:.
Step 2 Back substitution $1+2$2=0 =>m1 =—2:122=2t.
—2t —2
v[ t H1] 124 Then Set t = 1, and
Summary . p, = 4 is an eigenvalue of B with eigenvector V2 = _ : . NH p. = 9 is an eigenvalue of B with eigenvector v1 = [ The singular values of A are 3 and 2 and
3 0
D — i 0 2 l ' To ﬁnd the orthogonal matrix U in
ATA = UD2UT, we apply Gram—Schmidt to the eigenvectors {V1,v2}, as we did for ordinary
eigenvalues and eigenvectors. Step 1: 111
Recall that 1
‘11 _' "—V1
V1 Here V1=Vl2+22= 5,
so u iv = i 1 ] 1 x/E 1 . J5 2 Step 2: 112 Since B is symmetric, and V2 corresponds to a different eigenvalue (namely 4)
from v1 (namely 9), we already know that V2  u1 = 0, so we don’t have to
subtract out the component. All we need to do is form 1
u2=—v2.
V2
Iiere
v2 = #22 + 12 = J5,
SO
viva2]
2 «52 «5 1 ‘
Then _ _ 1N5 —2/;/3
U ‘ [mug—[2% l/x/Sl a: ﬂ 125 So
T _ 9 0 T
AA—U[0 4107. To ﬁnd the singular value decomposition of A, we use V = AUD“1 ll
l"'"'—‘I
I—IOM
o
L—...._l
mwo A
1.4
01
I"—I
[Oi—l
pal
ix: 1 l—""——1
(ﬁnAN: How “to
I
pp. G)
§u
rhth _1_.
fix/5 —12 1
8 6 = —
10 0 3‘5 Thus the singular value decomposition of A is I—"—"'_ A=VDUT,
m" r “l
0 2
1 2
= (slééﬁl)[321<ssﬂf
Remark The singular value decomposition is useful in many contexts. For example, recall
that the least squares solution of Ax = b is solved via the normal equations
ATAX =ATb. One can shew that the least squares solution is given by x = UDlva. 126 ...
View
Full Document
 Spring '08
 Morley
 Linear Algebra, Eigenvectors, Singular value decomposition, Orthogonal matrix

Click to edit the document details