class_6 - CS205 Class 6 Reading Heath 3.6(p137-143 4.7(p202...

Info iconThis preview shows pages 1–2. Sign up to view the full content.

View Full Document Right Arrow Icon
CS205 – Class 6 Reading: Heath 3.6 (p137-143), 4.7 (p202) Singular Value Decomposition (SVD) contd. 1. SVD is a transformation into a diagonal axis aligned space. a. Transform b into the space spanned by T U , b b U x V x V U U T T T T ˆ . No information is lost going from b to b ˆ because T U is square and orthogonal. b. Replace x V T by x ˆ to get a diagonal system, b x x V T ˆ ˆ . c. Now solve the system b x ˆ ˆ simply by scaling elements of b ˆ by the singular values. d. The original x is then recovered as x V x ˆ . e. Essentially the SVD solves the matrix by transforming the vectors in a space with eigenvectors along the unit axis. 2. T T i i i i A U V u v proof: define ) , min( n m l , U ˆ the first l columns of U , ˆ the square l l submatrix from the upper left corner of , V ˆ the first l columns of V . Then     T l l T l T l T l l T T v v u u v v u u V U V U A 1 1 1 1 1 1 ˆ ˆ ˆ l i T i i i l i n i m i i i m i i n i i i i i i l i n i m i i l i i m i i l i n i i i l i i i i v u v u v u v u v u v u v u v u v u 1 1 1 1 1 1 1 1 1 1 1 1 1 1 a. Note that “zero” or “small” i produce terms that contribute little to the sum, and that large i produce terms that contribute significantly to the sum. b. If the “zero” or “small” i are omitted from the summation, one obtains a matrix with lower rank. For example, if only the first k terms are summed, the result has rank k. i. Moreover, it can be shown that this new rank k matrix is the closest rank k matrix to A in both the L 2 and the Frobenius norm. ii. This is the key idea in PCA, clustering/data mining algorithms, etc. 3. The “pseudo-inverse” of a matrix A is defined by T A V U where is obtained from by replacing all “nonzero” i with 1/ i , and leaving all the zero entries identically zero . a. If A is square and nonsingular (
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 2
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 4

class_6 - CS205 Class 6 Reading Heath 3.6(p137-143 4.7(p202...

This preview shows document pages 1 - 2. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online