# Svd compression key idea given a collection of

• Notes
• 65

This preview shows 29 out of 42 pages.

SVD Compression Key Idea : Given a collection of vectors in n -dimensional space, find a good m -dimensional subspace ( m << n ) in which to represent the vectors.

Subscribe to view the full document.

SVD Compression Specifically : If P={ p 1 ,…,p k } is the initial n -dimensional point set, and { w 1 ,…,w m } is an orthonormal basis for the m -dimensional subspace, we will compress the point set by sending: ( 29 m w p w p p , , , , 1
SVD Compression Challenge : To find the m -dimensional subspace that will best capture the initial point information.

Subscribe to view the full document.

Variance of the Point Set Given a collection of points P ={ p 1 ,…, p k }, in an n -dimensional vector space, determine how the vectors are distributed across different directions. p i p 1 p 2 p k
Variance of the Point Set Define the Var P as the function: giving the variance of the point set P in direction v (assume | v |=1). p i p 1 p 2 p k ( 29 = = = = k i i v k i i P p v p v Var 1 2 1 2 , ) ( π

Subscribe to view the full document.

Variance of the Point Set More generally, for a subspace W V , define the variance of P in the subspace W as: If { w 1 ,…,w m } is an orthonormal basis for W , then: ( 29 = = n i i W P p W Var 1 2 ) ( π ∑ ∑ = = = = = m j j P n i m j j i P w Var w p W Var 1 1 1 2 ) ( , ) (
Variance of the Point Set Example : The variance in the direction v 1 is large, while the variance in the direction v 2 is small. If we want to compress down to one dimension, we should project the points onto v 1 p i p 2 p 1 v v 1 v v 2 p k

Subscribe to view the full document.

Covariance Matrix Definition : The covariance matrix M P , of a point set P ={ p 1 ,…,p k } is the symmetric matrix which is the sum of the outer products of the p i : = = k i t i i P p p M 1
Covariance Matrix Theorem : The variance of the point set P in a direction v is equal to: v M v v p v Var p t k i i P = = = 1 2 , ) (

Subscribe to view the full document.

Covariance Matrix Theorem : The variance of the point set P in a direction v is equal to: Proof : v M v v p v Var p t k i i P = = = 1 2 , ) ( ( 29 ( 29 ( 29 ( 29 = = = = = = = = = k i P i k i k i t i i t t i i t p n i t i i t p t v Var p v v p p v v p p v v p p v v M v 1 2 1 1 1 ,
Singular Value Decomposition Theorem : Every symmetric matrix M can be written out as the product: where O is a rotation/reflection matrix ( OO t =Id) and D is a diagonal matrix with the property: t ODO M = n n D λ λ λ λ λ = 2 1 1 with 0 0

Subscribe to view the full document.

Singular Value Decomposition Implication : Given a point set P , we can compute the covariance matrix of the point set, M P , and express the matrix in terms of its SVD factorization: where { v 1 ,…,v n } is an orthonormal basis and λ i is the variance of the point set in direction v i . ( 29 n t n t n n P v v v v M λ λ λ λ λ = 2 1 1 1 1 with 0 0
Singular Value Decomposition Compression : The vector subspace spanned by { v 1 ,…,v m } is the vector sub-space that maximizes the variance in the initial point set P .

Subscribe to view the full document.

You've reached the end of this preview.
• Spring '10
• unknown
• Variance, Singular value decomposition, Euclidean space, varp

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern