Machine learning Principal Component Analysis PCA Principal Component Analysis

# Machine learning principal component analysis pca

• Test Prep
• 124

This preview shows page 92 - 101 out of 124 pages.

Machine learning – Principal Component Analysis (PCA) Principal Component Analysis (PCA) Find the principal axes (eigenvectors of the covariance matrix), Keep the ones with largest variations (largest eigenvalues), Project the data on this low-dimensional space , 74

Subscribe to view the full document.

Machine learning – Principal Component Analysis (PCA) Principal Component Analysis (PCA) Find the principal axes (eigenvectors of the covariance matrix), Keep the ones with largest variations (largest eigenvalues), Project the data on this low-dimensional space, Change system of coordinate to reduce data dimension . 74
Machine learning – Principal Component Analysis (PCA) Principal Component Analysis (PCA) Find the principal axes (eigenvectors of the covariance matrix), Keep the ones with largest variations (largest eigenvalues), Project the data on this low-dimensional space, Change system of coordinate to reduce data dimension . 74

Subscribe to view the full document.

Machine learning – Principal Component Analysis (PCA) Principal Component Analysis (PCA) Find the principal axes of variations of x 1 , . . . , x N R d : μ = 1 N N X i =1 x i | {z } mean (vector) , Σ = 1 N N X i =1 ( x i - μ )( x i - μ ) T | {z } covariance (matrix) , Σ = V T Λ V | {z } eigen decomposition ( V V T = V T V = Id d ) V = ( v 1 , . . . , v d | {z } eigenvectors ) , Λ = diag( λ 1 , . . . , λ d | {z } eigenvalues ) and λ 1 > · · · > λ d Keep the K < d first dimensions: V K = ( v 1 , . . . , v K ) R d × K Project the data on this low-dimensional space: ˜ x i = μ + K X k =1 h v k , x i - μ i v k = μ + V K V T K ( x i - μ ) R d Change system of coordinate to reduce data dimension: h i = V T K ( ˜ x i - μ ) = V T K ( x i - μ ) R K 75
Machine learning – Clustering – K-means Principal Component Analysis (PCA) Typically: from hundreds to a few (one to ten) dimensions, Number K of dimensions often chosen to cover 95 % of the variability: K = min ( K \ K k =1 λ k d k =1 λ k > . 95 ) PCA is done on training data, not on testing data! : First, learn the low-dimensional subspace on training data only, Then, project both the training and testing samples on this subspace, It’s an affine transform (translation, rotation, projection, rescaling): h = W x + b ( with W = V T K and b = - V T K μ ) Deep learning does something similar but in an (extremely) non-linear way. 76

Subscribe to view the full document.

Machine learning – Feature extraction What features for an image? (Source: Michael Walker) 77
Image representation La Trahison des images, Ren´ e Magritte, 1928 (Los Angeles County Museum of Art)

Subscribe to view the full document.

Image representation How do we represent images? A two dimensional function Think of an image as a two dimensional function x . x ( s 1 , s 2 ) gives the intensity at location ( s 1 , s 2 ) . (Source: Steven Seitz) Convention: larger values correspond to brighter content. 78
Image representation How do we represent images? A two dimensional function Think of an image as a two dimensional function x . x ( s 1 , s 2 ) gives the intensity at location ( s 1 , s 2 ) .

Subscribe to view the full document.

• Fall '08
• staff

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask 0 bonus questions You can ask 0 questions (0 expire soon) You can ask 0 questions (will expire )
Answers in as fast as 15 minutes