Machine learning Principal Component Analysis PCA Principal Component Analysis

Machine learning principal component analysis pca

This preview shows page 92 - 101 out of 124 pages.

Machine learning – Principal Component Analysis (PCA) Principal Component Analysis (PCA) Find the principal axes (eigenvectors of the covariance matrix), Keep the ones with largest variations (largest eigenvalues), Project the data on this low-dimensional space , 74
Image of page 92

Subscribe to view the full document.

Machine learning – Principal Component Analysis (PCA) Principal Component Analysis (PCA) Find the principal axes (eigenvectors of the covariance matrix), Keep the ones with largest variations (largest eigenvalues), Project the data on this low-dimensional space, Change system of coordinate to reduce data dimension . 74
Image of page 93
Machine learning – Principal Component Analysis (PCA) Principal Component Analysis (PCA) Find the principal axes (eigenvectors of the covariance matrix), Keep the ones with largest variations (largest eigenvalues), Project the data on this low-dimensional space, Change system of coordinate to reduce data dimension . 74
Image of page 94

Subscribe to view the full document.

Machine learning – Principal Component Analysis (PCA) Principal Component Analysis (PCA) Find the principal axes of variations of x 1 , . . . , x N R d : μ = 1 N N X i =1 x i | {z } mean (vector) , Σ = 1 N N X i =1 ( x i - μ )( x i - μ ) T | {z } covariance (matrix) , Σ = V T Λ V | {z } eigen decomposition ( V V T = V T V = Id d ) V = ( v 1 , . . . , v d | {z } eigenvectors ) , Λ = diag( λ 1 , . . . , λ d | {z } eigenvalues ) and λ 1 > · · · > λ d Keep the K < d first dimensions: V K = ( v 1 , . . . , v K ) R d × K Project the data on this low-dimensional space: ˜ x i = μ + K X k =1 h v k , x i - μ i v k = μ + V K V T K ( x i - μ ) R d Change system of coordinate to reduce data dimension: h i = V T K ( ˜ x i - μ ) = V T K ( x i - μ ) R K 75
Image of page 95
Machine learning – Clustering – K-means Principal Component Analysis (PCA) Typically: from hundreds to a few (one to ten) dimensions, Number K of dimensions often chosen to cover 95 % of the variability: K = min ( K \ K k =1 λ k d k =1 λ k > . 95 ) PCA is done on training data, not on testing data! : First, learn the low-dimensional subspace on training data only, Then, project both the training and testing samples on this subspace, It’s an affine transform (translation, rotation, projection, rescaling): h = W x + b ( with W = V T K and b = - V T K μ ) Deep learning does something similar but in an (extremely) non-linear way. 76
Image of page 96

Subscribe to view the full document.

Machine learning – Feature extraction What features for an image? (Source: Michael Walker) 77
Image of page 97
Image representation La Trahison des images, Ren´ e Magritte, 1928 (Los Angeles County Museum of Art)
Image of page 98

Subscribe to view the full document.

Image representation How do we represent images? A two dimensional function Think of an image as a two dimensional function x . x ( s 1 , s 2 ) gives the intensity at location ( s 1 , s 2 ) . (Source: Steven Seitz) Convention: larger values correspond to brighter content. 78
Image of page 99
Image representation How do we represent images? A two dimensional function Think of an image as a two dimensional function x . x ( s 1 , s 2 ) gives the intensity at location ( s 1 , s 2 ) .
Image of page 100

Subscribe to view the full document.

Image of page 101

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask 0 bonus questions You can ask 0 questions (0 expire soon) You can ask 0 questions (will expire )
Answers in as fast as 15 minutes