Lecture8b.slides.pdf - CS4487 Machine Learning Lecture 8...

This preview shows 1 out of 7 pages.

CS4487 - Machine Learning Lecture 8 - Non-Linear Dimensionality Reduction, Manifold Embedding Dr. Antoni B. Chan Dept. of Computer Science, City University of Hong Kong Outline 1. Non-Linear Dimensionality Reduction A. Kernel Principal Component Analysis (KPCA) 2. Manifold Embedding A. Locally-linear embedding (LLE) B. Multi-dimensional Scaling (MDS) C. Isometric Mapping (Isomap) D. Spectral Embedding (Laplacian Eigenmaps) E. t-distributed Stochastic Neighbor Embedding (t-SNE) Manifold Embedding Reduce high-dimensional data to 2 or 3 dimensions for visualization Try to preserve the inherent structure of the data find a set of lower-dim points that optimize some criteria. Two types: 1) preserve local neighborhood structure assumes thata lies in a lower-dim manifold; unfold the manifold 2) preserve pairwise distances (similarities) between points
Image of page 1

Subscribe to view the full document.

Locally-linear Embedding (LLE) Idea: preserve linearity within local neighborhoods defined by nearest neighbors. 1) a point can be reconstructed by a linear combination of its neighbors . find the weights for the best reconstruction 2) the embedded point should also have the same local linearity. find the embedded points that best preserve the linearity LLE K x i N i = W argmin W i x i j N i w i , j x j 2 y i = Y argmin Y i y i j N i w i , j y j 2
Image of page 2
Image of page 3

Subscribe to view the full document.

In [4]: # n_neighbors = number of nearest neighbors to use for local neighborhood # n_components = number of dimensions of manifold embedding lle = manifold . LocallyLinearEmbedding(n_neighbors =12 , n_components =2 , random_sta te =121 ) Xr = lle . fit_transform(X) plt . figure(figsize = ( 15 , 6 )) plot_manifolds(X, Y, [Xr], [ "LLE embedding" ]) Sensitive to the number of neighbors for defining the local region.
Image of page 4
In [7]: lfig Variants of LLE Hessian LLE (HLLE) : LLE that also uses local curvature information Out[7]:
Image of page 5

Subscribe to view the full document.

In [8]: # set method to 'hessian' hlle = manifold . LocallyLinearEmbedding(method = 'hessian' , n_neighbors
Image of page 6
Image of page 7
You've reached the end of this preview.
  • Fall '16
  • Antoni B. CHAN
  • Manifold, Euclidean space, lle

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern