class07

class07 - Unsupervised Learning Techniques 9.520 Class 07,...

Info iconThis preview shows pages 1–9. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: Unsupervised Learning Techniques 9.520 Class 07, 1 March 2006 Andrea Caponnetto About this class Goal To introduce some methods for unsupervised learn- ing: Gaussian Mixtures, K-Means, ISOMAP, HLLE, Laplacian Eigenmaps. Unsupervised learning Only u i.i.d. samples drawn on X from the unknown marginal distribution p ( x ) { x 1 , x 2 , . . . , x u } . The goal is to infer properties of this probability density. In low-dimension many nonparametric methods allow di- rect estimation of p ( x ) itself. Owing to the curse of di- mensionality , this methods fail in high dimension. One must settle for estimation of crude global models . Unsupervised learning (cont.) Different types of simple descriptive statistics that characterize aspects of p ( x ) • mixture modelling representation of p ( x ) by a mixture of simple densities representing different types or classes of observations [eg. Gaussian mixtures] • combinatorial clustering attempt to find multiple regions of X that contain modes of X [eg. K-Means] • dimensionality reduction attempt to identify low-dimensional manifolds in X that represent high data density [eg. ISOMAP,HLLE, Laplacian Eigenmaps] • manifold learning attempt to determine very specific geometrical or topological in- variants of p ( x ) [eg. Homology learning] Limited formalization With supervised and semi-supervised learning there is a clear measure of effectiveness of different methods. The expected loss of various estimators I [ f S ] can be estimated on validation set . In the context of unsupervised learning, it is difficult to find such a direct measure of success . This situation has led to proliferation of proposed meth- ods . Mixture Modelling Assumption that data is i.i.d. sampled from some proba- bility distribution p ( x ). p ( x ) is modelled as a mixture of component density func- tions, each component corresponding to a cluster or mode . The free parameters of the model are fit to the data by maximum likelihood . Gaussian Mixtures We first choose a parametric model P θ for the unknown density p ( x ), hence maximize the likelihood of our data relative to the parameters θ . Example: two-component gaussian mixture model with pa- rameters θ = ( π, µ 1 , Σ 1 , µ 2 , Σ 2 ) . The model: P θ ( x ) = (1 − π ) G Σ 1 ( x − µ 1 ) + πG Σ 2 ( x − µ 2 ) Maximize the log-likelihood u ( θ { x 1 , . . . , x u } ) = log P θ ( x i ) | i =1 The EM algorithm Maximization of ( θ |{ x 1 , . . . , x u } ) is a difficult problem. Iterative max- imization strategies, as the EM algorithm, can be used in practice to get local maxima ....
View Full Document

This note was uploaded on 11/11/2011 for the course BIO 9.07 taught by Professor Ruthrosenholtz during the Spring '04 term at MIT.

Page1 / 33

class07 - Unsupervised Learning Techniques 9.520 Class 07,...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online