{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Chap6.1-KernelMethods

Chap6.1-KernelMethods - Machine Learning Srihari Kernel...

This preview shows pages 1–9. Sign up to view the full content.

Machine Learning Srihari Kernel Methods Sargur Srihari 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Machine Learning Srihari Topics in Kernel Methods 1. Kernel Methods vs Linear Models/Neural Networks 2. Stored Sample Methods 3. Kernel Functions 4. Dual Representations 5. Constructing Kernels 6. Extension to Symbolic Inputs 7. Fisher Kernel 2
Machine Learning Srihari Kernel Methods vs Linear Models/Neural Networks Linear parametric models for regression and classiFcation have the form y (x,w) During learning phase we either get a maximum likelihood estimate of w or a posterior distribution of w Training data is then discarded Prediction based only on vecto r w This is true of Neural networks as well Another class of methods use the training samples or a subset of them 3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Machine Learning Srihari Examples of Stored-Sample Methods Training data points are used in prediction phase Examples of such methods Parzen probability density model Linear combination of kernel functions centered on each training data point Nearest neighbor classiFcation These are memory-based methods Require a metric to be deFned ±ast to train, slow to predict 4
Machine Learning Srihari Kernel Functions Many linear parametric models can be re-cast into equivalent dual representations where predictions are based on a kernel function evaluated at training points Kernel function is given by k (x,x ) = φ (x) T (x ) where (x) is a ±xed nonlinear feature space mapping (basis function) Kernel is a symmetric function of its arguments k (x,x ) = k (x ,x) Kernel function can be interpreted as the similarity of x and x’ Simplest is identity mapping in feature space (x) = x In which case k (x,x ) = x T x Called Linear Kernel 5

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Machine Learning Srihari Kernel Trick (or Kernel Substitution) Formulated as inner product allows extending well- known algorithms by using the kernel trick ! Basic idea of kernel trick If an input vector x appears only in the form of scalar products then we can replace scalar products with some other choice of kernel Used widely in support vector machines in developing non-linear variant of PCA In kernel Fisher discriminant 6
Machine Learning Srihari Other Forms of Kernel Functions Function of difference between arguments k (x,x ) = k (x-x ) Called stationary kernel since invariant to translation in space Homogeneous kernels, also known as radial basis functions k (x,x ) = k (||x-x ||) Depend only on the magnitude of the distance between arguments Note that the kernel function is a scalar value while x is an M - dimensional vector 7 For these to be valid kernel functions they should be shown to have the property k (x,x ) = φ (x) T (x )

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Machine Learning Srihari Dual Representation Linear models for regression and classiFcation can be reformulated in terms of a dual representation In which kernel function arises naturally Plays important role in SVMs
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

Page1 / 29

Chap6.1-KernelMethods - Machine Learning Srihari Kernel...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online