Chap6.1-KernelMethods

Chap6.1-KernelMethods - Machine Learning Srihari Kernel...

Info iconThis preview shows pages 1–9. Sign up to view the full content.

View Full Document Right Arrow Icon
Machine Learning Srihari Kernel Methods Sargur Srihari 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Machine Learning Srihari Topics in Kernel Methods 1. Kernel Methods vs Linear Models/Neural Networks 2. Stored Sample Methods 3. Kernel Functions 4. Dual Representations 5. Constructing Kernels 6. Extension to Symbolic Inputs 7. Fisher Kernel 2
Background image of page 2
Machine Learning Srihari Kernel Methods vs Linear Models/Neural Networks Linear parametric models for regression and classiFcation have the form y (x,w) During learning phase we either get a maximum likelihood estimate of w or a posterior distribution of w Training data is then discarded Prediction based only on vecto r w This is true of Neural networks as well Another class of methods use the training samples or a subset of them 3
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Machine Learning Srihari Examples of Stored-Sample Methods Training data points are used in prediction phase Examples of such methods Parzen probability density model Linear combination of kernel functions centered on each training data point Nearest neighbor classiFcation These are memory-based methods Require a metric to be deFned ±ast to train, slow to predict 4
Background image of page 4
Machine Learning Srihari Kernel Functions Many linear parametric models can be re-cast into equivalent dual representations where predictions are based on a kernel function evaluated at training points Kernel function is given by k (x,x ) = φ (x) T (x ) where (x) is a ±xed nonlinear feature space mapping (basis function) Kernel is a symmetric function of its arguments k (x,x ) = k (x ,x) Kernel function can be interpreted as the similarity of x and x’ Simplest is identity mapping in feature space (x) = x In which case k (x,x ) = x T x Called Linear Kernel 5
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Machine Learning Srihari Kernel Trick (or Kernel Substitution) Formulated as inner product allows extending well- known algorithms by using the kernel trick ! Basic idea of kernel trick If an input vector x appears only in the form of scalar products then we can replace scalar products with some other choice of kernel Used widely in support vector machines in developing non-linear variant of PCA In kernel Fisher discriminant 6
Background image of page 6
Machine Learning Srihari Other Forms of Kernel Functions Function of difference between arguments k (x,x ) = k (x-x ) Called stationary kernel since invariant to translation in space Homogeneous kernels, also known as radial basis functions k (x,x ) = k (||x-x ||) Depend only on the magnitude of the distance between arguments Note that the kernel function is a scalar value while x is an M - dimensional vector 7 For these to be valid kernel functions they should be shown to have the property k (x,x ) = φ (x) T (x )
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Machine Learning Srihari Dual Representation Linear models for regression and classiFcation can be reformulated in terms of a dual representation In which kernel function arises naturally Plays important role in SVMs
Background image of page 8
Image of page 9
This is the end of the preview. Sign up to access the rest of the document.

This document was uploaded on 02/25/2012.

Page1 / 29

Chap6.1-KernelMethods - Machine Learning Srihari Kernel...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online