svmtutorialPSB2002

# svmtutorialPSB2002 - Support Vector and Kernel Methods for...

This preview shows pages 1–8. Sign up to view the full content.

1 Support Vector and Kernel Methods for Pattern Recognition Nello Cristianini BIOwulf Technologies [email protected] http:// www.support-vector.net/tutorial.html PSB 2002 www.support-vector.net A Little History ! Support Vector Machines (SVM) introduced in COLT- 92 (conference on learning theory) greatly developed since then. ! Result: a class of algorithms for Pattern Recognition (Kernel Machines) ! Now: a large and diverse community, from machine learning, optimization, statistics, neural networks, functional analysis, etc. etc ! Centralized website: www.kernel-machines.org ! Textbook (2000): see www.support-vector.net

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
2 www.support-vector.net Basic Idea ! Kernel Methods work by embedding the data into a vector space, and by detecting linear relations in that space ! Convex Optimization, Statistical Learning Theory, Functional Analysis are the main tools www.support-vector.net Basic Idea ! “Linear relations”: can be regressions, classifications, correlations, principal components, etc. ! If the feature space chosen suitably, pattern recognition can be easy
3 www.support-vector.net General Structure of Kernel-Based Algorithms ! Two Separate Modules: Learning Module Kernel Function A learning algorithm: performs the learning In the embedding space A kernel function: takes care of the embedding www.support-vector.net Overview of the Tutorial ! Introduce basic concepts with extended example of Kernel Perceptron ! Derive Support Vector Machines ! Other kernel based algorithms (PCA;regression; clustering;…) ! Bioinformatics Applications

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
4 www.support-vector.net Just in case … ! Inner product between vectors ! Hyperplane: xz ii i , = x x o o o x x o o o x x x w b wx b , += 0 www.support-vector.net Preview ! Kernel methods exploit information about the inner products between data items ! Many standard algorithms can be rewritten so that they only require inner products between data (inputs) ! Kernel functions = inner products in some feature space (potentially very complex) ! If kernel given, no need to specify what features of the data are being used
5 www.support-vector.net Basic Notation ! Input space ! Output space ! Hypothesis ! Real-valued: ! Training Set ! Test error ! Dot product x X yY hH fX Sx y x y xz ii ∈=−+ = {,} :R {( , ),. ..,( , ),. ...} , 11 ε www.support-vector.net Basic Example: the Kernel-Perceptron ! We will introduce the main ideas of this approach by using an example: the simplest algorithm with the simplest kernel ! Then we will generalize to general algorithms and general kernels

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
6 www.support-vector.net Perceptron ! Simplest case: classification. Decision function is a hyperplane in input space ! The Perceptron Algorithm (Rosenblatt, 57) ! Useful to analyze the Perceptron algorithm, before looking at SVMs and Kernel Methods in general www.support-vector.net Perceptron ! Linear Separation of the input space x x o o o x x o o o x x x w b fx wx b hx s ignf x () , (() ) =+ =
7 www.support-vector.net Perceptron Algorithm Update rule (ignoring threshold): ! if then ywx ik i (, ) 0 ww y x kk i i + ←+ 1 1 η www.support-vector.net Observations ! Solution is a linear combination of training points !

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 69

svmtutorialPSB2002 - Support Vector and Kernel Methods for...

This preview shows document pages 1 - 8. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online