lec06-SVM - SVM - Support Vector Machines A new...

Info iconThis preview shows pages 1–12. Sign up to view the full content.

View Full Document Right Arrow Icon
SVM - Support Vector Machines A new classification method for both linear and nonlinear data It uses a nonlinear mapping to transform the original training data into a higher dimension With the new dimension, it searches for the linear optimal separating hyperplane (i.e., “decision boundary”) With an appropriate nonlinear mapping to a sufficiently high dimension, data from two classes can always be separated by a hyperplane SVM finds this hyperplane using support vectors (“essential” training tuples) and margins (defined by the support vectors)
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
SVM - History and Applications Vapnik and colleagues (1992)—groundwork from Vapnik & Chervonenkis’ statistical learning theory in 1960s Features: training can be slow but accuracy is high owing to their ability to model complex nonlinear decision boundaries (margin maximization) Used both for classification and prediction Applications: handwritten digit recognition, object recognition, speaker identification, benchmarking time-series prediction tests
Background image of page 2
Linear Classifiers Consider a two dimensional dataset with two classes How would we classify this dataset?
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Linear Classifiers Both of the lines can be linear classifiers.
Background image of page 4
Linear Classifiers There are many lines that can be linear classifiers. Which one is the optimal classifier.
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Classifier Margin Define the margin of a linear classifier as the width that theboundary could be increased by before hitting a datapoint.
Background image of page 6
Maximum Margin The maximum margin linear classifier is the linear classifier with the maximum margin. This is the simplest kind of SVM (Called Linear SVM)
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Support Vectors Examples closest to the hyperplane are support vectors . Margin ρ of the separator is the distance between support vectors. ρ w . x + b > 0 w . x + b < 0 w . x + b = 0 Support Vectors f ( x ) = sign( w . x + b ) red is +1 blue is -1
Background image of page 8
Support Vectors Distance from example x i to the separator is w x w b r i + = . r ρ 2 2 1 ... || || n w w is w + +
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
SVM  -  Linearly Separable A separating hyperplane can be written as W X + b = 0 – where W ={w 1 , w 2 , …, w n } is a weight vector and b a scalar (bias) For 2-D it can be written as • w 0 + w 1 x 1 + w 2 x 2 = 0 The hyperplane defining the sides of the margin: • H 1 : w 0 + w 1 x 1 + w 2 x 2 ≥ 1 for y i = +1, and • H 2 : w 0 + w 1 x 1 + w 2 x 2 1 for y i = 1 Any training tuples that fall on hyperplanes H 1 or H 2 (i.e., the sides defining the margin) are support vectors
Background image of page 10
Linear SVM Mathematically CS464 Introduction to Machine Learning 11 Let training set {( x i , y i )} i =1. . n , x i R d , y i {-1, 1} be separated by a hyperplane with margin ρ . Then for each training example (
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 12
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 12/27/2009 for the course CS 464 taught by Professor Demir during the Fall '08 term at Bilkent University.

Page1 / 25

lec06-SVM - SVM - Support Vector Machines A new...

This preview shows document pages 1 - 12. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online