intro_svm_new - A Simple Introduction to Support Vector...

Info iconThis preview shows pages 1–9. Sign up to view the full content.

View Full Document Right Arrow Icon
A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
3/1/11 CSE 802. Prepared by Martin Law 2 Outline A brief history of SVM Large-margin linear classifier Linear separable Nonlinear separable Creating nonlinear classifiers: kernel trick A simple example Discussion on SVM Conclusion
Background image of page 2
3/1/11 CSE 802. Prepared by Martin Law 3 History of SVM SVM is related to statistical learning theory [3] SVM was first introduced in 1992 [1] SVM becomes popular because of its success in handwritten digit recognition 1.1% test error rate for SVM. This is the same as the error rates of a carefully constructed neural network, LeNet 4. See Section 5.11 in [2] or the discussion in [3] for details SVM is now regarded as an important example of “kernel methods”, one of the key area in machine learning Note: the meaning of “kernel” is different from the “kernel” function for Parzen windows [1] B.E. Boser et al . A Training Algorithm for Optimal Margin Classifiers. Proceedings of the Fifth Annual Workshop on Computational Learning Theory 5 144-152, Pittsburgh, 1992. [2] L. Bottou . Comparison of classifier methods: a case study in handwritten digit recognition. Proceedings of the 12th IAPR International Conference on Pattern Recognition, vol. 2, pp. 77-82. [3] V. Vapnik. The Nature of Statistical Learning Theory. 2 nd edition, Springer, 1999.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
3/1/11 CSE 802. Prepared by Martin Law 4 What is a good Decision Boundary? Consider a two-class, linearly separable classification problem Many decision boundaries! The Perceptron algorithm can be used to find such a boundary Different algorithms have been proposed (DHS ch. 5) Are all decision boundaries equally good? Class 1 Class 2
Background image of page 4
3/1/11 CSE 802. Prepared by Martin Law 5 Examples of Bad Decision Boundaries Class 1 Class 2 Class 1 Class 2
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
3/1/11 CSE 802. Prepared by Martin Law 6 Large-margin Decision Boundary The decision boundary should be as far away from the data of both classes as possible We should maximize the margin, m Distance between the origin and the line w t x =k is k/|| w || Class 1 Class 2 m
Background image of page 6
3/1/11 CSE 802. Prepared by Martin Law 7 Finding the Decision Boundary Let { x 1 , . .., n } be our data set and let y i {1,-1} be the class label of i The decision boundary should classify all points correctly The decision boundary can be found by solving the following constrained optimization problem This is a constrained optimization problem. Solving it requires some new tools Feel free to ignore the following several slides; what is important is the constrained optimization problem above
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
CSE 802. Prepared by Martin Law 8 Recap of Constrained Optimization Suppose we want to: minimize f( x ) subject to g( x ) = 0 A necessary condition for x 0 to be a solution: α : the Lagrange multiplier
Background image of page 8
Image of page 9
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 11/30/2011 for the course CIS 6930 taught by Professor Staff during the Spring '08 term at University of Florida.

Page1 / 56

intro_svm_new - A Simple Introduction to Support Vector...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online