{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

intro_svm_new

# intro_svm_new - A Simple Introduction to Support Vector...

This preview shows pages 1–9. Sign up to view the full content.

A Simple Introduction to Support Vector Machines Martin Law Lecture for CSE 802 Department of Computer Science and Engineering Michigan State University

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
3/1/11 CSE 802. Prepared by Martin Law 2 Outline A brief history of SVM Large-margin linear classifier Linear separable Nonlinear separable Creating nonlinear classifiers: kernel trick A simple example Discussion on SVM Conclusion
3/1/11 CSE 802. Prepared by Martin Law 3 History of SVM SVM is related to statistical learning theory [3] SVM was first introduced in 1992 [1] SVM becomes popular because of its success in handwritten digit recognition 1.1% test error rate for SVM. This is the same as the error rates of a carefully constructed neural network, LeNet 4. See Section 5.11 in [2] or the discussion in [3] for details SVM is now regarded as an important example of “kernel methods”, one of the key area in machine learning Note: the meaning of “kernel” is different from the “kernel” function for Parzen windows [1] B.E. Boser et al . A Training Algorithm for Optimal Margin Classifiers. Proceedings of the Fifth Annual Workshop on Computational Learning Theory 5 144-152, Pittsburgh, 1992. [2] L. Bottou . Comparison of classifier methods: a case study in handwritten digit recognition. Proceedings of the 12th IAPR International Conference on Pattern Recognition, vol. 2, pp. 77-82. [3] V. Vapnik. The Nature of Statistical Learning Theory. 2 nd edition, Springer, 1999.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
3/1/11 CSE 802. Prepared by Martin Law 4 What is a good Decision Boundary? Consider a two-class, linearly separable classification problem Many decision boundaries! The Perceptron algorithm can be used to find such a boundary Different algorithms have been proposed (DHS ch. 5) Are all decision boundaries equally good? Class 1 Class 2
3/1/11 CSE 802. Prepared by Martin Law 5 Examples of Bad Decision Boundaries Class 1 Class 2 Class 1 Class 2

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
3/1/11 CSE 802. Prepared by Martin Law 6 Large-margin Decision Boundary The decision boundary should be as far away from the data of both classes as possible We should maximize the margin, m Distance between the origin and the line w t x =k is k/|| w || Class 1 Class 2 m
3/1/11 CSE 802. Prepared by Martin Law 7 Finding the Decision Boundary Let { x 1 , . .., n } be our data set and let y i {1,-1} be the class label of i The decision boundary should classify all points correctly The decision boundary can be found by solving the following constrained optimization problem This is a constrained optimization problem. Solving it requires some new tools Feel free to ignore the following several slides; what is important is the constrained optimization problem above

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
CSE 802. Prepared by Martin Law 8 Recap of Constrained Optimization Suppose we want to: minimize f( x ) subject to g( x ) = 0 A necessary condition for x 0 to be a solution: α : the Lagrange multiplier
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 56

intro_svm_new - A Simple Introduction to Support Vector...

This preview shows document pages 1 - 9. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online