Stat841f09 - Wiki Course Notes

This is the only new condition added here 2 dual

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: e we optimize with respect to two dual variables and , Applying KKT conditions [34] (http://en.wikipedia.org/wiki/Karus h%E2%80%93Kuhn%E2%80%93Tucker_conditions ) 1. at an optimal solution , for each primal variable wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 69/74 10/09/2013 Stat841 - Wiki Cour se Notes since the sign does not make a difference . This is the only new condition added here 2. , dual feasibility 3. , complementary slackness: 4. and , primal feasibility. Putting it all together With our KKT conditions and the Lagrangian equation, we can now use quadratic programming to find . Similar to what we did for the separable case after apply KKT conditions, replace the primal variables in terms of dual variables into the Lagrangian equations and simplify. In matrix form, we want to solve the following optimization: , Solving this gives us , which we can use to find However, we cannot find as before: in the same way as before, even if we choose a point with From our discussion on the KKT conditions, we know that So, if then Therefore, we can solve for and consequently and , because we do not know the value of in the equation . . if we choose a point where: Note When , we are considering a point that is on the margin. If then and we're dealing with a point that has crossed the margin. In this case, the local minimization is also the global minimization. Since is positive semidefinite, then is convex. The SVM algorithm for non-s e parable data s e ts The algorithm, then, for non- separable data sets is: 1. Use q a p o (or another quadratic programming technique) to solve the above optimization and find udrg 2. Find by solving 3. Find by choosing a point where and then solving Pote ntial drawbacks Potential drawbacks of the SVM are the following two aspects: 1.Uncalibrated Class membership probabilities[35] (http://en.wikipedia.org/wiki/Class_membership_probabilities) 2.The SVM is only directly applicable for two- class tasks. Therefore, algorithms...
View Full Document

This document was uploaded on 03/07/2014.

Ask a homework question - tutors are online