This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ch stops when the search step size is reduced to a specified tolerance. Pos itives and Negatives When Optimizing SVM[30] (http://www.cs e.unr.edu/~bebis /MathMethods /SVM/lecture.pdf)
(Pos) Appears to avoid overfitting in high dimensional spaces and generalize well using a small training set (the complexity of SVM is characterized by the number of
support vectors rather than the dimensionality of the transformed space   no formal theory to justify this).
(Pos) Global optimization method, no local optima (SVM are based on exact optimization, not approximate methods).
(Neg) Applying trained classifiers can be expensive. The Support Vector Machine algorithm  November 18, 2009
wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 62/74 10/09/2013 Stat841  Wiki Cour se Notes Lagrange Duality (http://en.wikipedia.org/wiki/Lagrange_multipliers )
In convex optimization, consider the primal optimization problem: Define the generalized Lagrangian to be Then the dual optimization problem is Now instead of solving the primal problem, we can solve the dual problem without changing the solution as long as it subjects to the Karush Kuhn Tucker (KKT)
conditions: We are interested in the dual form because it gives bound on the primal problem and in some cases is easier to solve. For more information about convex optimization, see
the book by Boyd: [31] (http://www.stanford.edu/~boyd/cvxbook/) Solving the Lagrangian
Continuing from the above derivation, we now have the equation that we need to minimize, as well as two constraints.
The Support Vector Machine problem boils down to: such that
and
We are looking to minimize , which is our only unknown. Once we know , we can easily find and (see the Support Vector algorithm below for complete details). If we examine the Lagrangian equation, we can see that is multiplied by itself; that is, the Lagrangian is quadratic with respect to . Our constraints are linear. This is
therefore a problem that can be solved through quadratic programming (http://en.wikipedia.org/wiki/Quadratic_programmi...
View
Full
Document
This document was uploaded on 03/07/2014.
 Winter '13

Click to edit the document details