Stat841f09 - Wiki Course Notes

# Pos global optimization method no local optima svm

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: ch stops when the search step size is reduced to a specified tolerance. Pos itives and Negatives When Optimizing SVM[30] (http://www.cs e.unr.edu/~bebis /MathMethods /SVM/lecture.pdf) (Pos) Appears to avoid overfitting in high dimensional spaces and generalize well using a small training set (the complexity of SVM is characterized by the number of support vectors rather than the dimensionality of the transformed space - - no formal theory to justify this). (Pos) Global optimization method, no local optima (SVM are based on exact optimization, not approximate methods). (Neg) Applying trained classifiers can be expensive. The Support Vector Machine algorithm - November 18, 2009 wikicour senote.com/w/index.php?title= Stat841&amp;pr intable= yes 62/74 10/09/2013 Stat841 - Wiki Cour se Notes Lagrange Duality (http://en.wikipedia.org/wiki/Lagrange_multipliers ) In convex optimization, consider the primal optimization problem: Define the generalized Lagrangian to be Then the dual optimization problem is Now instead of solving the primal problem, we can solve the dual problem without changing the solution as long as it subjects to the Karush- Kuhn- Tucker (KKT) conditions: We are interested in the dual form because it gives bound on the primal problem and in some cases is easier to solve. For more information about convex optimization, see the book by Boyd: [31] (http://www.stanford.edu/~boyd/cvxbook/) Solving the Lagrangian Continuing from the above derivation, we now have the equation that we need to minimize, as well as two constraints. The Support Vector Machine problem boils down to: such that and We are looking to minimize , which is our only unknown. Once we know , we can easily find and (see the Support Vector algorithm below for complete details). If we examine the Lagrangian equation, we can see that is multiplied by itself; that is, the Lagrangian is quadratic with respect to . Our constraints are linear. This is therefore a problem that can be solved through quadratic programming (http://en.wikipedia.org/wiki/Quadratic_programmi...
View Full Document

## This document was uploaded on 03/07/2014.

Ask a homework question - tutors are online