This preview shows page 1. Sign up to view the full content.
Unformatted text preview: ion boundary,
Let is a vector lying on the is orthogonal to the decision boundary.
Figure 3: This figure illustrates the derivation of the
distance between the decision boundary and
misclassified points be a misclassified point. Then the projection of the vector on the direction that is orthogonal to the decision boundary is is also on the decision boundary, then and so . Now, if . Looking at Figure 3, it can be seen that the distance between and the decision boundary is the absolute value of
Notice that if is classified correct ly then this product is positive. This is because if it is classified correctly, then either both ( they are both negative. However, if is classified incorrect ly then one of and and are positive or is positive and the other is negative. The result is that the above product is negative for a point that is misclassified. For the algorithm, we need only consider the distance between the misclassified points and the decision boundary.
which is a summation of positive numbers and where is the set of all misclassified points. The goal now becomes to
This can be done using a gradient descent approach (http://en.wikipedia.org/wiki/Gradient_descent) , which is a numerical method that takes one predetermined step in the
direction of the gradient, getting closer to a minimum at each step, until the gradient is zero. A problem with this algorithm is the possibility of getting stuck in a local minimum.
wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 36/74 10/09/2013 Stat841 - Wiki Cour se Notes To continue, the following derivatives are needed: Then the gradient descent type algorithm (Perceptron Algorithm) is where is the magnitude of each step called the "learning rate" or the "convergence rate". The algorithm continues until or until it has iterated a specified number of times. If the algorithm converges, it has found a linear classifier, ie., there are no misclassified points. Proble ms with the Algorithm (http://www.cs .cmu.e du/~avr...
View Full Document
This document was uploaded on 03/07/2014.
- Winter '13