This preview shows page 1. Sign up to view the full content.
Unformatted text preview: and transformed feature space.
Pe rce ptron
Recall the use of Least Squares regression as a classifier, shown to be identical to LDA. To classify points with
least squares we take the sign of a linear combination of data points and assign a label equivalent to +1 or - 1.
Least Squares returns the sign of a linear combination of data points as the class label Figure 1: Diagram of a linear perceptron. In the 1950s Frank Rosenblatt (http://en.wikipedia.org/wiki/Frank_Rosenblatt) developed an iterative linear
classifier while at Cornell University known as the Perceptron. The concept of a perceptron was fundamental to the later development of the Artificial Neural Network
(http://en.wikipedia.org/wiki/Artificial_neural_network) models. The perceptron is a simple type of neural network which models the electrical signals of biological neurons
(http://en.wikipedia.org/wiki/Biological_neural_network) . In fact, it was the first neural network to be algorithmically described. 
As in other linear classification methods like Least Squares, Rosenblatt's classifier determines a hyperplane for the decision boundary. Linear methods all determine slightly
different decision boundaries, Rosenblatt's algorithm seeks to minimize the distance between the decision boundary and the misclassified points .
Particular to the iterative nature of the solution, the problem has no global mean (not convex). It does not converge to give a unique hyperplane, and the solutions depend on
the size of the gap between classes. If the classes are separable then the algorithm is shown to converge to a local mean. The proof of this convergence is known as the
percept ron conv ergence t heorem. However, for overlapping classes convergence to a local mean cannot be guaranteed.
If we find a hyperplane that is not unique between 2 classes, there will be infinitely many solutions obtained from the perceptron algorithm.
As seen in Figure 1, after training, the perceptron determines the label of the data by computing the sign of a line...
View Full Document
- Winter '13