This preview shows page 1. Sign up to view the full content.
Unformatted text preview: , we will get the
and predict the test set. Aside:
Similar in spirit to K means, there is EM algorithm (http://en.wikipedia.org/wiki/Expectation maximization_algorithm) with respect to Gaussian mixture model. Generally
speaking, the Gaussian mixture model is referred to as a soft clustering while K means is hard clustering.
Similar to K means, the following two steps are iterated alternately until convergernce.
E step, each point is assigned a weight for each cluster based on the likelihood of each of the corresponding Gaussians. Observations is assigned 1 for one cluster if they are
closer to the center of that cluster, and is assigned 0 for other clusters.
M step, compute the weighted means and covariances and make them as the new means and covariances for every cluster.
>[,upi1xr=dE(,,0,)
>Pm,h,Pt]mgMX2200; Support Vector Machine
Introduction
We have seen that linear discriminant analysis and logistic regression both estimate linear decision boundaries in similar but slightly different ways. Separating hyperplane
classifiers provide the basis for the support vector machine (SVM). An SVM constructs linear decision boundaries that explicitly try to separate the data into different
classes while maximizing the margin of separation. The techniques that make the extensions to the non separable case, where the classes overlap, are generalized to what is
wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 58/74 10/09/2013 Stat841  Wiki Cour se Notes known as the support vector machine. It produces nonlinear boundaries by constructing a linear boundary in a high dimensional, transformed version of the feature space. It
is also calculated based on only a fraction of the data rather than a calculation on every point in the data, much like the difference between the median and the mean.
The original basis for SVM was published in the 1960s by Vapnik (http://en.wikipedia.org/wiki/Vapnik) , Chervonenkis et al., however the ideas did not gain any attention
until st...
View
Full
Document
This document was uploaded on 03/07/2014.
 Winter '13

Click to edit the document details