Unformatted text preview: class having possibly different size. To be able to separate the two classes we must determine the class whose mean is closest to a given point while also
accounting for the different size of each class, which is represented by the covariance of each class.
wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 18/74 10/09/2013 Stat841  Wiki Cour se Notes Assume and 1st class, and , represent the mean and covariance of the
and represent the mean and covariance of the 2nd class. We have to find a transformation which satisfies the following goals:
1.To mak e t he means of t hese t wo classes as f ar apart as possible
In other words, the goal is to maximize the distance after projection between
class 1 and class 2. This can be done by maximizing the distance between the
means of the classes after projection. When projecting the data points to a
one dimensional space, all points will be projected to a single line; the line we
seek is the one with the direction that achieves maximum separation of
classes upon projetion. If the original points are
and the projected
points are then the mean of the projected points will be and for class 1 and class 2 respectively. The goal now becomes to
maximize the Euclidean distance between projected means,
. The steps of this maximization
are given below. P CA vs FDA 2.We want t o collapse all dat a point s of each class t o a single point , i.e.,
minimize t he cov ariance wit hin classes
Notice that the variance of the projected classes 1 and 2 are given by and . The second goal is to minimize the sum of these two covariances. As is demonstrated below, both of these goals can be accomplished simultaneously.
Original points are Projected points are with is a sclar Be twe e n clas s covariance
In this particular case, we want to project all the data points in one dimensional space.
We want to maximize the Euclidean distance between projected means, which is
which is scalar The quantity is called be twe e n clas s covariance or . The goal is to maximize :
Within clas s covariance
Covariance of class 1 is Covariance of class 2 is So co...
View
Full
Document
This document was uploaded on 03/07/2014.
 Winter '13

Click to edit the document details