This preview shows page 1. Sign up to view the full content.
Unformatted text preview: variance of projected points will be and If we sum this two quantities we have The quantity is called within clas s covariance or The goal is to minimize
Obje ctive Function
Instead of maximizing and minimizing wikicour senote.com/w/index.php?title= Stat841&pr intable= yes we can define the following objective function: 19/74 10/09/2013 Stat841 - Wiki Cour se Notes This maximization problem is equivalent to subject to constraint , where is no upper bound and is no lower bound.
We can use the Lagrange multiplier method to solve it:
where With we get: Note that
Here is the weight is sum of two positive matrices and so it has an inverse. is the eigenvector of corresponding to the largest eigenvalue . In facts, this expression can be simplified even more.
with The quantity and λ are scalars. So we can say the quantity is proportional to FDA vs . PCA Example in M atlab
We can compare PCA and FDA through the figure produced by matlab.
The following are the code to produce the figure step by step and the explanation for steps.
>=X;2; Create two multivariate normal random variables with . >po((:0,)X1302,.)
>1po((0:0,)X31602,r'; Plot the the data of the two classes respectively.
(,)1]; Using PCA to find the principal component and plot it.
>po(v11* 0,v21* 0,r)
>lt[(,)5 ][(,)5 ]'' Using FDA to find the principal component and plot it.
Now we can compare them through the figure. wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 20/74 10/09/2013 Stat841 - Wiki Cour se Notes P CA and FDA primary dimension for normal multivariate data, using matlab From the graph: when we see using PCA, we have a huge overlap for two classes, so PCA is not good. Ho...
View Full Document
- Winter '13