This preview shows page 1. Sign up to view the full content.
Unformatted text preview: A for Multi-class Problems - October 14, 2009
FDA method for Multi-clas s Problems
For the k - class problem, we need to find a projection from d- dimensional space to a (k − 1)- dimensional space.
(It is more reasonable to have at least 2 directions)
Basically, the within class covariance matrix where is easily to obtain: and . However, the between class covariance matrix
is not easy to obtain. One of the simplifications is that we may assume that the total covariance
is easy to compute, we can get
using the following relationship: Actually, there is another generation for Thus the total covariance matrix . Denote a total mean vector of the data is by is wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 22/74 10/09/2013 Stat841 - Wiki Cour se Notes Thus we obtain Since the total covariance
is the sum of the within class covariance
class covariance matrix
, thus we obtain and the between class covariance , we can denote the second term as the general between Therefore, Recall that in the two class case problem, we have From the general form, Apparently, they are very similar.
Now, we are trying to find the optimal transformation. Basically, we have where is a vector, is a transformation matrix, i.e. , and is a column vector. Thus we obtain Similarly, we obtain wikicour senote.com/w/index.php?title= Stat841&pr intable= yes 23/74 10/09/2013 Stat841 - Wiki Cour se Notes Now, we use the determinant of the matrix, i.e. the product of the eigenvalues of the matrix, as our measure. The solution for this question is that the columns of the transformation matrix are exactly the eigenvectors that correspond to largest k − 1 eigenvalues with respect to Also, note that we can use as our measure.
Recall that Thus we obtain that Similarly, we can get . Thus we have following criterion function Similar to the two class case problem, we have:
max subject to To solve this optimization problem a Lagrange multiplier Λ, which actually is a Differen...
View Full Document
This document was uploaded on 03/07/2014.
- Winter '13