This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: Locality Sensitive Discriminant Analysis ∗ Deng Cai Department of Computer Science University of Illinois at Urbana Champaign [email protected] Xiaofei He Yahoo! Research Labs [email protected] Kun Zhou Microsoft Research Asia [email protected] Jiawei Han Department of Computer Science University of Illinois at Urbana Champaign [email protected] Hujun Bao College of Computer Science Zhejiang University [email protected] Abstract Linear Discriminant Analysis (LDA) is a popular data-analytic tool for studying the class relation- ship between data points. A major disadvantage of LDA is that it fails to discover the local geometri- cal structure of the data manifold. In this paper, we introduce a novel linear algorithm for discriminant analysis, called Locality Sensitive Discriminant Analysis (LSDA). When there is no sufficient train- ing samples, local structure is generally more im- portant than global structure for discriminant analy- sis. By discovering the local manifold structure, LSDA finds a projection which maximizes the mar- gin between data points from different classes at each local area. Specifically, the data points are mapped into a subspace in which the nearby points with the same label are close to each other while the nearby points with different labels are far apart. Ex- periments carried out on several standard face data- bases show a clear improvement over the results of LDA-based recognition. 1 Introduction Practical algorithms in supervised machine learning degrade in performance (prediction accuracy) when faced with many features that are not necessary for predicting the desired out- put. An important question in the fields of machine learning, knowledge discovery, computer vision and pattern recogni- tion is how to extract a small number of good features. A common way to attempt to resolve this problem is to use di- mensionality reduction techniques. Two of the most popular ∗ The work was supported in part by the U.S. National Science Foundation NSF IIS-03-08215/IIS-05-13678, Specialized Research Fund for the Doctoral Program of Higher Education of China (No. 20030335083) and National Natural Science Foundation of China (No. 60633070). Any opinions, findings, and conclusions or recom- mendations expressed in this paper are those of the authors and do not necessarily reﬂect the views of the funding agencies. techniques for this purpose are Principal Component Analy- sis (PCA) and Linear Discriminant Analysis (LDA) [Duda et al. , 2000]. PCA is an unsupervised method. It aims to project the data along the direction of maximal variance. LDA is supervised. It searches for the project axes on which the data points of different classes are far from each other while requiring data points of the same class to be close to each other. Both of them are spectral methods, i.e., methods based on eigenvalue decomposition of either the covariance matrix for PCA or the scatter matrices (within-class scatter matrix and between- class scatter matrix) for LDA. Intrinsically, these methodsclass scatter matrix) for LDA....
View Full Document
This note was uploaded on 02/12/2010 for the course COMPUTER S 10586 taught by Professor Jilinwang during the Fall '09 term at Zhejiang University.
- Fall '09
- Computer Science