This preview shows pages 1–2. Sign up to view the full content.
This preview has intentionally blurred sections. Sign up to view the full version.View Full Document
Unformatted text preview: A Tutorial on Principal Component Analysis Jonathon Shlens * Systems Neurobiology Laboratory, Salk Insitute for Biological Studies La Jolla, CA 92037 and Institute for Nonlinear Science, University of California, San Diego La Jolla, CA 92093-0402 (Dated: December 10, 2005; Version 2) Principal component analysis (PCA) is a mainstay of modern data analysis - a black box that is widely used but poorly understood. The goal of this paper is to dispel the magic behind this black box. This tutorial focuses on building a solid intuition for how and why principal component analysis works; furthermore, it crystallizes this knowledge by deriving from simple intuitions, the mathematics behind PCA . This tutorial does not shy away from explaining the ideas informally, nor does it shy away from the mathematics. The hope is that by addressing both aspects, readers of all levels will be able to gain a better understanding of PCA as well as the when, the how and the why of applying this technique. I. INTRODUCTION Principal component analysis ( PCA ) has been called one of the most valuable results from applied linear al- gebra. PCA is used abundantly in all forms of analysis - from neuroscience to computer graphics - because it is a simple, non-parametric method of extracting relevant in- formation from confusing data sets. With minimal addi- tional effort PCA provides a roadmap for how to reduce a complex data set to a lower dimension to reveal the sometimes hidden, simplified structure that often under- lie it. The goal of this tutorial is to provide both an intuitive feel for PCA , and a thorough discussion of this topic. We will begin with a simple example and provide an intu- itive explanation of the goal of PCA . We will continue by adding mathematical rigor to place it within the frame- work of linear algebra to provide an explicit solution. We will see how and why PCA is intimately related to the mathematical technique of singular value decomposition ( SVD ). This understanding will lead us to a prescription for how to apply PCA in the real world. We will discuss both the assumptions behind this technique as well as possible extensions to overcome these limitations. The discussion and explanations in this paper are infor- mal in the spirit of a tutorial. The goal of this paper is to educate . Occasionally, rigorous mathematical proofs are necessary although relegated to the Appendix. Although not as vital to the tutorial, the proofs are presented for the adventurous reader who desires a more complete un- derstanding of the math. The only assumption is that the reader has a working knowledge of linear algebra. Please feel free to contact me with any suggestions, corrections or comments....
View Full Document
- Spring '08