15419lec6.pdf

# 15419lec6.pdf - Future reality Statistics 154 Spring 2019...

• 52

This preview shows 1 out of 11 pages.

Statistics 154, Spring 2019 Modern Statistical Prediction and Machine Learning Lecture 6: CV for PCA, NMF, and hierarchical Instructor: Bin Yu ( [email protected] ); office hours: Tu: 9:30-10:30 am; Wed : 1:30-2:30 pm (change) office: 409 Evans GSIs: Yuansi Chen (Mon: 10-12; 4-6); Raaz Dwivedi (Mon: 12-2; 2-4) [email protected] ; [email protected] (Yuansi: Tuesday 1-3; Raaz: Monday 10:30-11:30, Thurs. 9:30-10:30) 1 Future reality

Subscribe to view the full document.

of Machine Learning/Statistics Taxonomy Reinforcement & Bandit Learning Indirect (reward) Unlabeled Data Unsupervised Learning Dimensionality Reduction Clustering Labeled Data Supervised Learning Regression Classification Thanks to J. Gonzalez
How do we visualize beyond 4 dimensions? Ames data has 80 features We can at most “look” at 3-dim data (or 4-dim with movies of data) Visual cortex takes up about 30% of our brain so we want to see projections of data into low dim spaces (2d or 3d, for example) How to project? Random projection? Or?

Subscribe to view the full document.

2/8/19 4 Dimensionality reduction is needed for Visualization Fast computation Smaller storage and faster communication One form for regularization: Simpler models (to reduce variance of estimation) Dimension reduction for more interpretable results of high-dim data
The central idea of PCA is : to reduce the dimensionality of a data set that has a large number of interrelated variables, while retaining as much as possible of the variation present in the data set. This is achieved by transforming to a new set of variables (PCs) which are uncorrelated (orthogonal) , and which are ordered so that (hopefully) the first few retain most of the variation present in all of the original variables. Jolliffe, Principal Component Analysis, 2nd edition PCA: Core Idea

Subscribe to view the full document.

PCA recap, illustrated in p=2 dim 2/8/19 6 p d 1 u 1 p d 2 u 2 G = X 0 X = UDU T is the data matrix in the new coordinate system spanned by and u 1 u 2 Data matrix U = ( u 1 , ..., u p ) X n p The ith row of Z is called the “principal components” of the ith data point is the jth PC direction, unit vector in u j Z n p = XU R p (eigen decomposition) R p D = diag ( d 1 , . . . , d p )
2/8/19 7 As we have shown, an equivalent way to define principal components is to first find the linear combination (loading) with norm 1 of the columns of X such that its variance is maximized. Such a loading forms the first principal component vector. Similarly, we can define the second principal component vector that is the orthogonal to the first and has norm 1 and maximize the variance of the resulted linear combination of the columns of X. PCA: maximizing variance

Subscribe to view the full document.

PCA in 3-dim to 2-dim: 1-dim lost when is it not a bad idea? 2/8/19 8
Eigen values matter 2/8/19 9

Subscribe to view the full document.

2/8/19 10 An important property of the eigen values is that they add up to the trace of the sample covariance matrix or the variance of the original column vectors of X (with columns centered).
You've reached the end of this preview.
• Fall '17
• Singular value decomposition, eigenface

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern