STAT Principal Components Analysis

# 0 40 30 40 x2 60 120 120 100 plotthemajorandminoraxes

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: 1' c = xΣx = e 1x + L + e px λ1 % % λp % % % %% 2 ' 2 where the are the principal components of x. e 'i x ~ %% ' yi = eix Setting and substituting into the previous %% expression yields 12 12 c= y1 + L + yp λ1 λp 2 which defines an ellipsoid (note that λ i > 0 ∀ i) in a coordinate system with axes y1,…,yp lying in the directions of ~ ~ e1,…,ep, respectively. ~ ~ The major axis lies in the direction determined by the eigenvector ei associated with the largest eigenvalue λ i ­ the ~ remaining minor axes lie in the directions determined by the other eigenvectors. Example: For the principal components derived from the following population of four observations made on three random variables X1, X2, and X3: X1 1.0 4.0 3.0 4.0 X2 6.0 12.0 12.0 10.0 plot the major and minor axes. X3 9.0 10.0 15.0 12.0 We will need the centroid µ : 3.0 μ = 10.0 % 11.5 The direction of the major axis is given by e' X = 0.2910381X 1 + 0.7342493X 2 + 0.6133309X 3 1 %% while the directions of the two minor axis are given by e' X = 0.4150386X 1 + 0.4807165X 2 - 0.7724340X 3 2 %% e' X = 0.8619976X 1 - 0.4793640X 2 + 0.1648350X 3 3 %% We first graph the centroid: X2 3.0,10.0,15.0 X3 X1 …then use the first eigenvector to find a second point on the first principal axis: X2 Y1 X1 X3 The line connecting these two points is the Y1 axis. …then do the same thing with the second eigenvector: Y2 X2 Y1 X1 X3 The line connecting these two points is the Y2 axis. …and do the same thing with the third eigenvector: Y2 X2 Y1 X1 Y3 X3 The line connecting these two points is the Y3 axis. What we have done is a rotation… Y2 X2 Y1 X1 Y3 X3 and a translation in p = 3 dimensions. Y 2 Y2 X2 Note that the rotated axes remain orthogonal! Y1 X1 Y3 X3 Note that we can also construct principal components for the standardized variables Zi: X i -μ i Zi = , i = 1, K , p σii which in matrix notation is ( ) ( X -μ ) %% Z= V % % 12 -1 where V1/2 is the diagonal standard deviation matrix. ~ Obviously E ( Z) = 0 % % -1 -1 12 12 Cov ( Z) = VΣ V =ρ % % %% () () This suggests that the principal components for the standardized variables Zi may be obtained from the eigenvectors of the correlation matrix ρ ! The operations are ~ analogous to those used in conjunction with the covariance matrix. We can show that, for random vector Z of standardized ~ variables with covariance matrix ρ and eigenvalues λ 1 ≥ λ 2 ~ th ≥ L ≥ λ p ≥ 0, the i principal component is given by ' i ' i Yi = e Z = e %% % ( V ) ( X -μ ), i % % % 12 -1 = 1, K p , Note again that the principal components are not unique if some eigenvalues are equal. We can also show for random vector Z with covariance ~ matrix ρ and eigenvalue­eigenvector pairs (λ 1 , e1), …, (λ p , ep) ~ ~ where λ 1 ≥ λ 2 ≥ L ≥ λ p, ~ p ∑ i =1 Var ( Zi ) =λ 1+ L+ λ p = p ( ∑Var Y ) = i p i =1 and we can again assess how well a subset of the principal components Yi summarizes the original random variables Xi by using proportion of total λk population variance due to the kth p principal...
View Full Document

## This note was uploaded on 04/08/2014 for the course STAT 4503 taught by Professor Majidmojirsheibani during the Spring '09 term at Carleton CA.

Ask a homework question - tutors are online