This preview shows page 1. Sign up to view the full content.
Unformatted text preview: 1'
c = xΣx =
e 1x + L +
λ1 % %
λp % %
2 ' 2 where the are the principal components of x.
e 'i x ~
yi = eix
Setting and substituting into the previous %%
expression yields 12
y1 + L +
2 which defines an ellipsoid (note that λ i > 0 ∀ i) in a coordinate system with axes y1,…,yp lying in the directions of ~
~ The major axis lies in the direction determined by the eigenvector ei associated with the largest eigenvalue λ i the ~
remaining minor axes lie in the directions determined by the other eigenvectors. Example: For the principal components derived from the following population of four observations made on three random variables X1, X2, and X3: X1
10.0 plot the major and minor axes. X3
12.0 We will need the centroid µ : 3.0 μ = 10.0
% 11.5 The direction of the major axis is given by e' X = 0.2910381X 1 + 0.7342493X 2 + 0.6133309X 3
while the directions of the two minor axis are given by e' X = 0.4150386X 1 + 0.4807165X 2 - 0.7724340X 3
e' X = 0.8619976X 1 - 0.4793640X 2 + 0.1648350X 3
%% We first graph the centroid:
X2 3.0,10.0,15.0 X3 X1 …then use the first eigenvector to find a second point on the first principal axis:
X1 X3 The line connecting these two points is the Y1 axis. …then do the same thing with the second eigenvector:
X1 X3 The line connecting these two points is the Y2 axis. …and do the same thing with the third eigenvector:
X3 The line connecting these two points is the Y3 axis. What we have done is a rotation…
Y2 X2 Y1
X1 Y3 X3 and a translation in p = 3 dimensions.
Y 2 Y2
X2 Note that the rotated
axes remain orthogonal!
X3 Note that we can also construct principal components for the standardized variables Zi: X i -μ i
, i = 1, K , p
which in matrix notation is ( ) ( X -μ )
%% Z= V
% 12 -1 where V1/2 is the diagonal standard deviation matrix.
~ Obviously E ( Z) = 0
Cov ( Z) = VΣ V
%% () () This suggests that the principal components for the standardized variables Zi may be obtained from the eigenvectors of the correlation matrix ρ ! The operations are ~
analogous to those used in conjunction with the covariance matrix.
We can show that, for random vector Z of standardized ~
variables with covariance matrix ρ and eigenvalues λ 1 ≥ λ 2 ~
≥ L ≥ λ p ≥ 0, the i principal component is given by
i Yi = e Z = e
% ( V ) ( X -μ ), i
12 -1 = 1, K p
, Note again that the principal components are not unique if some eigenvalues are equal. We can also show for random vector Z with covariance ~
matrix ρ and eigenvalueeigenvector pairs (λ 1 , e1), …, (λ p , ep) ~
where λ 1 ≥ λ 2 ≥ L ≥ λ p,
p ∑ i =1 Var ( Zi ) =λ 1+ L+ λ p = p (
∑Var Y ) =
i p i =1 and we can again assess how well a subset of the principal components Yi summarizes the original random variables Xi by using
proportion of total
due to the kth
View Full Document