Unformatted text preview: 1'
c = xΣx =
e 1x + L +
e px
λ1 % %
λp % %
% %%
2 ' 2 where the are the principal components of x.
e 'i x ~
%% '
yi = eix
Setting and substituting into the previous %%
expression yields 12
12
c=
y1 + L +
yp
λ1
λp
2 which defines an ellipsoid (note that λ i > 0 ∀ i) in a coordinate system with axes y1,…,yp lying in the directions of ~
~
e1,…,ep, respectively.
~
~ The major axis lies in the direction determined by the eigenvector ei associated with the largest eigenvalue λ i the ~
remaining minor axes lie in the directions determined by the other eigenvectors. Example: For the principal components derived from the following population of four observations made on three random variables X1, X2, and X3: X1
1.0
4.0
3.0
4.0 X2
6.0
12.0
12.0
10.0 plot the major and minor axes. X3
9.0
10.0
15.0
12.0 We will need the centroid µ : 3.0 μ = 10.0
% 11.5 The direction of the major axis is given by e' X = 0.2910381X 1 + 0.7342493X 2 + 0.6133309X 3
1
%%
while the directions of the two minor axis are given by e' X = 0.4150386X 1 + 0.4807165X 2  0.7724340X 3
2
%%
e' X = 0.8619976X 1  0.4793640X 2 + 0.1648350X 3
3
%% We first graph the centroid:
X2 3.0,10.0,15.0 X3 X1 …then use the first eigenvector to find a second point on the first principal axis:
X2 Y1
X1 X3 The line connecting these two points is the Y1 axis. …then do the same thing with the second eigenvector:
Y2
X2 Y1
X1 X3 The line connecting these two points is the Y2 axis. …and do the same thing with the third eigenvector:
Y2
X2 Y1
X1 Y3
X3 The line connecting these two points is the Y3 axis. What we have done is a rotation…
Y2 X2 Y1
X1 Y3 X3 and a translation in p = 3 dimensions.
Y 2 Y2
X2 Note that the rotated
axes remain orthogonal!
Y1
X1 Y3
X3 Note that we can also construct principal components for the standardized variables Zi: X i μ i
Zi =
, i = 1, K , p
σii
which in matrix notation is ( ) ( X μ )
%% Z= V
%
% 12 1 where V1/2 is the diagonal standard deviation matrix.
~ Obviously E ( Z) = 0
%
%
1
1
12
12
Cov ( Z) = VΣ V
=ρ
%
%
%% () () This suggests that the principal components for the standardized variables Zi may be obtained from the eigenvectors of the correlation matrix ρ ! The operations are ~
analogous to those used in conjunction with the covariance matrix.
We can show that, for random vector Z of standardized ~
variables with covariance matrix ρ and eigenvalues λ 1 ≥ λ 2 ~
th
≥ L ≥ λ p ≥ 0, the i principal component is given by
'
i '
i Yi = e Z = e
%%
% ( V ) ( X μ ), i
%
%
%
12 1 = 1, K p
, Note again that the principal components are not unique if some eigenvalues are equal. We can also show for random vector Z with covariance ~
matrix ρ and eigenvalueeigenvector pairs (λ 1 , e1), …, (λ p , ep) ~
~
where λ 1 ≥ λ 2 ≥ L ≥ λ p,
~
p ∑ i =1 Var ( Zi ) =λ 1+ L+ λ p = p (
∑Var Y ) =
i p i =1 and we can again assess how well a subset of the principal components Yi summarizes the original random variables Xi by using
proportion of total
λk
population variance
due to the kth
p
principal...
View
Full
Document
This note was uploaded on 04/08/2014 for the course STAT 4503 taught by Professor Majidmojirsheibani during the Spring '09 term at Carleton CA.
 Spring '09
 MAJIDMOJIRSHEIBANI
 Covariance, Variance

Click to edit the document details