This preview shows page 1. Sign up to view the full content.
Unformatted text preview: (X).
Thus, it follows that C (W ) = C (X). Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 511 8 / 27 Estimation of E(y)
A fundamental goal of linear model analysis is to estimate E(y).
We could, of course, use y to estimate E(y).
y is obviously an unbiased estimator of E(y), but it is often not a
very sensible estimator.
For example, suppose
y1
y2 = 1
1 µ+ Should we estimate E(y) = Copyright c 2012 Dan Nettleton (Iowa State University) 1 , and we observe y = 2 µ
µ by y = 6.1
2.3 . 6.1
?
2.3 Statistics 511 9 / 27 Estimation of E(y)
The GaussMarkov linear models says that E(y) ∈ C (X), so we
should use that information when estimating E(y).
Consider estimating E(y) by the point in C (X) that is closest to y
(as measured by the usual Euclidean distance).
This unique point is called the orthogonal projection of y onto C (X)
and denoted by ˆ (although it could be argued that E(y) might be
y
better notation).
By deﬁnition, y − ˆ = minz∈C (X) y − z, where a ≡
y Copyright c 2012 Dan Nettleton (Iowa State University) n
2
i=1 ai . Statistics 511 10 / 27 Orthogonal Projection Matrices
In Homework Assignment 2, we will formally prove the following:
1
∀ y ∈ IRn , ˆ = PX y, where PX is a unique n × n matrix known as an
y
orthogonal projection matrix.
2 PX is idempotent: PX PX = PX . 3 PX is symmetric: PX = PX . 4 PX X = X and X PX = X . 5 PX = X(X X)− X , where (X X)− is any generalized inverse of X X. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 511 11 / 27 Why Does PX X = X? PX X = PX [x1 , . . . , xp ]
= [ P X x1 , . . . , P X xp ]
= [ x1 , . . . , xp ]
= X. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 511 12 / 27 Generalized Inverses
G is a generalized inverse of a matrix A if AGA = A.
We usually denote a generalized inverse of A by A− .
If A is nonsingular, i.e., if A−1 exists, then A−1 is the one and only
generalized inverse of A.
AA−1 A = AI = IA = A
If A is singular, i.e., if A−1 does not...
View
Full
Document
This document was uploaded on 03/27/2014.
 Spring '14
 Statistics

Click to edit the document details