This preview shows page 1. Sign up to view the full content.
Unformatted text preview: and only if b∗ is a
solution to the normal equations:
X Xb = X y.
If X X is nonsingular, multiplying both sides of the normal
equations by (X X)−1 shows that the only solution to the normal
equations is b∗ = (X X)−1 X y.
If X X is singular, there are inﬁnitely many solutions that include
(X X)− X y for all choices of generalized inverse of X X.
X X[(X X)− X y] = X [X(X X)− X ]y = X PX y = X y
ˆ
Henceforth, we will use β to denote any solution to the normal
equations.
opyright c 2012 Dan Nettleton (Iowa State University) Statistics 511 26 / 27 Ordinary Least Squares Estimator of E(y) = Xβ
ˆ
ˆ
ˆ
We call Xβ = PX Xβ = X(X X)− X Xβ = X(X X)− X y = PX y = ˆ the
y
OLS estimator of E(y) = Xβ .
ˆ
It might be more appropriate to use Xβ rather than Xβ to denote
our estimator because we are estimating Xβ rather than
premultiplying an estimator of β by X.
As we shall soon see, it does not make sense to estimate β when
X X is singular.
However, it does make sense to estimate E(y) = Xβ whether X X
is singular or nonsingular. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 511 27 / 27...
View
Full
Document
This document was uploaded on 03/27/2014.
 Spring '14
 Statistics

Click to edit the document details