Geometry of the GaussMarkov Linear Model
Reminder from the last section of the notes:
y
=
X
β
+
We saw two possible
X
matrices for the ttest.
This section focuses on the Q: does it matter which
X
we use?
Important pieces of information for what follows:
X
is sometimes referred to as the
design matrix
. It is an
n
×
p
matrix of constants with columns corresponding to explanatory
variables.
β
is an unknown parameter vector in
IR
p
.
Copyright
c
2011 Dept. of Statistics (Iowa State University)
Statistics 511
1 / 32
The Column Space of the Design Matrix
X
β
is a
linear combination
of the columns of
X
:
X
β
= [
x
1
, . . . ,
x
p
]
⎡
⎢
⎣
β
1
.
.
.
β
p
⎤
⎥
⎦
=
β
1
x
1
+
· · ·
+
β
p
x
p
.
The set of all possible linear combinations of the columns of
X
is
called the
column space
of
X
and is denoted by
C
(
X
) =
{
Xa
:
a
∈
IR
p
}
.
The GaussMarkov linear model says
y
is a random vector whose:
mean is in the column space of
X
and
whose variance is
σ
2
I
for some positive real number
σ
2
, i.e.,
E
(
y
)
∈ C
(
X
)
and
Var
(
y
) =
σ
2
I
, σ
2
∈
IR
+
.
Copyright
c
2011 Dept. of Statistics (Iowa State University)
Statistics 511
2 / 32
An Example Column Space
X
=
1
1
=
⇒ C
(
X
)
=
{
Xa
:
a
∈
IR
}
=
1
1
a
1
:
a
1
∈
IR
=
a
1
a
1
:
a
1
∈
IR
What does this column space “look like”?
Copyright
c
2011 Dept. of Statistics (Iowa State University)
Statistics 511
3 / 32
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.0
0.5
1.0
X1
X2
Copyright
c
2011 Dept. of Statistics (Iowa State University)
Statistics 511
4 / 32
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Another Example Column Space
X
=
⎡
⎢
⎢
⎣
1
0
1
0
0
1
0
1
⎤
⎥
⎥
⎦
=
⇒ C
(
X
)
=
⎧
⎪
⎪
⎨
⎪
⎪
⎩
⎡
⎢
⎢
⎣
1
0
1
0
0
1
0
1
⎤
⎥
⎥
⎦
a
1
a
2
:
a
∈
IR
2
⎫
⎪
⎪
⎬
⎪
⎪
⎭
=
⎧
⎪
⎪
⎨
⎪
⎪
⎩
a
1
⎡
⎢
⎢
⎣
1
1
0
0
⎤
⎥
⎥
⎦
+
a
2
⎡
⎢
⎢
⎣
0
0
1
1
⎤
⎥
⎥
⎦
:
a
1
,
a
2
∈
IR
⎫
⎪
⎪
⎬
⎪
⎪
⎭
=
⎧
⎪
⎪
⎨
⎪
⎪
⎩
⎡
⎢
⎢
⎣
a
1
a
1
a
2
a
2
⎤
⎥
⎥
⎦
:
a
1
,
a
2
∈
IR
⎫
⎪
⎪
⎬
⎪
⎪
⎭
What is this column space?
A plane “living” in
IR
4
.
Copyright
c
2011 Dept. of Statistics (Iowa State University)
Statistics 511
5 / 32
A Third Column Space Example
X
2
=
⎡
⎢
⎢
⎣
1
1
0
1
1
0
1
0
1
1
0
1
⎤
⎥
⎥
⎦
x
∈ C
(
X
2
)
=
⇒
x
=
X
2
a
for some
a
∈
IR
3
=
⇒
x
=
a
1
⎡
⎢
⎢
⎣
1
1
1
1
⎤
⎥
⎥
⎦
+
a
2
⎡
⎢
⎢
⎣
1
1
0
0
⎤
⎥
⎥
⎦
+
a
3
⎡
⎢
⎢
⎣
0
0
1
1
⎤
⎥
⎥
⎦
for some
a
∈
IR
3
=
⇒
x
=
⎡
⎢
⎢
⎣
a
1
+
a
2
a
1
+
a
2
a
1
+
a
3
a
1
+
a
3
⎤
⎥
⎥
⎦
=
⎡
⎢
⎢
⎣
b
1
b
1
b
2
b
2
⎤
⎥
⎥
⎦
for some
b
1
,
b
2
∈
IR
This is also a plane in
R
4
. Is it the same plane?
Copyright
c
2011 Dept. of Statistics (Iowa State University)
Statistics 511
6 / 32
Proving that two column spaces,
C
(
X
1
)
and
C
(
X
2
)
, are
the same
Concept:
If you can start with
x
∈ C
(
X
1
)
and derive
x
=
X
2
b
,
this implies
x
∈ C
(
X
2
)
and
C
(
X
1
)
⊆ C
(
X
2
)
.
N.B. Not (at least yet)
C
(
X
1
) =
C
(
X
2
)
because there may be some
b
for which
X
2
b
is not in
C
(
X
1
)
.
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '08
 Staff
 Linear Algebra, Dept. of Statistics

Click to edit the document details