2.1 Sampling Distributions and Inference for the parameters
c
circlecopyrt
HYON-JUNG KIM, 2017
2.1
Sampling Distributions and Inference for the parameters
Sampling Distribution of
ˆ
β
Since
ˆ
β
is a linear combination of
Y
,
ˆ
β
has a multivariate Normal distribution as
ǫ
and
Y
.
ˆ
β
∼
MVN (
β
, σ
2
(
X
′
X
)
−
1
)
.
Now (
X
′
X
)
−
1
is a
p
×
p
matrix. So is Var(
ˆ
β
), the variance-covariance matrix of
ˆ
β
.
IMPLICATIONS:
(a)
E
(
ˆ
β
k
) =
β
k
, for
k
= 0
,
1
, ..., p
−
1
.
; unbiased.
(b) Var(
ˆ
β
k
) = (
X
′
X
)
−
1
kk
σ
2
, for
k
= 0
,
1
, ..., p
−
1
.
The value (
X
′
X
)
−
1
kk
represents the
k
th diagonal element of the (
X
′
X
)
−
1
matrix.
(c) Cov(
ˆ
β
k
,
ˆ
β
l
)) = (
X
′
X
)
−
1
kl
σ
2
, for
k
negationslash
=
l
.
The value (
X
′
X
)
−
1
kl
is the entry in the
k
th row and
l
th column of the (
X
′
X
)
−
1
matrix.
(d) Marginally,
ˆ
β
k
∼
N
(
β
k
,
(
X
′
X
)
−
1
kk
σ
2
)
,
for
k
= 0
,
1
, ..., p
−
1
.
(e) E(MSE) =
σ
2
where
SSE
σ
2
=
MSE(
n
−
p
)
σ
2
∼
χ
2
n
−
p
.
CONFIDENCE INTERVALS:
The 100(1
−
α
)% confidence interval for
β
k
is
ˆ
β
k
±
t
1
−
α
2
,n
−
p
s(
ˆ
β
k
)
,
k
= 0
, . . . , p
−
1 where
s(
ˆ
β
k
) is the standard error of
ˆ
β
k
, i.e.,
radicalBig
(
X
′
X
)
−
1
kk
MSE
.
Since
(
ˆ
β
−
β
)
′
X
′
X
(
ˆ
β
−
β
)
σ
2
∼
χ
2
p
is independent of MSE
,
(
ˆ
β
−
β
)
′
X
′
X
(
ˆ
β
−
β
)
p
MSE
∼
χ
2
p
/p
χ
2
n
−
p
/
(
n
−
p
)
≡
F
p,n
−
p
Thus, a 100(1
−
α
)%
(simultaneous) confidence region
for
β
can be formed as
(
ˆ
β
−
β
)
′
X
′
X
(
ˆ
β
−
β
)
≤
p
MSE
F
1
−
α,
(
p,n
−
p
)
.
These regions are ellisoidally shaped and cannot be easily visualized due to high-dimensionality.
PAGE 24