ISYE6414
Summer 2010
Lecture 6
The Linear Model: Multiple Regression
Inference and Diagnostics
Dr. Kobi Abayomi
July 6, 2010
1 Introduction
Inference on the parameters of multiple regression follow arguments similar to those in simple
regression. Diagnostics, however, can be a bit more involved.
2 Inference
The least squares and MLE of
ˆ
β
are unbiased:
E
(
ˆ
β
) =
β
...just like in simple regression.
This is the variancecovariance matrix Σ
β
=
σ
2
(
X
T
X
)

1
; the full distribution for the estimators
ˆ
β
is:
ˆ
β
∼
N
(
β,
Σ
β
)
and if we replace Σ
β
=
σ
2
(
X
T
X
)

1
with an estimate
ˆ
Σ
β
= ˆ
σ
2
(
X
T
X
)

1
which is
ˆ
Σ
β
=
MSE
·
(
X
T
X
)

1
, then full joint distribution for the estimators is the multivariate tdistribution.
2.1 Joint and Univariate inference on
β
coeﬃcients
Each individual univariate hypothesis test of
H
0
:
β
j
= 0 vs.
H
a
:
β
j
6
= 0 uses a
t

statistic:
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Documentt
=
ˆ
β
j
q
ˆ
V ar
(
β
j
)
If we desire inference on
g
parameters from
β
1
,...,β
k
, we can appeal to univariate tests with a
Bonferroni correction. We compute each CI or hypothesis test separately, using the adjusted Type
I error level
α
2
·
g
at each univariate. This yields overall Type I error level
α
.
Or use the joint procedure outlined in section 6.1 of Lecture 4.
2.2 Inference on Mean Response and Prediction
Call
X
h
= (1
,X
h,
1
,...,X
h,k
), a vector of observed predictors. Under the model
E
(
Y
h
) =
X
T
h
β
and
the ﬁtted values
ˆ
Y
h
=
X
T
h
ˆ
β
. Of course, this ﬁtted value the estimate of the expected value, or:
ˆ
Y
h
=
ˆ
E
(
Y
h
).
We know this estimator is unbiased (
E
(
ˆ
Y
h
) =
X
h
T
β
) and we can compute its variance (
V ar
(
ˆ
Y
h
) =
V ar
(
X
h
ˆ
β
) =
σ
·
X
T
h
(
X
T
X
)

1
X
h
=
X
T
h
Σ
ˆ
β
X
h
); its distribution is also the distribution of the
residuals:
ˆ
Y
h
=
N
(
X
T
h
β,
X
T
h
Σ
ˆ
β
X
h
)
If we have to use an estimate of the variance, i.e. we replace
σ
2
with ˆ
σ
2
=
MSE
then
ˆ
V ar
(
ˆ
Y
h
) =
MSE
·
X
T
h
(
X
T
X
)

1
X
h
=
X
T
h
ˆ
Σ
ˆ
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 Staff
 Normal Distribution, Regression Analysis, Variance, residuals, Xt, Σβ Xh

Click to edit the document details