This preview shows pages 1–3. Sign up to view the full content.
ISYE6414
Summer 2010
Lecture 6
The Linear Model: Multiple Regression
Inference and Diagnostics
Dr. Kobi Abayomi
July 6, 2010
1 Introduction
Inference on the parameters of multiple regression follow arguments similar to those in simple
regression. Diagnostics, however, can be a bit more involved.
2 Inference
The least squares and MLE of
ˆ
β
are unbiased:
E
(
ˆ
β
) =
β
...just like in simple regression.
This is the variancecovariance matrix Σ
β
=
σ
2
(
X
T
X
)

1
; the full distribution for the estimators
ˆ
β
is:
ˆ
β
∼
N
(
β,
Σ
β
)
and if we replace Σ
β
=
σ
2
(
X
T
X
)

1
with an estimate
ˆ
Σ
β
= ˆ
σ
2
(
X
T
X
)

1
which is
ˆ
Σ
β
=
MSE
·
(
X
T
X
)

1
, then full joint distribution for the estimators is the multivariate tdistribution.
2.1 Joint and Univariate inference on
β
coeﬃcients
Each individual univariate hypothesis test of
H
0
:
β
j
= 0 vs.
H
a
:
β
j
6
= 0 uses a
t

statistic:
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Documentt
=
ˆ
β
j
q
ˆ
V ar
(
β
j
)
If we desire inference on
g
parameters from
β
1
,...,β
k
, we can appeal to univariate tests with a
Bonferroni correction. We compute each CI or hypothesis test separately, using the adjusted Type
I error level
α
2
·
g
at each univariate. This yields overall Type I error level
α
.
Or use the joint procedure outlined in section 6.1 of Lecture 4.
2.2 Inference on Mean Response and Prediction
Call
X
h
= (1
,X
h,
1
,...,X
h,k
), a vector of observed predictors. Under the model
E
(
Y
h
) =
X
T
h
β
and
the ﬁtted values
ˆ
Y
h
=
X
T
h
ˆ
β
. Of course, this ﬁtted value the estimate of the expected value, or:
ˆ
Y
h
=
ˆ
E
(
Y
h
).
We know this estimator is unbiased (
E
(
ˆ
Y
h
) =
X
h
T
β
) and we can compute its variance (
V ar
(
ˆ
Y
h
) =
V ar
(
X
h
ˆ
β
) =
σ
·
X
T
h
(
X
T
X
)

1
X
h
=
X
T
h
Σ
ˆ
β
X
h
); its distribution is also the distribution of the
residuals:
ˆ
Y
h
=
N
(
X
T
h
β,
X
T
h
Σ
ˆ
β
X
h
)
If we have to use an estimate of the variance, i.e. we replace
σ
2
with ˆ
σ
2
=
MSE
then
ˆ
V ar
(
ˆ
Y
h
) =
MSE
·
X
T
h
(
X
T
X
)

1
X
h
=
X
T
h
ˆ
Σ
ˆ
This is the end of the preview. Sign up
to
access the rest of the document.
 Fall '08
 Staff

Click to edit the document details