Lecture 5. A Multiple Linear Regression Model
Summary of previous lectures on MLR
Here we consider the case
p
explanatory variables
Y
t
=
β
0
+
β
1
X
t,
1
+
...
+
β
p
X
t,p
+
²
t
.
(5.1)
This can be expressed more compactly in matrix notation as
Y
=
Xβ
+
²,
(5.2)
where
Y
= (
Y
1
,...,Y
n
)
T
,
β
= (
β
0
,...,β
p
)
T
,
²
= (
²
1
,...,²
n
)
T
;
X
is an
n
×
(
p
+ 1)design
matrix
X
=
1
X
1
,
1
... X
1
,p
1
X
2
,
1
... X
2
,p
.
.
.
.
.
.
.
.
.
.
.
.
1
X
n,
1
... X
1
,p
Here the historical data for the dependent variable consist of the observations
Y
1
,...,Y
n
; the
historical data for the independent variable consist of the observations in the matrix
X
.
Minimizing SSE = (
Y

X
β
)
T
(
Y

X
β
) yields the least squares solutions
ˆ
β
= (
X
T
X
)

1
X
T
Y
(5.3)
for nonsingular
X
T
X
.
The forecast of a future value
Y
k
is then given by
ˆ
Y
k
=
x
T
k
ˆ
β,
(5.4)
where
x
k
is a (column) vector of regressors’ values for the
k
th case.
Under the OLS assumptions (recall them!) we obtain
Var
‡
ˆ
β
j
·
=
σ
2
‡
X
T
X
·

1
jj
,
(5.5)
where the
‡
X
T
X
·

1
jj
denotes the
j
th diagonal element of
‡
X
T
X
·

1
.
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentThis yields
s
.
e
.
‡
ˆ
β
j
·
=
σ
*
r
(
X
T
X
)

1
jj
.
(5.6)
Note that here the degrees of freedom (d.f.) are
n

(
p
+ 1) =
n

p

1. (In fact, since the
number of estimated parameters, i.e.
p
parameters for the independent variables and one
for the intercept, is
p
+ 1.)
Under the OLS assumptions, a 100(1

α
)%
C.I.
for the parameter
β
j
,j
= 0
,
1
,...,p
, is
given by
ˆ
β
j
±
t
α/
2
,n

(
p
+1)
s.e.
‡
ˆ
β
j
·
=
ˆ
β
j
±
t
α/
2
,n

(
p
+1)
s
r
(
X
T
X
)

1
jj
,
(5.7)
where
s
=
SSE/
(
n

p

1).
Typically s
.
e
.
(
β
*
j
) is read directly oﬀ the
R
output and so the calculations 5.7 should not be
done manually.
Under the OLS assumptions it can be shown that
Var
‡
ˆ
Y
t

Y
t
·
=
σ
2
±
x
T
t
‡
X
T
X
·

1
x
t
+ 1
¶
,
(5.8)
yielding a 100(1

α
)%
P.I.
for
Y
t
:
x
T
t
ˆ
β
±
t
α/
2
,n

(
p
+1)
s
q
x
T
t
(
X
T
X
)

1
x
t
+ 1
.
(5.9)
We usually never perform these calculations by hand and will use the corresponding software
functions, e.g.
predict.lm
in R (see the example for the code template).
What else can you get from the regression output?
This is the end of the preview.
Sign up
to
access the rest of the document.
 Fall '08
 YuliaGel
 Linear Regression, Regression Analysis, Albuquerque, MLR residuals

Click to edit the document details