STAT 371 F18 Chap 4 part 1.pdf - Chapter 4 Multiple Linear...

Chapter 4
Multiple Linear Regression

• We now consider the general linear model:
? = 𝛽
0
+ 𝛽
1
?
1
+ 𝛽
2
?
2
+ ⋯ + 𝛽
𝑝
?
𝑝
+ 𝜀
• We will discuss how to estimate the model parameters,
𝛽
0
, 𝛽
1
, 𝛽
2
, … , 𝛽
𝑝
, and how to test various hypotheses about them.
• To start, suppose we have information on n cases, or subjects
𝑖 = 1, 2, … , 𝑛
• Let ?
?
be the observed response value for subject 𝑖 and let
?
?1
, ?
?2
, … , ?
?𝑝
be the values on the explanatory or predictor
variables.

• Recall, the values of the p predictor variables are treated as fixed
constants; however, the responses are subject to variability.
• Hence the model for subject 𝒊 is written as
?
?
= 𝛽
0
+ 𝛽
1
?
?1
+ 𝛽
2
?
?2
+ ⋯ + 𝛽
𝑝
?
?𝑝
+ 𝜀
?
• Also assume, as before that 𝜀
?
is a random variable having a mean of
0 and constant variance, 𝜎
2
. We suppose that they are normally
distributed and that errors for different cases ( 𝜀
?
, 𝜀
?
) are assumed
independent.
• Recall these points also imply that the responses ?
1
, ?
2
, … , ?
𝑛
are
independent normal random variables with mean 𝐸 ?
?
= 𝜇
?
and
variance 𝑉𝑎𝑟 ?
?
= 𝜎
2
.

This model can be expressed in vector form, we write
?
?
= ?
?
′𝜷 + 𝜀
?
where
?
?
′
=
1
?
?1
⋯
?
?𝑝
and 𝜷 =
𝛽
0
𝛽
1
⋮
𝛽
𝑝

Combining the vectors we obtain the following model:
?
1
?
2
⋮
?
𝑛
=
1
?
11
⋯
?
1𝑝
1
⋮
⋯ ⋯
⋱ ⋱
⋮
⋮
1
⋯ ⋯
?
𝑛𝑝
𝛽
0
𝛽
1
⋮
𝛽
𝑝
+
𝜀
1
𝜀
2
⋮
𝜀
𝑛
𝒚 = 𝑋𝜷 + 𝝐
• 𝒚 is known as the response vector
• X is a non-random matrix of our explanatory (predictor) variables known as
the design matrix
• β is a vector of unknown parameters
• 𝝐 is a random vector of errors

•