Introduction to Econometrics
51
Technical Notes
Introduction to Econometrics
Lecture 5: Multiple Regression  Orthogonality and Collinearity
Ordinary Least Squares Estimation for Multiple Regression Models
Multiple
regression analysis is needed in economics partly because data are not the outcome of a
controlled experiment, and cannot be selected to satisfy the
ceteris paribus
condition used for the thought
experiments of theory. In simple regression analysis we assumed that each observation on the dependent
variable y
i
could be written as the sum of explained and unexplained components
y
i
=
α
+
β
x
i
+ u
i
Multiple regression analysis extends the scope of the explained component so that it becomes the sum of a
number of distinct effects. So
y
i
=
α
+
β
x
i
+
θ
z
i
+
φ
w
i
+ u
i
The explained component of y
i
(its conditional mean) is given by
E[y
i
X,Z,W] =
α
+
β
x
i
+
θ
z
i
+
φ
w
i
The OLS estimators are the values of
α
,
β
,
θ
and
φ
which minimise the residual sum of squares, so we
form the minimand
Σ
(u
i
)
2
=
Σ
(y
i

α

β
x
i

θ
z
i

φ
w
i
)
2
and differentiate with respect to each of these parameters
0 = 2
Σ
(y
i

α

β
x
i

θ
z
i

φ
w
i
)
∂
{
Σ
(u
i
)
2
}/
∂α
0 = 2
Σ
x
i
(y
i

α

β
x
i

θ
z
i

φ
w
i
)
∂
{
Σ
(u
i
)
2
}/
∂β
0 = 2
Σ
z
i
(y
i

α

β
x
i

θ
z
i

φ
w
i
)
∂
{
Σ
(u
i
)
2
}/
∂θ
0 = 2
Σ
w
i
(y
i

α

β
x
i

θ
z
i

φ
w
i
)
∂
{
Σ
(u
i
)
2
}/
∂φ
The first equation implies that the regression ‘line’ passes through the point of means, so that
mean
(Y) =
α
e
+
β
e
mean
(X) +
θ
e
mean
(Z) +
φ
e
mean
(W)
mean
(U) = 0
where
α
e
is the solution for
α
, and so on.
Subtracting
N
mean
(X){
mean
(Y) 
α
e

β
e
mean
(X) 
θ
e
mean
(Z) 
φ
e
mean
(W)}
which is zero
from the second equation, and multiplying by (1/2(N1))
β
e
Var(X) +
θ
e
Cov(X,Z) +
φ
e
Cov(X,W) = Cov(X,Y)
Similarly the third and fourth equations can be written as
β
e
Cov(Z,X) +
θ
e
Var(Z) +
φ
e
Cov(Z,W) = Cov(Z,Y)
β
e
Cov(W,X) +
θ
e
Cov(W,Z) +
φ
e
Var(W) = Cov(W,Y)
This preview has intentionally blurred sections. Sign up to view the full version.
View Full DocumentIntroduction to Econometrics
52
Technical Notes
This is a set of three linear equations in three unknowns: the OLS regression coefficients
β
e
,
θ
e
and
φ
e
are
functions of the variances and covariances of the data variables, in exactly the same way as in the case of
simple regression.
The regression ‘line’ makes full use of the data contained in the explanatory variables, so the sample
covariance between the residual and each of the explanatory variables is zero. Thus for the variable X
Σ
x
i
(y
i

α
e

β
e
x
i

θ
e
z
i

φ
e
w
i
) =
Σ
x
i
u
e
i
= 0
with similar equations for Z and W. Since the predicted values y
p
i
are just linear combinations of the
independent variables, we also have zero covariance between the prediction y
p
i
and the residual u
e
i
.
Σ
This is the end of the preview.
Sign up
to
access the rest of the document.
 Spring '10
 Cowell
 Economics, Econometrics, Regression Analysis

Click to edit the document details