This preview shows pages 1–2. Sign up to view the full content.
3.
The OLS estimator fit without a constant term is
b
=
x
′
y
/
x
′
x
.
Assuming that the constant term is, in fact,
zero, the variance of this estimator is Var[
b
]
=
σ
2
/
x
′
x
.
If a constant term is included in the regression, then,
b
′
=
( )
( )
1
n
ii
i
x
xy y
=
Σ−
−
/
()
2
1
n
x
x
=
The appropriate variance is
σ
2
/
(
2
1
n
)
x
x
=
as always.
The ratio of these two is
V
a
r
[
b
]/Var[
b
′
] =
[
σ
2
/
x
′
x
] / [
σ
2
/
2
1
n
x
x
=
]
But,
(
2
1
n
)
x
x
=
=
x
′
x
+
n
x
2
so the ratio is
Var[
b
]/Var[
b
′
]
=
[
x
′
x
+
n
x
2
]/
x
′
x
=
1 
n
x
2
/
x
′
x
=
1  {
n
x
2
/[
S
xx
+
n
x
2
]} <
1
It follows that fitting the constant term when it is unnecessary inflates the variance of the least squares
estimator if the mean of the regressor is not zero.
4.
We could write the regression as
y
i
=
(
α
+
λ
)
+
β
x
i
+
(
ε
i

λ
)
=
α
*
+
β
x
i
+
ε
i
*
.
Then, we know that
E
[
ε
i
*
] = 0, and that it is independent of
x
i
.
Therefore, the second form of the model satisfies all of our
assumptions for the classical regression.
Ordinary least squares will give unbiased estimators of
α
*
and
β
.
As
long as
λ
is not zero, the constant term will differ from
α
.
5.
Let the constant term be written as
a
=
Σ
i
d
i
y
i
=
Σ
i
d
i
(
α
+
β
x
i
+
ε
i
)
=
αΣ
i
d
i
+
βΣ
i
d
i
x
i
+
Σ
i
d
i
ε
i
.
In order for
a
to be unbiased for all samples of
x
i
, we must have
Σ
i
d
i
= 1 and
Σ
i
d
i
x
i
= 0.
Consider, then, minimizing the
variance of
a
subject to these two constraints.
The Lagrangean is
L
*
=
Var[
a
] +
λ
1
(
Σ
i
d
i
 1) +
λ
2
Σ
i
d
i
x
i
where
Var[
a
] =
Σ
i
σ
2
d
i
2
.
Now, we minimize this with respect to
d
i
,
λ
1
, and
λ
2
.
The (
n
+2) necessary conditions are
∂
L
*
/
∂
d
i
=
2
σ
2
d
i
+
λ
1
+
λ
2
x
i
,
∂
L
*
/
∂λ
1
=
Σ
i
d
i
 1,
∂
L
*
/
∂λ
2
=
Σ
i
d
i
x
i
The first equation implies that
d
i
=
[1/(2
σ
2
)](
λ
1
+
λ
2
x
i
).
Therefore,
Σ
i
d
i
=
1
=
[1/(2
σ
2
)][
n
λ
1
+ (
Σ
i
x
i
)
λ
2
]
and
Σ
i
d
i
x
i
=
0
=
[1/(2
σ
2
)][(
Σ
i
x
i
)
λ
1
+ (
Σ
i
x
i
2
)
λ
2
].
We can solve these two equations for
λ
1
and
λ
2
by first multiplying both equations by 2
σ
2
then writing the
resulting equations as
The solution is
nx
xx
Σ
ΣΣ
2
1
2
2
2
0
⎡
⎣
⎢
⎤
⎦
⎥
⎛
⎝
⎜
⎜
⎞
⎠
⎟
⎟
=
−
⎡
⎣
⎢
⎢
⎤
⎦
⎥
⎥
λ
λ
σ
.
1
λ
σ
1
2
2
2
2
0
⎛
⎝
⎜
⎜
⎜
⎞
⎠
⎟
⎟
⎟
=
⎡
⎣
⎢
⎢
⎤
⎦
⎥
⎥
−
⎡
⎣
⎢
⎢
⎤
⎦
⎥
⎥
Σ
.
Note, first, that
Σ
i
x
i
=
n
x
.
Thus, the determinant of the matrix is
n
Σ
i
x
i
2
 (
n
x
)
2
=
n
(
Σ
i
x
i
2

n
x
2
) =
nS
xx
where
S
xx
(
2
1
n
)
x
x
=
.
The solution is, therefore,
22
1
2
2
1
00
xx
xn
x
nS
nx
λ
⎡
⎤⎡
⎤
Σ
−−
σ
⎛⎞
=
⎜⎟
⎢
⎥
⎢
λ
−
⎝⎠
⎥
⎣
⎦⎣
⎦
or
λ
1
=
(2
σ
2
)(
Σ
i
x
i
2
/
n
)/
S
xx
λ
2
=
(2
σ
2
x
)/
S
xx
Then,
d
i
=
[
Σ
i
x
i
2
/
n

x x
i
]/
S
xx
This simplifies if we write
Σ
x
i
2
=
S
xx
+
n
x
2
, so
Σ
i
x
i
2
/
n
=
S
xx
/
n
+
x
2
.
Then,
d
i
=
1/
n
+
x
(
x

x
i
)/
S
xx
, or, in a more familiar form,
d
i
=
1/
n

x
(
x
i

x
)/
S
xx
.
This makes the intercept term
Σ
i
d
i
y
i
=
(1/
n
)
Σ
i
y
i

x
( )
1
n
i
x xy
=
/
S
xx
=
y

b
x
which was to be shown.
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
This is the end of the preview. Sign up
to
access the rest of the document.
This note was uploaded on 11/13/2011 for the course ECE 4105 taught by Professor Dr.fang during the Spring '10 term at University of Florida.
 Spring '10
 Dr.Fang

Click to edit the document details