STAT 100B: Homework 4 Solutions
Denise Tsai
February 24, 2011
1. For the simplest regression model
y
i
=
βx
i
+
i
,
i
= 1
, ..., n
, derive the least squares
estimate of
β
.
Solution:
We can find
ˆ
β
by minimizing
R
(
β
) =
n
X
i
=1
2
i
=
n
X
i
=1
(
y
i

βx
i
)
2
Set the derivative of
R
with respect to
β
equal to zero:
R
0
(
β
) = 2
n
X
i
=1
(
y
i

βx
i
)(

x
i
) = 0
Then,
n
X
i
=1
x
i
y
i
=
β
n
X
i
=1
(
x
i
)
2
∴
ˆ
β
=
∑
n
i
=1
x
i
y
i
∑
n
i
=1
x
2
i
(1)
2. For the simple regression
y
i
=
β
0
+
β
1
x
i
+
i
,
i
= 1
, ..., n
, derive the least squares
estimates of
β
0
and
β
1
. Show that the least squares problem for the simple regression
can be transformed into the least squares problem for the simplest regression.
Solution:
We can find
ˆ
β
0
and
ˆ
β
1
by minimizing
R
(
β
0
, β
1
) =
n
X
i
=1
2
i
=
n
X
i
=1
[
y
i

(
β
0
+
βx
i
)]
2
1
This preview has intentionally blurred sections. Sign up to view the full version.
View Full Document
Set the partial derivative of
R
with respect to
β
0
equal to zero:
∂R
∂β
0
= 2
n
X
i
=1
(
y
i

β
0

β
1
x
i
)(

1) = 0
Then,
n
X
i
=1
y
i
=
nβ
0
+
β
1
n
X
i
=1
x
i
n
¯
y
=
nβ
0
+
nβ
1
¯
x
∴
ˆ
β
0
= ¯
y

ˆ
β
1
¯
x
(2)
Set the partial derivative of
R
with respect to
β
1
This is the end of the preview.
Sign up
to
access the rest of the document.
 Winter '11
 Wu
 Least Squares, Yi, i=1

Click to edit the document details