HW3.pdf - Regression Analysis Homework Assignment 2 Due Nov...

Regression Analysis: Homework Assignment 2
Due: Nov. 16 (Tue.)
1. Let X be the n × p design matrix
X =







1 x
11
x
12
. . . x
1 ,p − 1
1 x
21
x
22
. . . x
2 ,p − 1
.
.
.
.
.
.
.
.
.
.
.
.
1 x
n 1
x
n 2
. . . x
n,p − 1







and consider the following linear regression model
y = Xβ + ϵ
where ϵ ∼ N (0 , I
n
σ
2
). Denote the least squares estimate of β by
b = ( X
′
X )
− 1
X
′
y , the fitted value by ˆ y = Xb = X ( X
′
X )
− 1
X
′
y , and
H = X ( X
′
X )
− 1
X
′
.
(a) (10 points) Show that
n ∑
i =1
var(ˆ y
i
) = p · σ
2
.
(b) (20 points) Derive the distribution of
( b − β )
′
X
′
X ( b − β )
MSE · p
.
(c) (20 points) Note that for a random vector z with E ( z ) = µ, Cov( z ) =
V , we have E ( z
′
Az ) = tr( AV ) + µ
′
Aµ . Use this to show
E ( MSE ) = σ
2
E ( MSR ) = σ
2
+
1
p − 1
( Xβ )
′
( H −
1
n
11
′
)( Xβ ) .
1

2. Assume the regression model