*This preview shows
page 1. Sign up
to
view the full content.*

**Unformatted text preview: **STAT 100B HWV Solution
Problem 1: Suppose we want to ﬁt the linear regression model yi = xi1 β1 + xi2 β2 + ... + xip βp + i for i = 1, 2, ..., n by the least squares principle, i.e., we estimate β = (β1 , β2 , ..., βp )T by minimizing
n n i=1 R(β ) =
i=1 [yi − (xi1 β1 + xi2 β2 + ... + xip βp )]2 = (yi − xT β )2 , i where xi = (xi1 , xi2 , ..., xip )T . ˆ (1) Calculate the least squares estimate β . n n T β )x . ∂R(β )/∂β = −2 T A: ∂R(β )/∂βj = −2 i=1 (yi −xi ij i=1 (yi −xi β )xi . Setting ∂R(β )/∂β = ˆ 0, we get n xi yi = n xi xT β . Solving this equation, we get β = ( n xi xT )−1 n xi yi . i=1 i=1 i=1 i=1 i i (2) Compare the result in (1) with the result we obtained before for p = 1. ˆ A: When p = 1, xi becomes a scaler, and xT = xi , so β = n xi yi / n x2 . i=1 i=1 i i (3) Compare the result in (1) with the result we obtained before for simple linear regression with the intercept. A: For notational clarity, let xi be the vector xi in question (1). For simple linear regression where yi = α + βxi + i , the column vector xi becomes the two dimensional vector (1, xi )T . We can then apply the formula in the answer to question (1) to ﬁnd the least squares estimates of the intercept α and slope β . 1 ...

View
Full
Document