100BHWVS

# 100BHWVS - STAT 100B HWV Solution Problem 1 Suppose we want to ﬁt the linear regression model yi = xi1 β1 xi2 β2 xip βp i for i = 1 2 n by the

This preview shows page 1. Sign up to view the full content.

This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: STAT 100B HWV Solution Problem 1: Suppose we want to ﬁt the linear regression model yi = xi1 β1 + xi2 β2 + ... + xip βp + i for i = 1, 2, ..., n by the least squares principle, i.e., we estimate β = (β1 , β2 , ..., βp )T by minimizing n n i=1 R(β ) = i=1 [yi − (xi1 β1 + xi2 β2 + ... + xip βp )]2 = (yi − xT β )2 , i where xi = (xi1 , xi2 , ..., xip )T . ˆ (1) Calculate the least squares estimate β . n n T β )x . ∂R(β )/∂β = −2 T A: ∂R(β )/∂βj = −2 i=1 (yi −xi ij i=1 (yi −xi β )xi . Setting ∂R(β )/∂β = ˆ 0, we get n xi yi = n xi xT β . Solving this equation, we get β = ( n xi xT )−1 n xi yi . i=1 i=1 i=1 i=1 i i (2) Compare the result in (1) with the result we obtained before for p = 1. ˆ A: When p = 1, xi becomes a scaler, and xT = xi , so β = n xi yi / n x2 . i=1 i=1 i i (3) Compare the result in (1) with the result we obtained before for simple linear regression with the intercept. A: For notational clarity, let xi be the vector xi in question (1). For simple linear regression where yi = α + βxi + i , the column vector xi becomes the two dimensional vector (1, xi )T . We can then apply the formula in the answer to question (1) to ﬁnd the least squares estimates of the intercept α and slope β . 1 ...
View Full Document

## This note was uploaded on 01/27/2011 for the course STATS 100b taught by Professor Staff during the Fall '08 term at UCLA.

Ask a homework question - tutors are online