100BHWVS - STAT 100B HWV Solution Problem 1 Suppose we want to fit the linear regression model yi = xi1 β1 xi2 β2 xip βp i for i = 1 2 n by the

Info iconThis preview shows page 1. Sign up to view the full content.

View Full Document Right Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: STAT 100B HWV Solution Problem 1: Suppose we want to fit the linear regression model yi = xi1 β1 + xi2 β2 + ... + xip βp + i for i = 1, 2, ..., n by the least squares principle, i.e., we estimate β = (β1 , β2 , ..., βp )T by minimizing n n i=1 R(β ) = i=1 [yi − (xi1 β1 + xi2 β2 + ... + xip βp )]2 = (yi − xT β )2 , i where xi = (xi1 , xi2 , ..., xip )T . ˆ (1) Calculate the least squares estimate β . n n T β )x . ∂R(β )/∂β = −2 T A: ∂R(β )/∂βj = −2 i=1 (yi −xi ij i=1 (yi −xi β )xi . Setting ∂R(β )/∂β = ˆ 0, we get n xi yi = n xi xT β . Solving this equation, we get β = ( n xi xT )−1 n xi yi . i=1 i=1 i=1 i=1 i i (2) Compare the result in (1) with the result we obtained before for p = 1. ˆ A: When p = 1, xi becomes a scaler, and xT = xi , so β = n xi yi / n x2 . i=1 i=1 i i (3) Compare the result in (1) with the result we obtained before for simple linear regression with the intercept. A: For notational clarity, let xi be the vector xi in question (1). For simple linear regression where yi = α + βxi + i , the column vector xi becomes the two dimensional vector (1, xi )T . We can then apply the formula in the answer to question (1) to find the least squares estimates of the intercept α and slope β . 1 ...
View Full Document

This note was uploaded on 01/27/2011 for the course STATS 100b taught by Professor Staff during the Fall '08 term at UCLA.

Ask a homework question - tutors are online