of are b1and b2. Figure 3 demonstrates the idea of OLS. Figure 3 The sum of squares residuals and the minimising values b1and b2 Definition: Among all possible straight lines, the line of best fit corresponds to those values of b1and b2that minimise n22i12ii 1S(Yx ) . The values of b1andb2that satisfy the above definition are called the least squares estimates as they correspond to the least of the sum of squares. The estimates are the values that minimise the above sum of squares, and can be obtained by setting the following partial derivatives to 0: 2ni12i1i 1S2(yx )(2.6) and 2ni12ii2i 1S2(yx )x(2.7)Note that from (1.6), we have for the solutions b1and b2that nnni12ii12i12i 1i 1i 112(yx )0ynxYbb xbYb x.(2.8)

16 Equation (1.7) gives ni12iii 1(ybb x )x0which, after some simplifications, yields xY2xxSbS, (2.9) where nnxYiiiii 1i 1nn222xxiii 1i 1nn222YYiii 1i 1S(xx)(yY)x ynxY;S(xx)xnx ;S(yY)ynY .So the line of best fit is xYxxSyYxx .SThis line passes through the point of means, (x,Y). We denote the fitted values of Y by ˆY,i.e. xYi12iixxSE(Y)ybb xYxxS.The values ii iˆeyydenote the residualsof the fitted model. Now, note that the sample correlation coefficient can be written as niii 1xYxYnnxxYY22iii 1i 11(xx)(YY)n1SrSS11(xx)(YY)n1n1; which means that the regression line has the alternative form YYxYxxSyYrxxS. Also, we can set the derivatives to zeros for equations (2.6) and (2.7): 122122020iiiiiiyNbxbx yx bxb(2.10) Simplify them to 12212iiiiiiNbxbyxbxbx y(2.11) From (1.9) b2can be derived as:

17 222iiiiiiNx yxybNxx and further in the mean form as 22iiixxyybxx(2.12.1) In the case where there is no intercept in the model, ie., 2iiiyx, the estimator b2is 22iiix ybx(2.12.2) 5. Assessing the OLS estimators Formulas (1.8) and (1.12) are used to compute the least squares estimates b1and b2. However, they are not well suited for examining the theoretical properties of the estimators. We can rewrite the estimator b2 as a linear estimator of y: 21Niiibw y(2.13) where 2()iiixxwxx(2.14) The term wi

#### You've reached the end of your free preview.

Want to read all 62 pages?

- Fall '16
- Jane Lee
- Least Squares, Linear Regression, Regression Analysis, Yi