lecture3 - ISYE6414 Summer 2010 Lecture 3 The Linear Model...

Info iconThis preview shows pages 1–4. Sign up to view the full content.

View Full Document Right Arrow Icon
ISYE6414 Summer 2010 Lecture 3 The Linear Model: OLS Regression Cont. Dr. Kobi Abayomi June 7, 2010 1 Deviations, Residuals, R-squared 1.1 Deviations We have spent some time thinking about variation (variance) as the mean squared deviation. Remember the equations we know already s 2 x = ( x i - x ) 2 n - 1 (1) s 2 y = ( y i - y ) 2 n - 1 (2) Remember that x i , y i are what we observe , and x, y are what we expect . Let’s look at deviation with respect to the regression model again. We call the expected value for y given an observed value x i , ˆ y i = ˆ β 0 + ˆ β 1 x i (3) the fitted value of y for observed value x . ˆ y i is the predicted value for the observed value x . So, ˆ y i is the expected value for y at the observed level of x and y i is the observed value. So then the deviation with respect to the regression model is 1
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
e i = ˆ ± i = y i - ˆ y i (4) 1.2 Residuals Remember our assumption (or hypothesis ) about the error variable ± ∼ N (0 , σ 2 ). We suppose that the error has a zero mean and some known, constant variance. Our sample estimate for σ 2 is the square root of the sum of squared error (SSE) divided by n - 2: s 2 e = SSE n - 2 = e i 2 n - 2 = ( y i - ˆ y i ) 2 n - 2 (5) Notice that we compute ˆ y i by substituting x i into the formula of the regression line. The observed value of ± is e – we call it the residuals. The square root of our estimate for the error in the model, s e is called the standard error of the estimate . The smallest value that s e can assume is 0, which occurs when SSE = 0, that is, when all the points fall on the regression line. Thus when s e is small, the fit is good, and the linear model is likely to be an effective tool. If is s e is large, the linear model is unlikely and we should discard or modify it. We should always, at least graphically, check the assumptions for the residuals: The residuals are approximately normally distributed The mean of the residuals is approximately zero The standard deviation of the residuals is relatively constant for regardless of the value of x . The values of the residuals appear to be independent of y – there are no patterns 1.3 Properties of the Fitted Regression Line e i = 0 OLS estimates minimize e 2 Y = ˆ Y X i e i = 0 = ˆ Y e i These properties follow directly from the least squares procedure which yields unbiased point estimators of the regression parameters. The normal error regression assumption, alternately, yields probabilistic results: inference on regression parameters, prediction, etc. ... 2
Background image of page 2
2 MLE parameter estimation The difference between the estimation procedure (OLS) and the error assumption is high- lighted when we estimate the model parameters by MLE.
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 4
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 09/01/2011 for the course ISYE 6414 taught by Professor Staff during the Fall '08 term at Georgia Tech.

Page1 / 14

lecture3 - ISYE6414 Summer 2010 Lecture 3 The Linear Model...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online