lecture3

# lecture3 - ISYE6414 Summer 2010 Lecture 3 The Linear Model...

This preview shows pages 1–4. Sign up to view the full content.

ISYE6414 Summer 2010 Lecture 3 The Linear Model: OLS Regression Cont. Dr. Kobi Abayomi June 7, 2010 1 Deviations, Residuals, R-squared 1.1 Deviations We have spent some time thinking about variation (variance) as the mean squared deviation. Remember the equations we know already s 2 x = ( x i - x ) 2 n - 1 (1) s 2 y = ( y i - y ) 2 n - 1 (2) Remember that x i , y i are what we observe , and x, y are what we expect . Let’s look at deviation with respect to the regression model again. We call the expected value for y given an observed value x i , ˆ y i = ˆ β 0 + ˆ β 1 x i (3) the ﬁtted value of y for observed value x . ˆ y i is the predicted value for the observed value x . So, ˆ y i is the expected value for y at the observed level of x and y i is the observed value. So then the deviation with respect to the regression model is 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
e i = ˆ ± i = y i - ˆ y i (4) 1.2 Residuals Remember our assumption (or hypothesis ) about the error variable ± ∼ N (0 , σ 2 ). We suppose that the error has a zero mean and some known, constant variance. Our sample estimate for σ 2 is the square root of the sum of squared error (SSE) divided by n - 2: s 2 e = SSE n - 2 = e i 2 n - 2 = ( y i - ˆ y i ) 2 n - 2 (5) Notice that we compute ˆ y i by substituting x i into the formula of the regression line. The observed value of ± is e – we call it the residuals. The square root of our estimate for the error in the model, s e is called the standard error of the estimate . The smallest value that s e can assume is 0, which occurs when SSE = 0, that is, when all the points fall on the regression line. Thus when s e is small, the ﬁt is good, and the linear model is likely to be an eﬀective tool. If is s e is large, the linear model is unlikely and we should discard or modify it. We should always, at least graphically, check the assumptions for the residuals: The residuals are approximately normally distributed The mean of the residuals is approximately zero The standard deviation of the residuals is relatively constant for regardless of the value of x . The values of the residuals appear to be independent of y – there are no patterns 1.3 Properties of the Fitted Regression Line e i = 0 OLS estimates minimize e 2 Y = ˆ Y X i e i = 0 = ˆ Y e i These properties follow directly from the least squares procedure which yields unbiased point estimators of the regression parameters. The normal error regression assumption, alternately, yields probabilistic results: inference on regression parameters, prediction, etc. ... 2
2 MLE parameter estimation The diﬀerence between the estimation procedure (OLS) and the error assumption is high- lighted when we estimate the model parameters by MLE.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 09/01/2011 for the course ISYE 6414 taught by Professor Staff during the Fall '08 term at Georgia Tech.

### Page1 / 14

lecture3 - ISYE6414 Summer 2010 Lecture 3 The Linear Model...

This preview shows document pages 1 - 4. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online