Coursehero >> California >> UCSB >> ESM 206

Course Hero has millions of student submitted documents similar to the one below including study guides, practice problems, reference materials, practice exams, textbook help and tutor support.

of Assumptions Ordinary Least Squares Regression (Part 1) ESM 206 Jan 17, 2008 1 Assumptions of OLS regression 1. Model is linear in parameters 2. The data are a random sample of the population 1. The errors are statistically independent from one another If assumptions 1-5 are satisfied, then OLS estimator is unbiased If assumption 6 is also satisfied, then 3. The expected value of the errors is always zero 4. The independent variables are not too strongly collinear 5. The independent variables are measured precisely 6. The residuals have constant variance 7. The errors are normally distributed OLS estimator has minimum variance of all unbiased estimators. If assumption 7 is also satisfied, then we can do hypothesis testing using t and F tests How can we test these assumptions? If assumptions are violated, what does this do to our conclusions? how do we fix the problem? 2 1. Model not linear in parameters Problem: Can't fit the model! Diagnosis: Look at the model Solutions: 1. Re-frame the model 2. Use nonlinear least squares (NLS) regression 3 2. Errors not independent Problem: parameter estimates are biased Diagnosis (1): look for correlation between residuals and another variable (not in the model) Diagnosis (2): look at autocorrelation function of residuals to find patterns in time Space I.e., observations that are nearby in time or space have residuals that are more similar than average I.e., residuals are dominated by another variable, Z, which is not random with respect to the other independent variables Solution (1): add the variable to the model Solution (2): fit model using generalized least squares (GLS) 4 Consumption 300 350 400 450 0.4 0.6 0.8 1.0 residuals.RegModel.10 -50 0 50 Price 1960 1970 Year 1980 1990 5 Autocorrelated residuals 50 Durbin-Watson test for autocorrelation Null hypothesis: no autocorrelation Only makes sense if observations are ordered in time Durbin-Watson test residual(t) -50 0 -50 0 residual(t-1) 50 data: Consumption ~ Price DW = 0.1946, p-value = 2.699e-16 alternative hypothesis: true autocorelation is not 0 6 Looking at autocorrelation in Rcmdr Durbin-Watson test: Fit model Model -> Numerical Diagnostics -> Durbin-Watson Test Plots: add the residuals to the dataset Fit model Models -> Add Observation Statistics to Data Make scatterplot of residuals & Year 7 3. Average error not everywhere zero ("nonlinearity") Problem: indicates that model is wrong Diagnosis: Consumption 450 300 350 400 For one-variable regression, look for curvature in plot of Y vs. X 0.4 0.6 Price 0.8 1.0 8 3. Average error not everywhere zero ("nonlinearity") Problem: indicates that model is wrong Diagnosis: Chlorophyll.a 150 0 50 100 For multiple regression, look for curvature in plot of observed Y vs. predicted Y Add "Fitted Values" to dataset These are the "Predicted Y" 50 100 150 fitted.Chlor.model.1 9 3. Average error not everywhere zero ("nonlinearity") 40 residuals.Chlor.model.1 -20 -10 0 10 20 30 Problem: indicates that model is wrong Diagnosis: Look for curvature in plot of observed vs. predicted Y Look for curvature in plot of residuals vs. predicted Y 50 100 150 fitted.Chlor.model.1 10 3. Average error not everywhere zero ("nonlinearity") indicates Problem: that model is wrong Diagnosis: Look for curvature in plot of observed vs. predicted Y Look for curvature in plot of residuals vs. predicted Y look for curvature in partial-residual plots (also component+residual plots [CR plots]) 11 Models -> Graphs -> Component + Residual Plots Component+Residual Plot Component+Residual(Chlorophyll.a) Component+Residual(Chlorophyll.a) 80 -20 0 Component+Residual Plot 60 40 20 0 -20 1000 2000 NP 3000 4000 0 20 40 60 100 200 300 400 500 600 Phosphorus 12 Average error not everywhere zero ("nonlinearity") Solutions: If pattern is monotonic*, try transforming independent variable Downward curving: use powers less than one E.g. Square root, log, inverse If not, try adding additional terms in the independent variable (e.g., quadratic) Upward curving: use powers greater than one E.g. square * Monotonic: always increasing or always decreasing 13 4. Independent variables are collinear Problem: parameter estimates are imprecise Diagnosis: Look for correlations among independent variables In regression output, none of the individual terms are significant, even though the model as a whole is Solutions: Live with it Remove statistically redundant variables Special case: variables that are perfectly correlated 14 Parameter b0 b1 b2 Residual St dev R2 Est value 16.37383 1.986335 -1.22964 31.6315 0.534192 St dev 41.50584 1.02642 2.131899 t student 0.394495 1.935206 -0.57678 Prob(>|t|) 0.696315 0.063504 0.568867 y = b0 + b1.x1 + b2.x2 R2(adj) F 0.499688 15.48191 Prob(>F) 3.32E-05 0 0; 1 1; 2 0.5; XZ 0.95 15 5. Independent variables not precise ("measurement error") Problem: parameter estimates are biased Diagnosis: know how your data were collected! Solution: very hard State space models Restricted maximum likelihood (REML) Use simulations to estimate bias Consult a professional! 16 6. Errors have non-constant variance ("heteroskedasticity") Problem: Parameter estimates are unbiased P-values are unreliable Diagnosis: plot residuals against fitted values 17 60 50 40 30 Residuals 20 10 0 -10 -20 -30 0 20 40 60 80 100 120 140 160 Predicted chlorophyll-a 18 Errors have non-constant variance ("heteroskedasticity") Problem: Parameter estimates are unbiased P-values are unreliable Solutions: Transform the dependent variable If residual variance increases with predicted value, try transforming with power less than one Diagnosis: plot studentized residuals against fitted values 19 Try square root transform 4 sqrt(Chlorophyll-a) Residual 3 2 1 0 -1 -2 -3 .0 2.5 5.0 7.5 10.0 12.5 15.0 s qrt(Chlorophyll-a) Predicted 20 Errors have non-constant variance ("heteroskedasticity") Problem: Parameter estimates are unbiased P-values are unreliable Solutions: Transform the dependent variable May create nonlinearity in the model Diagnosis: plot studentized residuals against fitted values Fit a generalized linear model (GLM) For some distributions, the variance changes with the mean in predictable ways Fit a generalized least squares model (GLS) Specifies how variance depends on one or more variables Fit a weighted least squares regression (WLS) Also good when data points have differing amount of precision 21

Find millions of documents on Course Hero - Study Guides, Lecture Notes, Reference Materials, Practice Exams and more. Course Hero has millions of course specific materials providing students with the best way to expand their education.

Below is a small sample set of documents: