{[ promptMessage ]}

Bookmark it

{[ promptMessage ]}

Lecture18

# Lecture18 - Lecture 18 Weighted least squares ridge...

This preview shows pages 1–7. Sign up to view the full content.

Lecture 18: Weighted least squares & ridge regression Stat 704: Data Analysis I, Fall 2010 Tim Hanson, Ph.D. University of South Carolina T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 1 / 21

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares 11.1: Weighted least squares * Chapters 3 and 6 discuss transformations of x 1 , . . . , x k and/or Y . * This is “quick and dirty” but may not solve the problem. Or can create an uninterpretable mess (book: “inappropriate”). * More advanced remedy: weighted least squares regression. * Model is as before Y i = β 0 + β 1 x i 1 + · · · β k x ik + ² i , but now ² i ind . N (0 , σ 2 i ) . Note the subscript on σ i ... T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 2 / 21
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares * Here var( Y i ) = σ 2 i . Give observations with higher variance less weight in the regression fitting. * Say σ 1 , . . . , σ n are known. Let w i = 1 2 i and define the weight matrix W = w 1 0 · · · 0 0 w 2 · · · 0 . . . . . . . . . . . . 0 0 · · · w n = σ - 2 1 0 · · · 0 0 σ - 2 2 · · · 0 . . . . . . . . . . . . 0 0 · · · σ - 2 n . * Maximizing the likelihood (pp. 422-423) gives the estimates for β : b w = ( XWX 0 ) - 1 X 0 WY . T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 3 / 21

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares * However, σ 1 , . . . , σ n are almost always unknown. * If the mean function is appropriate, then E ( e 2 i ) = σ 2 i where e i is obtained from ordinary least squares, so e 2 i estimates σ 2 i and | e i | estimates σ i (pp. 424-425). * Look at plots of | e i | from a normal fit against predictors in the model and the fitted values ˆ Y i to see how σ i changes with predictors or fitted values. T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 4 / 21
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares 1 Regress Y against predictor variable(s) as usual (OLS) & obtain e 1 , . . . , e n & ˆ Y 1 , . . . , ˆ Y n . 2 Regress | e i | against predictors x 1 , . . . , x k or fitted values ˆ Y i . 3 Let ˆ s i be the fitted values for the regression in 2. 4 Define w i = 1 / ˆ s 2 i for i = 1 , . . . , n . 5 Use b w = ( XWX 0 ) - 1 X 0 WY as estimated coefficients – automatic in SAS. SAS will also use the correct cov b w = ( X 0 WX ) - 1 (p. 423). This is developed formally in linear models. T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 5 / 21

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares SAS code: initial fit * SAS example for Weighted Least Squares ; * Blood pressure data in Table 11.1 ; data bloodp; input age dbp @@; datalines; 27 73 21 66 22 63 24 75 25 71 23 70 20 65 20 70 29 79 24 72 25 68 28 67 26 79 38 91 32 76 33 69 31 66 34 73 37 78 38 87 33 76 35 79 30 73 31 80 37 68 39 75 46 89 49 101 40 70 42 72 43 80 46 83 43 75 44 71 46 80 47 96 45 92 49 80 48 70 40 90 42 85 55 76 54 71 57 99 52 86 53 79 56 92 52 85 50 71 59 90 50 91 52 100 58 80 57 109 ; run;
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### Page1 / 21

Lecture18 - Lecture 18 Weighted least squares ridge...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online