Lecture18

# Lecture18 - Lecture 18: Weighted least squares &amp; ridge...

This preview shows pages 1–7. Sign up to view the full content.

Stat 704: Data Analysis I, Fall 2010 Tim Hanson, Ph.D. University of South Carolina T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 1 / 21

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares 11.1: Weighted least squares * Chapters 3 and 6 discuss transformations of x 1 ,..., x k and/or Y . * This is “quick and dirty” but may not solve the problem. Or can create an uninterpretable mess (book: “inappropriate”). * More advanced remedy: weighted least squares regression. * Model is as before Y i = β 0 + β 1 x i 1 + ··· β k x ik + ² i , but now ² i ind . N (0 2 i ) . Note the subscript on σ i ... T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 2 / 21
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares * Here var( Y i ) = σ 2 i . Give observations with higher variance less weight in the regression ﬁtting. * Say σ 1 ,...,σ n are known. Let w i = 1 2 i and deﬁne the weight matrix W = w 1 0 ··· 0 0 w 2 ··· 0 . . . . . . . . . . . . 0 0 ··· w n = σ - 2 1 0 ··· 0 0 σ - 2 2 ··· 0 . . . . . . . . . . . . 0 0 ··· σ - 2 n . * Maximizing the likelihood (pp. 422-423) gives the estimates for β : b w = ( XWX 0 ) - 1 X 0 WY . T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 3 / 21

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares * However, σ 1 ,...,σ n are almost always unknown. * If the mean function is appropriate, then E ( e 2 i ) = σ 2 i where e i is obtained from ordinary least squares, so e 2 i estimates σ 2 i and | e i | estimates σ i (pp. 424-425). * Look at plots of | e i | from a normal ﬁt against predictors in the model and the ﬁtted values ˆ Y i to see how σ i changes with predictors or ﬁtted values. T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 4 / 21
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares 1 Regress Y e 1 ,..., e n ˆ Y 1 ,..., ˆ Y n . 2 Regress | e i | against predictors x 1 ,..., x k or ﬁtted values ˆ Y i . 3 Let ˆ s i be the ﬁtted values for the regression in 2. 4 Deﬁne w i = 1 / ˆ s 2 i for i = 1 ,..., n . 5 Use b w = ( XWX 0 ) - 1 X 0 WY as estimated coeﬃcients – automatic in SAS. SAS will also use the correct cov b w = ( X 0 WX ) - 1 (p. 423). This is developed formally in linear models. T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 5 / 21

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares SAS code: initial ﬁt * SAS example for Weighted Least Squares ; * Blood pressure data in Table 11.1 ; data bloodp; input age dbp @@; datalines; 27 73 21 66 22 63 24 75 25 71 23 70 20 65 20 70 29 79 24 72 25 68 28 67 26 79 38 91 32 76 33 69 31 66 34 73 37 78 38 87 33 76 35 79 30 73 31 80 37 68 39 75 46 89 49 101 40 70 42 72 43 80 46 83 43 75 44 71 46 80 47 96 45 92 49 80
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 12/14/2011 for the course STAT 704 taught by Professor Staff during the Fall '11 term at South Carolina.

### Page1 / 21

Lecture18 - Lecture 18: Weighted least squares &amp; ridge...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online