Lecture18 - Lecture 18: Weighted least squares & ridge...

Info iconThis preview shows pages 1–7. Sign up to view the full content.

View Full Document Right Arrow Icon
Stat 704: Data Analysis I, Fall 2010 Tim Hanson, Ph.D. University of South Carolina T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 1 / 21
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares 11.1: Weighted least squares * Chapters 3 and 6 discuss transformations of x 1 ,..., x k and/or Y . * This is “quick and dirty” but may not solve the problem. Or can create an uninterpretable mess (book: “inappropriate”). * More advanced remedy: weighted least squares regression. * Model is as before Y i = β 0 + β 1 x i 1 + ··· β k x ik + ² i , but now ² i ind . N (0 2 i ) . Note the subscript on σ i ... T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 2 / 21
Background image of page 2
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares * Here var( Y i ) = σ 2 i . Give observations with higher variance less weight in the regression fitting. * Say σ 1 ,...,σ n are known. Let w i = 1 2 i and define the weight matrix W = w 1 0 ··· 0 0 w 2 ··· 0 . . . . . . . . . . . . 0 0 ··· w n = σ - 2 1 0 ··· 0 0 σ - 2 2 ··· 0 . . . . . . . . . . . . 0 0 ··· σ - 2 n . * Maximizing the likelihood (pp. 422-423) gives the estimates for β : b w = ( XWX 0 ) - 1 X 0 WY . T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 3 / 21
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares * However, σ 1 ,...,σ n are almost always unknown. * If the mean function is appropriate, then E ( e 2 i ) = σ 2 i where e i is obtained from ordinary least squares, so e 2 i estimates σ 2 i and | e i | estimates σ i (pp. 424-425). * Look at plots of | e i | from a normal fit against predictors in the model and the fitted values ˆ Y i to see how σ i changes with predictors or fitted values. T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 4 / 21
Background image of page 4
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares 1 Regress Y e 1 ,..., e n ˆ Y 1 ,..., ˆ Y n . 2 Regress | e i | against predictors x 1 ,..., x k or fitted values ˆ Y i . 3 Let ˆ s i be the fitted values for the regression in 2. 4 Define w i = 1 / ˆ s 2 i for i = 1 ,..., n . 5 Use b w = ( XWX 0 ) - 1 X 0 WY as estimated coefficients – automatic in SAS. SAS will also use the correct cov b w = ( X 0 WX ) - 1 (p. 423). This is developed formally in linear models. T. Hanson (USC) Stat 704: Data Analysis I, Fall 2010 5 / 21
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Chapter 11 11.1 Unequal variance rem. measure: Weighted least squares SAS code: initial fit * SAS example for Weighted Least Squares ; * Blood pressure data in Table 11.1 ; data bloodp; input age dbp @@; datalines; 27 73 21 66 22 63 24 75 25 71 23 70 20 65 20 70 29 79 24 72 25 68 28 67 26 79 38 91 32 76 33 69 31 66 34 73 37 78 38 87 33 76 35 79 30 73 31 80 37 68 39 75 46 89 49 101 40 70 42 72 43 80 46 83 43 75 44 71 46 80 47 96 45 92 49 80
Background image of page 6
Image of page 7
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 12/14/2011 for the course STAT 704 taught by Professor Staff during the Fall '11 term at South Carolina.

Page1 / 21

Lecture18 - Lecture 18: Weighted least squares & ridge...

This preview shows document pages 1 - 7. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online