Chapter03

Chapter03 - Multiple Regression STAT 563 Spring 2007 Design...

Info iconThis preview shows pages 1–14. Sign up to view the full content.

View Full Document Right Arrow Icon
Multiple Regression STAT 563 Spring 2007
Background image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Design Matrix Define the n x p matrix And the column vectors X j =[X 1j ,…. ., X nj ] Model now can be written as Y = X β + ε = nk n n k k x x x x x x x x x X .. 1 . . . . .. 1 .. 1 2 1 2 22 21 1 12 11
Background image of page 2
Model Where = = = k k n y y y Y ε β . , . , . 2 1 1 0 2 1
Background image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Model With p=k+1 predictors, the model can also be written as n i X X X X y i ij k j j i ik k i i i ,.... , 2 , 1 ..... 1 0 2 2 1 1 0 = + + = + + + + + = = ε β
Background image of page 4
Fitting the Model Like in simple linear regression, define the SSE as [ ] [ ] β ε X y X y S n i i - - = = = = ' ) ( 1 2
Background image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Criteria Goal is to make all residuals as small as possible All Residuals Zero n parameters with Average residuals zero and many others Minimize average absolute residual L 1 norm Robust with respect to outliers Computational problems Theoretical difficulties 0 ˆ ˆ , ˆ 1 = - = - = - β X Y Y Y Y X 0 . . ) 1 ( 1 : 1 ˆ = - = - = = Y Y Y Y r Y Y
Background image of page 6
Criteria Minimize average squared residuals (or sum of a squared residuals) Least Squares Minimize a bounded, convex function of residuals Robust Estimation
Background image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Least Squares Find that minimizes Determine β ˆ ( 29 ( 29 X X y X y y X X X y y X y y X y X y S ' ' ' ' 2 ' ' ' ' ' ' ' ' ) ( + - = + - - = - - = Why? ( 29 y X X X y X X X implies which X X y X S ' ' ˆ ' ˆ ' 0 ˆ ' 2 ' 2 1 - = = = + - = Normal equations LS Estimate
Background image of page 8
Least Squares Provided that (X’X) -1 exists. A linear independence between the regressors (ie, no column of X is a linear combination of the other columns) will guarantee this. Dependence between columns of X will lead to multicollinearity (more later) The fitted value is given by ( 29 ( 29 Matrix Hat ' ' ' ' ˆ ˆ 1 1 the called is X X X X H where y H y X X X X X y - - = = = = β
Background image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Unit length scaling With the unit length scaling, we can define • Note that each regressor w j has The regression now is 2 1 0 ) ( ,...... , 2 , 1 ; ,.... , 2 , 1 , i n i ij jj Y i i jj j ij ij x x S where SS y y y and k j n i S x x w - = - = = = - = = 1 ) ( 0 1 2 = - = = n i j ij j w w length and w mean 0 1 2 2 1 1 0 ' ) ' ( ˆ ...... y W W W b is estimate squares least the and w b w b w b y i ik k i i i - = + + + + = ε
Background image of page 10
Unit length scaling Let us take a look at an off-diagonal element in W’W We can write ) ( . 2 1 12 22 21 11 12 21 12 x and x between n correlatio r S x x S x x w w = - - = = 1 ... .. 1 .. .. ... 1 ... 1 ' 2 1 2 12 1 12 k k k k r r r r r r W W Correlation matrix = ky y y r r r y W .. ' 2 1 0
Background image of page 11

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Unit length scaling Where j jj jy jj n u u j uj jy j i jj ii ij jj ii n u j uj i ui ij x and y between n correlatio SST S S SST S y y x x r x and x between n correlatio S S S S S x x x x r = - - = = - - = = = 1 2 / 1 1 ) )( ( ) ( ) )( (
Background image of page 12
Unit length scaling If unit normal scaling (z-score) is used, then you can show that Z’Z=(n-1)W’W Estimates obtained from both scaling are identical irrespective of scaling is usually called standardized regression coefficients
Background image of page 13

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
Image of page 14
This is the end of the preview. Sign up to access the rest of the document.

This note was uploaded on 03/08/2009 for the course 960 563 taught by Professor Unknown during the Spring '07 term at Rutgers.

Page1 / 61

Chapter03 - Multiple Regression STAT 563 Spring 2007 Design...

This preview shows document pages 1 - 14. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online