Chapter03

# Chapter03 - Multiple Regression STAT 563 Spring 2007 Design...

This preview shows pages 1–14. Sign up to view the full content.

Multiple Regression STAT 563 Spring 2007

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Design Matrix Define the n x p matrix And the column vectors X j =[X 1j ,…. ., X nj ] Model now can be written as Y = X β + ε = nk n n k k x x x x x x x x x X .. 1 . . . . .. 1 .. 1 2 1 2 22 21 1 12 11
Model Where = = = k k n y y y Y ε β . , . , . 2 1 1 0 2 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Model With p=k+1 predictors, the model can also be written as n i X X X X y i ij k j j i ik k i i i ,.... , 2 , 1 ..... 1 0 2 2 1 1 0 = + + = + + + + + = = ε β
Fitting the Model Like in simple linear regression, define the SSE as [ ] [ ] β ε X y X y S n i i - - = = = = ' ) ( 1 2

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Criteria Goal is to make all residuals as small as possible All Residuals Zero n parameters with Average residuals zero and many others Minimize average absolute residual L 1 norm Robust with respect to outliers Computational problems Theoretical difficulties 0 ˆ ˆ , ˆ 1 = - = - = - β X Y Y Y Y X 0 . . ) 1 ( 1 : 1 ˆ = - = - = = Y Y Y Y r Y Y
Criteria Minimize average squared residuals (or sum of a squared residuals) Least Squares Minimize a bounded, convex function of residuals Robust Estimation

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Least Squares Find that minimizes Determine β ˆ ( 29 ( 29 X X y X y y X X X y y X y y X y X y S ' ' ' ' 2 ' ' ' ' ' ' ' ' ) ( + - = + - - = - - = Why? ( 29 y X X X y X X X implies which X X y X S ' ' ˆ ' ˆ ' 0 ˆ ' 2 ' 2 1 - = = = + - = Normal equations LS Estimate
Least Squares Provided that (X’X) -1 exists. A linear independence between the regressors (ie, no column of X is a linear combination of the other columns) will guarantee this. Dependence between columns of X will lead to multicollinearity (more later) The fitted value is given by ( 29 ( 29 Matrix Hat ' ' ' ' ˆ ˆ 1 1 the called is X X X X H where y H y X X X X X y - - = = = = β

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Unit length scaling With the unit length scaling, we can define • Note that each regressor w j has The regression now is 2 1 0 ) ( ,...... , 2 , 1 ; ,.... , 2 , 1 , i n i ij jj Y i i jj j ij ij x x S where SS y y y and k j n i S x x w - = - = = = - = = 1 ) ( 0 1 2 = - = = n i j ij j w w length and w mean 0 1 2 2 1 1 0 ' ) ' ( ˆ ...... y W W W b is estimate squares least the and w b w b w b y i ik k i i i - = + + + + = ε
Unit length scaling Let us take a look at an off-diagonal element in W’W We can write ) ( . 2 1 12 22 21 11 12 21 12 x and x between n correlatio r S x x S x x w w = - - = = 1 ... .. 1 .. .. ... 1 ... 1 ' 2 1 2 12 1 12 k k k k r r r r r r W W Correlation matrix = ky y y r r r y W .. ' 2 1 0

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Unit length scaling Where j jj jy jj n u u j uj jy j i jj ii ij jj ii n u j uj i ui ij x and y between n correlatio SST S S SST S y y x x r x and x between n correlatio S S S S S x x x x r = - - = = - - = = = 1 2 / 1 1 ) )( ( ) ( ) )( (
Unit length scaling If unit normal scaling (z-score) is used, then you can show that Z’Z=(n-1)W’W Estimates obtained from both scaling are identical irrespective of scaling is usually called standardized regression coefficients

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

## This note was uploaded on 03/08/2009 for the course 960 563 taught by Professor Unknown during the Spring '07 term at Rutgers.

### Page1 / 61

Chapter03 - Multiple Regression STAT 563 Spring 2007 Design...

This preview shows document pages 1 - 14. Sign up to view the full document.

View Full Document
Ask a homework question - tutors are online