# Chp7 - Multiple Regression Multiple Regression ANOVA...

This preview shows pages 1–10. Sign up to view the full content.

Multiple Regression Multiple Regression ANOVA Standardized Regression Multi-collinearity Polynomial Regression Interaction Regression Model Constrained Regression 1

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Extra sum of squares. decompose SSR Consider the following regression model with two covariates, Y = β 0 + X 1 β 1 + X 2 β 2 + , note the ANOVA as SSTO = SSR ( X 1 , X 2 ) + SSE ( X 1 , X 2 ) , for regression model with just X 1 Y = β 0 + X 1 β 1 + , the ANOVA is SSTO = SSR ( X 1 ) + SSE ( X 1 ) . 2
Obviously SSE ( X 1 , X 2 ) SSE ( X 1 ) , or equivalently SSR ( X 1 , X 2 ) SSR ( X 1 ) . Define the extra (regression) sum of squares as SSR ( X 2 | X 1 ) = SSR ( X 1 , X 2 ) - SSR ( X 1 ) = SSE ( X 1 ) - SSE ( X 1 , X 2 ) (1) This increase (reduction) in the regression (error) sum of squares is the result of adding X 2 to the regression model when X 1 is already included in the model. Notice that generally SSR ( X 2 | X 1 ) 6 = SSR ( X 1 | X 2 ) , SSR ( X 1 ) 6 = SSR ( X 2 ) . 3

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
We can similarly define general extra sum of squares SSR ( X S | X F ) = SSR ( X S , X F ) - SSR ( X F ) , (2) where X S , X F are two sets of covariates, e.g. SSR ( X 2 , X 3 | X 1 ) = SSR ( X 1 , X 2 , X 3 ) - SSR ( X 1 ) , SSR ( X 3 | X 1 , X 2 ) = SSR ( X 1 , X 2 , X 3 ) - SSR ( X 1 , X 2 ) . By definition we have SSR ( X 2 , X 3 | X 1 ) = SSR ( X 2 | X 1 ) + SSR ( X 3 | X 1 , X 2 ) , SSR ( X 1 , X 2 , X 3 ) = SSR ( X 1 ) + SSR ( X 2 | X 1 ) + SSR ( X 3 | X 1 , X 2 ) 4
Extra sum of squares for testing regression coefficients Consider the following general multiple regression model Y = β 0 + p X j =1 X j β j + , to test a single regression coefficient β k H 0 : β k = 0 vs H a : β k 6 = 0 , (3) we can use the t-test t * = b k s ( b k ) t ( n - p - 1) under H 0 . 5

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
Equivalently we can use the following general linear test approach: com- pare the reduced model (R): E ( Y ) = β 0 + X j 6 = k X j β j , to the full model (F): E ( Y ) = β 0 + p X j =1 X j β j , if β k = 0, the two SSE should be very close. We can use the following partial F test statistic F * = SSE ( R ) - SSE ( F ) 1 / SSE ( F ) n - p - 1 = SSR ( X k | X - k ) SSE ( X 1 , · · · , X p ) / ( n - p - 1) F (1 , n - p - 1) under H 0 . 6
We can show that F * = t * 2 , so the two tests are equivalent. The idea can be carried over to test for several regression coefficients being zero H 0 : β i = 0 , i S, H a : i S, β i 6 = 0 , (4) where S is a collection of integers, e.g. for S = 1 , 2 , 3, we’re testing β 1 , β 2 and β 3 being zero simultaneously. The partial F test statistic being F * = SSR ( X S | X - S ) / #( S ) SSE ( X 1 , · · · , X p ) / ( n - p - 1) F (#( S ) , n - p - 1) under H 0 , where #( S ) equals the number of elements in set S. 7

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
We can also write the regression sum of squares for the full model in an extra sum of squares notation as SSR = SSR ( X 1 , · · · , X p | 1) , which is the extra variation explained by adding all covariates in the model. And hence we have the following F test F = SSR/p SSE/ ( n - p - 1) for testing all β k being zero. We can also test for linear equations of some regression coefficients, e.g. H 0 : β 1 = β 2 vs H a : β 1 6 = β 2 , 8
we can compare the SSE of the following two regression models E ( Y ) = β 0 + X j X j β j vs E ( Y ) = β 0 + β + ( X 1 + X 2 ) + p X j =3 X j β j , note that we can equivalently write the full model as E ( Y ) = β 0 + β + ( X 1 + X 2 ) + β - ( X 1 - X 2 ) + p X j =3 X j β j , so the testing transforms to the extra sum of squares framework.

This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

### What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern