Chp7 - Multiple Regression Multiple Regression ANOVA...

Info icon This preview shows pages 1–10. Sign up to view the full content.

View Full Document Right Arrow Icon
Multiple Regression Multiple Regression ANOVA Standardized Regression Multi-collinearity Polynomial Regression Interaction Regression Model Constrained Regression 1
Image of page 1

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Extra sum of squares. decompose SSR Consider the following regression model with two covariates, Y = β 0 + X 1 β 1 + X 2 β 2 + , note the ANOVA as SSTO = SSR ( X 1 , X 2 ) + SSE ( X 1 , X 2 ) , for regression model with just X 1 Y = β 0 + X 1 β 1 + , the ANOVA is SSTO = SSR ( X 1 ) + SSE ( X 1 ) . 2
Image of page 2
Obviously SSE ( X 1 , X 2 ) SSE ( X 1 ) , or equivalently SSR ( X 1 , X 2 ) SSR ( X 1 ) . Define the extra (regression) sum of squares as SSR ( X 2 | X 1 ) = SSR ( X 1 , X 2 ) - SSR ( X 1 ) = SSE ( X 1 ) - SSE ( X 1 , X 2 ) (1) This increase (reduction) in the regression (error) sum of squares is the result of adding X 2 to the regression model when X 1 is already included in the model. Notice that generally SSR ( X 2 | X 1 ) 6 = SSR ( X 1 | X 2 ) , SSR ( X 1 ) 6 = SSR ( X 2 ) . 3
Image of page 3

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
We can similarly define general extra sum of squares SSR ( X S | X F ) = SSR ( X S , X F ) - SSR ( X F ) , (2) where X S , X F are two sets of covariates, e.g. SSR ( X 2 , X 3 | X 1 ) = SSR ( X 1 , X 2 , X 3 ) - SSR ( X 1 ) , SSR ( X 3 | X 1 , X 2 ) = SSR ( X 1 , X 2 , X 3 ) - SSR ( X 1 , X 2 ) . By definition we have SSR ( X 2 , X 3 | X 1 ) = SSR ( X 2 | X 1 ) + SSR ( X 3 | X 1 , X 2 ) , SSR ( X 1 , X 2 , X 3 ) = SSR ( X 1 ) + SSR ( X 2 | X 1 ) + SSR ( X 3 | X 1 , X 2 ) 4
Image of page 4
Extra sum of squares for testing regression coefficients Consider the following general multiple regression model Y = β 0 + p X j =1 X j β j + , to test a single regression coefficient β k H 0 : β k = 0 vs H a : β k 6 = 0 , (3) we can use the t-test t * = b k s ( b k ) t ( n - p - 1) under H 0 . 5
Image of page 5

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Equivalently we can use the following general linear test approach: com- pare the reduced model (R): E ( Y ) = β 0 + X j 6 = k X j β j , to the full model (F): E ( Y ) = β 0 + p X j =1 X j β j , if β k = 0, the two SSE should be very close. We can use the following partial F test statistic F * = SSE ( R ) - SSE ( F ) 1 / SSE ( F ) n - p - 1 = SSR ( X k | X - k ) SSE ( X 1 , · · · , X p ) / ( n - p - 1) F (1 , n - p - 1) under H 0 . 6
Image of page 6
We can show that F * = t * 2 , so the two tests are equivalent. The idea can be carried over to test for several regression coefficients being zero H 0 : β i = 0 , i S, H a : i S, β i 6 = 0 , (4) where S is a collection of integers, e.g. for S = 1 , 2 , 3, we’re testing β 1 , β 2 and β 3 being zero simultaneously. The partial F test statistic being F * = SSR ( X S | X - S ) / #( S ) SSE ( X 1 , · · · , X p ) / ( n - p - 1) F (#( S ) , n - p - 1) under H 0 , where #( S ) equals the number of elements in set S. 7
Image of page 7

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
We can also write the regression sum of squares for the full model in an extra sum of squares notation as SSR = SSR ( X 1 , · · · , X p | 1) , which is the extra variation explained by adding all covariates in the model. And hence we have the following F test F = SSR/p SSE/ ( n - p - 1) for testing all β k being zero. We can also test for linear equations of some regression coefficients, e.g. H 0 : β 1 = β 2 vs H a : β 1 6 = β 2 , 8
Image of page 8
we can compare the SSE of the following two regression models E ( Y ) = β 0 + X j X j β j vs E ( Y ) = β 0 + β + ( X 1 + X 2 ) + p X j =3 X j β j , note that we can equivalently write the full model as E ( Y ) = β 0 + β + ( X 1 + X 2 ) + β - ( X 1 - X 2 ) + p X j =3 X j β j , so the testing transforms to the extra sum of squares framework.
Image of page 9

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 10
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern