ECE592-074_LECTURE_10.pdf - Multiple Linear Regression...

This preview shows page 1 - 3 out of 5 pages.

10/6/2020 Multiple Linear Regression 1/5 Multiple Linear Regression Wenyuan Tang Lecture 10 Multiple linear regression The multiple linear regression model takes the form where represents the th predictor and quantifies the association between that variable and the response. We interpret as the average effect on of a one unit increase in , holding all other predicators fixed . Given the training data , define Again, we use the least squares approach to estimate the coefficients: Solving the optimization problem we obtain the least squares coefficient estimates Since the RSS always decreases as more variables are added to the model, the always increases as more variables are added. If we use the to select the best model, we will always end up with a model involving all of the variables (overfitting). The problem is that a high indicates a model with a low training error, whereas we wish to choose a model that has a low test error. Therefore, we introduce the adjusted statistic: The adjusted is always smaller than . The adjusted can be negative. The adjusted increases only when the decrease in RSS (due to the inclusion of a new variable) is more than one would expect to see by chance. In theory, the model with the largest adjusted will have only correct variables and no noise variables.
10/6/2020 Multiple Linear Regression 2/5 Extensions of the linear model The assumptions in the standard linear regression model can be relaxed: Removing the additive assumption. Consider the standard linear regression model with two variables: In addition to and which represent the main effects , we can extend the model by including an interaction term which represent the interaction effect : It is sometimes the case that an interaction term has a very small -value, but the associated main effects do not. The hierarchical principle states that if we include an interaction in a model, we should also include the main effects, even if the -values associated with their coefficients are not significant . Removing the linear assumption. We can extend the linear model to accommodate nonlinear relationships through transformations of quantitative inputs, such as log, square-root, or square. For example, we can capture a quadratic relationship by using polynomial regression : which is still a linear model by considering and .

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture

  • Left Quote Icon

    Student Picture