# Lecture8 - G5205 Linear Regression Lecture 8 Bias-variance...

• Notes
• 10
• 100% (2) 2 out of 2 people found this document helpful

This preview shows page 1 - 3 out of 10 pages.

G5205 Linear Regression Lecture 8 - 11/10/2016 Bias-variance trade-o and cross-validation Lecturer: Arian Maleki Scribe: Arian Maleki 1 What is next? So far we have only considered linear models. Linear models are quite useful for applications. However, in many applications the vanilla linear model we have considered so far does not work well and we have to do some more work. In the next few lectures our goal is to show you how we can evaluate the performance of the linear model and show you some simple remedies that make linear regression work in a wide range of applications. In summary we would like to answer the following questions. 1. How can we evaluate the performance of the current model? Is the model we have used so far good or we should change it? 2. What if the linear model is not good? How should we change it?
2 Simple extensions of linear regression Before I tell you how we would like to evaluate linear models, let me tell you how you can go beyond linear model when/if you know that linear model is not good. We will see some examples how these things can be used in practice. 2.1 Transforming response variables In some cases when we plot our data in terms of certain predictors we observe that the data does not follow a linear trend. For instance, if you remember when we plotted the average price of houses in US as a function of year, then we realized that the price is not growing linearly. In many of these cases transforming the response variable helps. Some transformations you should always keep in mind are ˜ Y i = log Y i , ˜ Y i = Y γ i , ˜ Y i = 1 Y γ i , (1) I suggest you always do some visual inspection to see which transformation is better. However, there are some automated ways that can help you as well. One of these tools is the Box-Cox approach. The Box- Cox approach tries to automatically find the best γ with which you can transform your Y i s (in the Y γ i transformation). In this approach you first model your data in the following way: Suppose that for every γ we can model our data in the following way: Y γ i = β 0 + β 1 X i + i , 1
where i N (0 , σ 2 ), where γ , β 0 , β 1 , and σ 2 are all unknown. Then how do we estimate all the parameters?