COS424SML 302 Linear Regression 8 46 Linear regression as a

Cos424sml 302 linear regression 8 46 linear

• Notes
• 46

This preview shows page 8 - 19 out of 46 pages.

COS424/SML 302 Linear Regression February 25, 2019 8 / 46 Subscribe to view the full document.

Linear regression as a discriminative model Gaussian linear regression A Gaussian linear regression model has the form: y i = x i β + i where i ∼ N (0 , σ 2 ). Y X n Writing this out in terms of the graphical model, we have: p ( x , y ) = p ( x ) p ( y | x ) . We know the conditional distribution of y given x is Gaussian: y i | x i , β, σ 2 ∼ N ( x i β, σ 2 ) . We have not specified the distribution of x ; does this matter? COS424/SML 302 Linear Regression February 25, 2019 9 / 46 Model fitting and prediction Fitting a linear regression model With our model y i = x i β + i and training data D = { ( x 1 , y 1 ) , . . . , ( x n , y n ) } , we can estimate parameters β and σ 2 , giving us ˆ β (coefficient) and ˆ σ 2 (residual variance). Prediction for a new point x * We can predict y * for a new point x * , given ˆ β : ˆ y * = E [ y * | x * , ˆ β ] = ˆ β x * . This is not quite right. What else do we need to specify an arbitrary linear relationship between two variables? COS424/SML 302 Linear Regression February 25, 2019 10 / 46 Subscribe to view the full document.

Adding the intercept term With our current model, when X Y Y X Y What if this is not true in our data (e.g., height at age 0)? COS424/SML 302 Linear Regression February 25, 2019 11 / 46 Adding the intercept term Let’s add a y -axis intercept term, β 0 . y i = β 0 + β x i + i We can do this in the laziest way possible: add a 1 as the first element of each x i vector, and a β 0 term as the first element of a β vector: x i = [1 , x i ] T ˜ β = [ β 0 , β ] T Our model is then: y i = β 0 + β x i + i = x i T ˜ β + i . X Y COS424/SML 302 Linear Regression February 25, 2019 12 / 46 Subscribe to view the full document.

Linear regression: conditional distribution We have defined the conditional probability of the response y : y i | x i , β, σ 2 ∼ N ( β 0 + x i β, σ 2 ) . X Y Y Y At every value of x i , the response y i has a conditionally Gaussian distribution with mean β 0 + β x i . COS424/SML 302 Linear Regression February 25, 2019 13 / 46 Linear regression: prediction We predict the value of y * for x * with the conditional Gaussian mean: y * = β 0 + β x * . X Y y * x * COS424/SML 302 Linear Regression February 25, 2019 14 / 46 Subscribe to view the full document.

Example: predicting height using shoe size Let’s use our survey data to step through linear regression. Let y = height (in) and x = shoe size (cm). 60 65 70 75 80 85 22.5 25.0 27.5 Foot size (cm) Height (inches) Does it look like there is a linear relationship between x and y ? COS424/SML 302 Linear Regression February 25, 2019 15 / 46 Example: predicting height using shoe size We fit a linear model to these data, and find that ˆ β 0 = 25 . 39, ˆ β = 1 . 65: 60 65 70 75 80 85 22.5 25.0 27.5 Foot size (cm) Height (inches) Do these parameter estimates support a hypothesis about a linear relationship existing between x and y ? COS424/SML 302 Linear Regression February 25, 2019 16 / 46 Subscribe to view the full document.

Example: predicting height using number of siblings We fit a linear model to these data, and find that ˆ β 0 = 68 . 0, ˆ β = 0 . 4: 60 65 70 75 80 85 0 2 4 6 Number of Siblings Height (inches) Do these parameter estimates support a hypothesis about a linear relationship existing between x and y ? COS424/SML 302 Linear Regression February 25, 2019 17 / 46 Example: predicting height using shoe size We fit a linear model to D with no intercept term, and find that ˆ β = 2 . 62 versus ˆ β = 1 . 65 60 65 70 75 80 85 22.5 25.0 27.5 Foot size (cm) Height (inches) How can we determine how much worse this fit is relative to the model with an intercept term? Subscribe to view the full document. • Spring '09

What students are saying

• As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

Kiran Temple University Fox School of Business ‘17, Course Hero Intern

• I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

Dana University of Pennsylvania ‘17, Course Hero Intern

• The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask 0 bonus questions You can ask 0 questions (0 expire soon) You can ask 0 questions (will expire )
Answers in as fast as 15 minutes