COS424SML 302 Linear Regression 8 46 Linear regression as a

Cos424sml 302 linear regression 8 46 linear

This preview shows page 8 - 19 out of 46 pages.

COS424/SML 302 Linear Regression February 25, 2019 8 / 46
Image of page 8

Subscribe to view the full document.

Linear regression as a discriminative model Gaussian linear regression A Gaussian linear regression model has the form: y i = x i β + i where i ∼ N (0 , σ 2 ). Y X n Writing this out in terms of the graphical model, we have: p ( x , y ) = p ( x ) p ( y | x ) . We know the conditional distribution of y given x is Gaussian: y i | x i , β, σ 2 ∼ N ( x i β, σ 2 ) . We have not specified the distribution of x ; does this matter? COS424/SML 302 Linear Regression February 25, 2019 9 / 46
Image of page 9
Model fitting and prediction Fitting a linear regression model With our model y i = x i β + i and training data D = { ( x 1 , y 1 ) , . . . , ( x n , y n ) } , we can estimate parameters β and σ 2 , giving us ˆ β (coefficient) and ˆ σ 2 (residual variance). Prediction for a new point x * We can predict y * for a new point x * , given ˆ β : ˆ y * = E [ y * | x * , ˆ β ] = ˆ β x * . This is not quite right. What else do we need to specify an arbitrary linear relationship between two variables? COS424/SML 302 Linear Regression February 25, 2019 10 / 46
Image of page 10

Subscribe to view the full document.

Adding the intercept term With our current model, when X Y Y X Y What if this is not true in our data (e.g., height at age 0)? COS424/SML 302 Linear Regression February 25, 2019 11 / 46
Image of page 11
Adding the intercept term Let’s add a y -axis intercept term, β 0 . y i = β 0 + β x i + i We can do this in the laziest way possible: add a 1 as the first element of each x i vector, and a β 0 term as the first element of a β vector: x i = [1 , x i ] T ˜ β = [ β 0 , β ] T Our model is then: y i = β 0 + β x i + i = x i T ˜ β + i . X Y COS424/SML 302 Linear Regression February 25, 2019 12 / 46
Image of page 12

Subscribe to view the full document.

Linear regression: conditional distribution We have defined the conditional probability of the response y : y i | x i , β, σ 2 ∼ N ( β 0 + x i β, σ 2 ) . X Y Y Y At every value of x i , the response y i has a conditionally Gaussian distribution with mean β 0 + β x i . COS424/SML 302 Linear Regression February 25, 2019 13 / 46
Image of page 13
Linear regression: prediction We predict the value of y * for x * with the conditional Gaussian mean: y * = β 0 + β x * . X Y y * x * COS424/SML 302 Linear Regression February 25, 2019 14 / 46
Image of page 14

Subscribe to view the full document.

Example: predicting height using shoe size Let’s use our survey data to step through linear regression. Let y = height (in) and x = shoe size (cm). 60 65 70 75 80 85 22.5 25.0 27.5 Foot size (cm) Height (inches) Does it look like there is a linear relationship between x and y ? COS424/SML 302 Linear Regression February 25, 2019 15 / 46
Image of page 15
Example: predicting height using shoe size We fit a linear model to these data, and find that ˆ β 0 = 25 . 39, ˆ β = 1 . 65: 60 65 70 75 80 85 22.5 25.0 27.5 Foot size (cm) Height (inches) Do these parameter estimates support a hypothesis about a linear relationship existing between x and y ? COS424/SML 302 Linear Regression February 25, 2019 16 / 46
Image of page 16

Subscribe to view the full document.

Example: predicting height using number of siblings We fit a linear model to these data, and find that ˆ β 0 = 68 . 0, ˆ β = 0 . 4: 60 65 70 75 80 85 0 2 4 6 Number of Siblings Height (inches) Do these parameter estimates support a hypothesis about a linear relationship existing between x and y ? COS424/SML 302 Linear Regression February 25, 2019 17 / 46
Image of page 17
Example: predicting height using shoe size We fit a linear model to D with no intercept term, and find that ˆ β = 2 . 62 versus ˆ β = 1 . 65 60 65 70 75 80 85 22.5 25.0 27.5 Foot size (cm) Height (inches) How can we determine how much worse this fit is relative to the model with an intercept term?
Image of page 18

Subscribe to view the full document.

Image of page 19

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern

Ask Expert Tutors You can ask 0 bonus questions You can ask 0 questions (0 expire soon) You can ask 0 questions (will expire )
Answers in as fast as 15 minutes