APME_3 - Applied Probability Methods for Engineers Slide...

Info icon This preview shows pages 1–14. Sign up to view the full content.

View Full Document Right Arrow Icon
Applied Probability Methods for Engineers Slide Set 3
Image of page 1

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Chapter 12 Simple Linear Regression and Correlation
Image of page 2
Simple Linear Regression Used when we believe some random variable is dependent on some other factor and is a linear function of this factor We consider a random variable Y i that depends on the value of an independent variable x i Assume that an observation y i is the sum of a linear function of x i and an error term ε i , i.e., y i = β 0 + β 1 x i + ε i Generally assumed that the error terms are normally distributed (and iid) with zero mean and variance σ 2 Observations y 1 , …, y n are therefore observations of independent random variables Y i ~ N(β 0 + β 1 x i , σ 2 ) Expected value of Y i equals β 0 + β 1 x i
Image of page 3

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Simple Linear Regression
Image of page 4
Simple Linear Regression y is the dependent variable and x is the explanatory variable β 0 and β 1 are the slope and intercept parameters β 1 slope parameter determines how expected value of y changes as a function of x β 1 = 0 implies y and x are unrelated β 0 , β 1 , and σ 2 typically estimated from a data set Should first look at a graph of the data to make sure linearity is a reasonable assumption
Image of page 5

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Example Data Set Factory electricity usage as a function of production
Image of page 6
Example Data Set
Image of page 7

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Fitting a Regression Line How do we take a data set and fit the best line to it? Consider vertical deviations
Image of page 8
Fitting a Regression Line Given the data set of x i , y i values, we want the β 0 and β 1 values that minimize the sum of squared deviations from the line The vertical error is given by ε i = y i – (β 0 + β 1 x i ) Why not minimize the sum of the errors? Min Σ i ε i = Min Σ i {y i – (β 0 + β 1 x i )} = -Max Σ i 0 + β 1 x i } Since we don’t have sign restrictions on β 0 and β 1 , optimal solution is β 0 = β 1 = (assuming x i ’s nonnegative) This is a meaningless result that essentially interprets negative errors as a good thing
Image of page 9

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Fitting a Regression Line We need to consider negative and positive errors as equally bad Min Σ i ε i 2 = Min Q = Σ i {y i – (β 0 + β 1 x i )} 2 Setting these to zero gives ( 29 ( 29 0 1 1 0 2 n i i i Q y x β β β = = - - + ( 29 ( 29 0 1 1 1 2 n i i i i Q x y x β β β = = - - + 0 1 1 1 n n i i i i y n x β β = = = + 2 0 1 1 1 1 n n n i i i i i i i x y x x β β = = = = +
Image of page 10
Side Note Is Q a convex function of β 0 and β 1 ? Jointly convex in β 0 and β 1 if a ≥ 0, c ≥ 0, and ac – b 2 ≥ 0 2 2 0 2 Q a n β = = 2 2 2 1 1 2 n i i Q c x β = = = 2 1 0 1 2 n i i Q b x β β = = = 2 1 1 0 2 n i i Q b x β β = = = ( 29 ( 29 2 2 2 1 1 1 4 4 4 n n n i i i i i i ac b n x x n x x = = = - = - = -
Image of page 11

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Back to the Fitted Line Solve the two equations simultaneously: 0 1 1 1 n n i i i i y n x β β = = = + 2 0 1 1 1 1 n n n i i i i i i i x y x x β β = = = = + 1 1 0 1 1 ˆ ˆ ˆ n n i i i i y x y x n n β β β = = = - = - ( 29 ( 29 ( 29 1 1 1 1 2 2 1 1 ˆ n n n i i i i i i i n n i i i i n x y x y n x x β = = = = = - = -
Image of page 12
Some Simplifying Notation Fitted regression line: ( 29 ( 29 2 1 2 2 2 2 1 1 1 n i n n n i XX i i i i i i x S x x x nx x n = = = = = - = - = -
Image of page 13

Info icon This preview has intentionally blurred sections. Sign up to view the full version.

View Full Document Right Arrow Icon
Image of page 14
This is the end of the preview. Sign up to access the rest of the document.

{[ snackBarMessage ]}

What students are saying

  • Left Quote Icon

    As a current student on this bumpy collegiate pathway, I stumbled upon Course Hero, where I can find study resources for nearly all my courses, get online help from tutors 24/7, and even share my old projects, papers, and lecture notes with other students.

    Student Picture

    Kiran Temple University Fox School of Business ‘17, Course Hero Intern

  • Left Quote Icon

    I cannot even describe how much Course Hero helped me this summer. It’s truly become something I can always rely on and help me. In the end, I was not only able to survive summer classes, but I was able to thrive thanks to Course Hero.

    Student Picture

    Dana University of Pennsylvania ‘17, Course Hero Intern

  • Left Quote Icon

    The ability to access any university’s resources through Course Hero proved invaluable in my case. I was behind on Tulane coursework and actually used UCLA’s materials to help me move forward and get everything together on time.

    Student Picture

    Jill Tulane University ‘16, Course Hero Intern